Configure Hot warm and cold architecture in an existing openserch cluster

[narendrareddy.ala]

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
opensearch: 2.3.0

Describe the issue:

I want to configure H/W/C on my existing cluster, I have configured k8 cluster using opster openserch operator, unfortunately I couldn’t find any helpful documentation on how to configure hot, warm and clod architecture on opensearch.

Configuration:

#Minimal configuration of a cluster with version 2.X of the operator.
#Note the replacement of ‘master’ role with ‘cluster_manager’ on line 49
apiVersion: opensearch.opster.io/v1
kind: OpenSearchCluster
metadata:
name: opensearch
namespace: default
spec:
security:
config:
tls:
http:
generate: true
transport:
generate: true
perNode: true
general:
httpPort: 9200
serviceName: opensearch
version: 2.3.0
pluginsList: [“repository-s3”]
drainDataNodes: true
dashboards:
tls:
enable: true
generate: true
version: 2.3.0
enable: true
replicas: 1
resources:
requests:
memory: “1Gi”
cpu: “1000m”
limits:
memory: “1Gi”
cpu: “1000m”
nodePools:
- component: masters
replicas: 3
diskSize: “10Gi”
resources:
requests:
memory: “2Gi”
cpu: “1000m”
limits:
memory: “2Gi”
cpu: “1000m”
roles:
- “cluster_manager”
- component: nodes
replicas: 3
diskSize: “100Gi”
resources:
requests:
memory: “2Gi”
cpu: “1000m”
limits:
memory: “2Gi”
cpu: “1000m”
roles:
- “data”
- component: coordinators
replicas: 1
diskSize: “10Gi”
resources:
requests:
memory: “2Gi”
cpu: “1000m”
limits:
memory: “2Gi”
cpu: “1000m”
roles:
- “ingest”

Relevant Logs or Screenshots:

[vi4life]

Hi @narendrareddy.ala,
you need to configure the Variable “node.attr.temp” and set it to hot, warm or cold.

Cluster formation

Then you can configure the Index Template and the ISM to move the Shards to the OpenSearch Nodes that have these Attributes.

[narendrareddy.ala]

@vi4life

node.attr.temp

Thanks for your reply, I understood on how to apply this on a classic cluster, however I am trying to figure out on how I can add these changes in my k8 config file.
I couldnt find any references to make this change

[narendrareddy.ala]

#Minimal configuration of a cluster with version 2.X of the operator.
#Note the replacement of 'master' role with 'cluster_manager' on line 49
apiVersion: opensearch.opster.io/v1
kind: OpenSearchCluster
metadata:
  name: opensearch-admin-stage
  namespace: default
spec:
  security:
    config:
    tls:
       http:
         generate: true
       transport:
         generate: true
         perNode: true
  general:
    httpPort: 9200
    serviceName: opensearch-admin-stage
    version: 2.3.0
    pluginsList: ["repository-s3"]
    drainDataNodes: true
  dashboards:
    tls:
      enable: true
      generate: true
    version: 2.3.0
    enable: true
    replicas: 1
    resources:
      requests:
         memory: "1Gi"
         cpu: "1000m"
      limits:
         memory: "1Gi"
         cpu: "1000m"
  nodePools:
    - component: masters
      replicas: 3
      diskSize: "10Gi"
      resources:
         requests:
            memory: "2Gi"
            cpu: "1000m"
         limits:
            memory: "2Gi"
            cpu: "1000m"
      roles:
        - "cluster_manager"
    - component: hot_nodes
      replicas: 3
      diskSize: "100Gi"
      resources:
         requests:
            memory: "2Gi"
            cpu: "1000m"
         limits:
            memory: "2Gi"
            cpu: "1000m"
      node.attr.temp: hot
      roles:
        - "data"
    - component: warm_nodes
      replicas: 3
      diskSize: "100Gi"
      resources:
         requests:
            memory: "2Gi"
            cpu: "1000m"
         limits:
            memory: "2Gi"
            cpu: "1000m"
      node.attr.temp: warm
      roles:
        - "data"
    - component: cold_nodes
      replicas: 3
      diskSize: "100Gi"
      resources:
         requests:
            memory: "2Gi"
            cpu: "1000m"
         limits:
            memory: "2Gi"
            cpu: "1000m"
      node.attr.temp: cold
      roles:
        - "data"
    - component: coordinators
      replicas: 1
      diskSize: "10Gi"
      resources:
         requests:
            memory: "2Gi"
            cpu: "1000m"
         limits:
            memory: "2Gi"
            cpu: "1000m"
      roles:
        - "ingest"

I have tried above changes and got the below error

The request is invalid: patch: Invalid value: "{\"apiVersion\":\"opensearch.opster.io/v1\",\"kind\":\"OpenSearchCluster\",\"metadata\":{\"annotations\":{\"kubectl.kubernetes.io/last-applied-configuration\":\"{\\\"apiVersion\\\":\\\"opensearch.opster.io/v1\\\",\\\"kind\\\":\\\"OpenSearchCluster\\\",\\\"metadata\\\":{\\\"annotations\\\":{},\\\"name\\\":\\\"opensearch-admin-stage\\\",\\\"namespace\\\":\\\"default\\\"},\\\"spec\\\":{\\\"dashboards\\\":{\\\"enable\\\":true,\\\"replicas\\\":1,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"1000m\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"1000m\\\",\\\"memory\\\":\\\"1Gi\\\"}},\\\"tls\\\":{\\\"enable\\\":true,\\\"generate\\\":true},\\\"version\\\":\\\"2.3.0\\\"},\\\"general\\\":{\\\"drainDataNodes\\\":true,\\\"httpPort\\\":9200,\\\"pluginsList\\\":[\\\"repository-s3\\\"],\\\"serviceName\\\":\\\"opensearch-admin-stage\\\",\\\"version\\\":\\\"2.3.0\\\"},\\\"nodePools\\\":[{\\\"component\\\":\\\"masters\\\",\\\"diskSize\\\":\\\"10Gi\\\",\\\"replicas\\\":3,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"1000m\\\",\\\"memory\\\":\\\"2Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"1000m\\\",\\\"memory\\\":\\\"2Gi\\\"}},\\\"roles\\\":[\\\"cluster_manager\\\"]},{\\\"component\\\":\\\"hot_nodes\\\",\\\"diskSize\\\":\\\"100Gi\\\",\\\"node.attr.temp\\\":\\\"hot\\\",\\\"replicas\\\":3,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"1000m\\\",\\\"memory\\\":\\\"2Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"1000m\\\",\\\"memory\\\":\\\"2Gi\\\"}},\\\"roles\\\":[\\\"data\\\"]},{\\\"component\\\":\\\"warm_nodes\\\",\\\"diskSize\\\":\\\"100Gi\\\",\\\"node.attr.temp\\\":\\\"warm\\\",\\\"replicas\\\":3,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"1000m\\\",\\\"memory\\\":\\\"2Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"1000m\\\",\\\"memory\\\":\\\"2Gi\\\"}},\\\"roles\\\":[\\\"data\\\"]},{\\\"component\\\":\\\"cold_nodes\\\",\\\"diskSize\\\":\\\"100Gi\\\",\\\"node.attr.temp\\\":\\\"cold\\\",\\\"replicas\\\":3,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"1000m\\\",\\\"memory\\\":\\\"2Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"1000m\\\",\\\"memory\\\":\\\"2Gi\\\"}},\\\"roles\\\":[\\\"data\\\"]},{\\\"component\\\":\\\"coordinators\\\",\\\"diskSize\\\":\\\"10Gi\\\",\\\"replicas\\\":1,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"1000m\\\",\\\"memory\\\":\\\"2Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"1000m\\\",\\\"memory\\\":\\\"2Gi\\\"}},\\\"roles\\\":[\\\"ingest\\\"]}],\\\"security\\\":{\\\"config\\\":null,\\\"tls\\\":{\\\"http\\\":{\\\"generate\\\":true},\\\"transport\\\":{\\\"generate\\\":true,\\\"perNode\\\":true}}}}}\\n\"},\"creationTimestamp\":\"2023-01-19T13:21:03Z\",\"finalizers\":[\"Opster\"],\"generation\":2,\"managedFields\":[{\"apiVersion\":\"opensearch.opster.io/v1\",\"fieldsType\":\"FieldsV1\",\"fieldsV1\":{\"f:metadata\":{\"f:annotations\":{\".\":{},\"f:kubectl.kubernetes.io/last-applied-configuration\":{}}},\"f:spec\":{\".\":{},\"f:dashboards\":{\".\":{},\"f:enable\":{},\"f:replicas\":{},\"f:resources\":{\".\":{},\"f:limits\":{\".\":{},\"f:memory\":{}},\"f:requests\":{\".\":{},\"f:memory\":{}}},\"f:tls\":{\".\":{},\"f:enable\":{},\"f:generate\":{}},\"f:version\":{}},\"f:general\":{\".\":{},\"f:drainDataNodes\":{},\"f:httpPort\":{},\"f:pluginsList\":{},\"f:serviceName\":{},\"f:version\":{}},\"f:security\":{\".\":{},\"f:tls\":{\".\":{},\"f:http\":{\".\":{},\"f:generate\":{}},\"f:transport\":{\".\":{},\"f:generate\":{},\"f:perNode\":{}}}}}},\"manager\":\"kubectl-client-side-apply\",\"operation\":\"Update\",\"time\":\"2023-01-19T13:21:03Z\"},{\"apiVersion\":\"opensearch.opster.io/v1\",\"fieldsType\":\"FieldsV1\",\"fieldsV1\":{\"f:metadata\":{\"f:finalizers\":{\".\":{},\"v:\\\"Opster\\\"\":{}}},\"f:spec\":{\"f:bootstrap\":{\".\":{},\"f:resources\":{}},\"f:confMgmt\":{},\"f:dashboards\":{\"f:opensearchCredentialsSecret\":{},\"f:resources\":{\"f:limits\":{\"f:cpu\":{}},\"f:requests\":{\"f:cpu\":{}}},\"f:service\":{\".\":{},\"f:type\":{}},\"f:tls\":{\"f:caSecret\":{},\"f:secret\":{}}},\"f:initHelper\":{},\"f:nodePools\":{},\"f:security\":{\"f:tls\":{\"f:http\":{\"f:caSecret\":{},\"f:secret\":{}},\"f:transport\":{\"f:caSecret\":{},\"f:secret\":{}}}}}},\"manager\":\"manager\",\"operation\":\"Update\",\"time\":\"2023-01-19T13:21:03Z\"},{\"apiVersion\":\"opensearch.opster.io/v1\",\"fieldsType\":\"FieldsV1\",\"fieldsV1\":{\"f:status\":{\".\":{},\"f:componentsStatus\":{},\"f:initialized\":{},\"f:phase\":{},\"f:version\":{}}},\"manager\":\"manager\",\"operation\":\"Update\",\"subresource\":\"status\",\"time\":\"2023-01-19T13:32:18Z\"}],\"name\":\"opensearch-admin-stage\",\"namespace\":\"default\",\"resourceVersion\":\"25076\",\"uid\":\"130bceca-ecd9-430d-a3bc-d0c774753f89\"},\"spec\":{\"bootstrap\":{\"resources\":{}},\"confMgmt\":{},\"dashboards\":{\"enable\":true,\"opensearchCredentialsSecret\":{},\"replicas\":1,\"resources\":{\"limits\":{\"cpu\":\"1000m\",\"memory\":\"1Gi\"},\"requests\":{\"cpu\":\"1000m\",\"memory\":\"1Gi\"}},\"service\":{\"type\":\"ClusterIP\"},\"tls\":{\"caSecret\":{},\"enable\":true,\"generate\":true,\"secret\":{}},\"version\":\"2.3.0\"},\"general\":{\"drainDataNodes\":true,\"httpPort\":9200,\"pluginsList\":[\"repository-s3\"],\"serviceName\":\"opensearch-admin-stage\",\"version\":\"2.3.0\"},\"initHelper\":{},\"nodePools\":[{\"component\":\"masters\",\"diskSize\":\"10Gi\",\"replicas\":3,\"resources\":{\"limits\":{\"cpu\":\"1000m\",\"memory\":\"2Gi\"},\"requests\":{\"cpu\":\"1000m\",\"memory\":\"2Gi\"}},\"roles\":[\"cluster_manager\"]},{\"component\":\"hot_nodes\",\"diskSize\":\"100Gi\",\"node.attr.temp\":\"hot\",\"replicas\":3,\"resources\":{\"limits\":{\"cpu\":\"1000m\",\"memory\":\"2Gi\"},\"requests\":{\"cpu\":\"1000m\",\"memory\":\"2Gi\"}},\"roles\":[\"data\"]},{\"component\":\"warm_nodes\",\"diskSize\":\"100Gi\",\"node.attr.temp\":\"warm\",\"replicas\":3,\"resources\":{\"limits\":{\"cpu\":\"1000m\",\"memory\":\"2Gi\"},\"requests\":{\"cpu\":\"1000m\",\"memory\":\"2Gi\"}},\"roles\":[\"data\"]},{\"component\":\"cold_nodes\",\"diskSize\":\"100Gi\",\"node.attr.temp\":\"cold\",\"replicas\":3,\"resources\":{\"limits\":{\"cpu\":\"1000m\",\"memory\":\"2Gi\"},\"requests\":{\"cpu\":\"1000m\",\"memory\":\"2Gi\"}},\"roles\":[\"data\"]},{\"component\":\"coordinators\",\"diskSize\":\"10Gi\",\"replicas\":1,\"resources\":{\"limits\":{\"cpu\":\"1000m\",\"memory\":\"2Gi\"},\"requests\":{\"cpu\":\"1000m\",\"memory\":\"2Gi\"}},\"roles\":[\"ingest\"]}],\"security\":{\"tls\":{\"http\":{\"caSecret\":{},\"generate\":true,\"secret\":{}},\"transport\":{\"caSecret\":{},\"generate\":true,\"perNode\":true,\"secret\":{}}}}},\"status\":{\"componentsStatus\":[{}],\"initialized\":true,\"phase\":\"RUNNING\",\"version\":\"2.3.0\"}}": strict decoding error: unknown field "spec.nodePools[1].node.attr.temp", unknown field "spec.nodePools[2].node.attr.temp", unknown field "spec.nodePools[3].node.attr.temp"

[madhukarmmallia]

Hi @narendrareddy.ala ,

You could use the additionalConfig field available for nodepool components in the operator.

#Minimal configuration of a cluster with version 2.X of the operator.
#Note the replacement of 'master' role with 'cluster_manager' on line 49
apiVersion: opensearch.opster.io/v1
kind: OpenSearchCluster
metadata:
  name: opensearch-admin-stage
  namespace: default
spec:
  security:
    config:
    tls:
       http:
         generate: true
       transport:
         generate: true
         perNode: true
  general:
    httpPort: 9200
    serviceName: opensearch-admin-stage
    version: 2.3.0
    pluginsList: ["repository-s3"]
    drainDataNodes: true
  dashboards:
    tls:
      enable: true
      generate: true
    version: 2.3.0
    enable: true
    replicas: 1
    resources:
      requests:
         memory: "1Gi"
         cpu: "1000m"
      limits:
         memory: "1Gi"
         cpu: "1000m"
  nodePools:
    - component: masters
      replicas: 3
      diskSize: "10Gi"
      resources:
         requests:
            memory: "2Gi"
            cpu: "1000m"
         limits:
            memory: "2Gi"
            cpu: "1000m"
      roles:
        - "cluster_manager"
    - component: hot_nodes
      replicas: 3
      diskSize: "100Gi"
      resources:
         requests:
            memory: "2Gi"
            cpu: "1000m"
         limits:
            memory: "2Gi"
            cpu: "1000m"
      additionalConfig:
        node.attr.temp: hot
      roles:
        - "data"
    - component: warm_nodes
      replicas: 3
      diskSize: "100Gi"
      resources:
         requests:
            memory: "2Gi"
            cpu: "1000m"
         limits:
            memory: "2Gi"
            cpu: "1000m"
      additionalConfig:
        node.attr.temp: warm
      roles:
        - "data"
    - component: cold_nodes
      replicas: 3
      diskSize: "100Gi"
      resources:
         requests:
            memory: "2Gi"
            cpu: "1000m"
         limits:
            memory: "2Gi"
            cpu: "1000m"
      additionalConfig:
        node.attr.temp: cold
      roles:
        - "data"
    - component: coordinators
      replicas: 1
      diskSize: "10Gi"
      resources:
         requests:
            memory: "2Gi"
            cpu: "1000m"
         limits:
            memory: "2Gi"
            cpu: "1000m"
      roles:
        - "ingest"

I hope this helps.