How to decouple dedicate index storage by project?

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
2.2.0, running on VMs

Describe the issue:
Hi ,
I push all applicative logs on 3 Opensearch nodes
Everything is working perfectly fine for 1 year now.

We would need to push logs from our APIC Tools : but its 10 X more logs than the applicative logs.

We would like to be able to make these logs available in the same Opensearch Instance
But without compromising our applicative indexes.

We would like to set the shards routing properly to fullfill our needs
And we wonder how we can implement it properly.

In other words we wonder how we can set Opensearch Worker nodes to store exclusively

  • applicative shards ( master / replicas )
  • APIC shards ( master / replicas )

Is is possible to decouple Worker nodes

  • some nodes would be dedicated to Applicative shards storing
  • the others nodes would be dedicated to APIC shards storing

Should I set a different rack for each worker node in conf ( applicative rack or apic rack ) as follow :
node.attr.rack_id: rack_applicative
or
node.attr.rack_id: rack_apic

And then set explicitly for any kind of indexes where he should be stored ?
as follow :
to force the index routing for a test index to be stored on applicative rack nodes for example :
PUT test/_settings
{
ā€œindex.routing.allocation.require.rackā€: ā€œrack_applicativeā€
}

Is it a good solution ? is there an other way ?

Thank you for support

2

This is basically the only way to specifically assign shards to specific nodes.

1 Like