RAM usage is too high

OpenSearch
,
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):OpenSearch-2.18.0

Describe the issue:

We are using standalone opensaerch-2.18.0(tar file) in RHEL OS for indexing .

We have set heap allocation to this

-Xms1g
-Xmx1g

but memory utilization is quite high (1.9GB) even our document size in index is below 50 MB

Configuration:

OS-RHEL-9.4

RAM-16 GB

CPU-8

Relevant Logs or Screenshots: Out put of top command

Tasks: 297 total, 1 running, 296 sleeping, 0 stopped, 0 zombie
%Cpu(s): 1.0 us, 0.4 sy, 0.0 ni, 98.4 id, 0.1 wa, 0.1 hi, 0.0 si, 0.0 st
MiB Mem : 15578.7 total, 1621.2 free, 10672.4 used, 5621.6 buff/cache
MiB Swap: 8060.0 total, 8043.1 free, 16.9 used. 4906.3 avail Mem

PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND

4071 testuser 20 0 7373772 1.9g 32480 S 0.7 12.8 10:45.72 java

2

@bimlesh_singh What’s the reason for setting a 1GB Java heap size when you have 16GB of RAM?
OpenSearch uses that space for many operations, which also include indexing.

3

Hi Pablo, Thanks for reply. This is onPrem setup. So there are other services running on same server. Thats why kept memory size to small(1gb) as data size is also small . but when we start using API like- _plugins/_sql to fetch data then Stack overflow error being thrown and OpenSearch service crashes for document size around 300000 documents and takes around 15 seconds to process the below query.

{ “query”:SELECT attributes.Employee Specification.First_Name_New AS COLUMN1 , COUNT(*) AS COUNT FROM 23 WHERE identifier IN (‘391938’,‘112233’) AND (attributes.Employee Specification.ID IS NOT NULL) GROUP BY attributes.Employee Specification.First_Name_New HAVING attributes.Employee Specification.First_Name_New IS NOT NULL ORDER BY COUNT DESC"}

Please suggest if there are any fine-tuning settings needed for such queries

4

@bimlesh_singh As I mentioned before, Java heap size is used for many ither tasks executed by OpenSearch service. This also includes an index management, queries etc. Try increase the Java heap size.
Do you monitor Java heap size utilization? Do you see any errors or messages regarding Java hep size or garbage collection in OpenSearch logs?