Opensearch Startup ERROR [o.o.p.c.e.EventLogFileHandler] [node-1] Error writing entry 'NOT_INITIALIZED'

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 2.9.0

Describe the issue: Start the single node opensearch1.3.1.tar version node-2 first, and then add a data node nodes-1 of opensearch2.9.0.rpm version. It was found that the data node node-1 reported an error. Please refer to the errorlog for details

[2023-11-07T17:05:46,005][ERROR][o.o.p.c.e.EventLogFileHandler] [node-1] Error writing entry ‘NOT_INITIALIZED’. Cause:
java.nio.file.AccessDeniedException: /dev/shm/performanceanalyzer/1699347945000.tmp
at sun.nio.fs.UnixException.translateToIOException( ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException( ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException( ~[?:?]
at sun.nio.fs.UnixFileSystemProvider.newByteChannel( ~[?:?]
at java.nio.file.spi.FileSystemProvider.newOutputStream( ~[?:?]
at java.nio.file.Files.newOutputStream( ~[?:?]
at org.opensearch.performanceanalyzer.commons.event_process.EventLogFileHandler.writeTmpFileWithPrivilege( [performance-analyzer-commons-1.0.0.jar:?]
at org.opensearch.performanceanalyzer.commons.event_process.EventLogFileHandler.lambda$writeTmpFile$0( [performance-analyzer-commons-1.0.0.jar:?]
at org.opensearch.performanceanalyzer.commons.util.Util.lambda$invokePrivileged$1( [performance-analyzer-commons-1.0.0.jar:?]
at [?:?]
at org.opensearch.performanceanalyzer.commons.util.Util.invokePrivileged( [performance-analyzer-commons-1.0.0.jar:?]
at org.opensearch.performanceanalyzer.commons.event_process.EventLogFileHandler.writeTmpFile( [performance-analyzer-commons-1.0.0.jar:?]
at org.opensearch.performanceanalyzer.writer.EventLogQueueProcessor.purgeQueueAndPersist( [opensearch-performance-analyzer-]
at java.util.concurrent.Executors$ [?:?]
at java.util.concurrent.FutureTask.runAndReset( [?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker( [?:?]
at java.util.concurrent.ThreadPoolExecutor$ [?:?]
at [?:?]

Reason for error reporting:Permission issues

The account used for the 1.3.0 version of Node-2 that was first launched is: elk
The account used for the newly added data node version 2.9.0 of node-1 is: opensearch

After investigation, it was found that the error path of node-1,/dev/shm/performanceanalyzer, belongs to elk. (Previously, an elk account was used to launch a test node for a tar package, resulting in the directory belonging to elk.) Modify the corresponding directory to belong to opensearch, or modify the startup account number of node-1 to elk and simultaneously modify the data, log, and configuration directories to belong to elk

Scheme two I used.

Scheme one:
chown -R opensearch:opensearch performanceanalyzer

Scheme two:

chwon -R elk:elk /etc/opensearch/
chwon -R elk:elk /var/log/opensearch/
chwon -R elk:elk /var/lib/opensearch/


@_free What’s the reason for mixing the versions? According to your description, node-1 has version 2.9.0 and node-2 1.3.1.

Upgrade version ,The node name was not modified during previous testing

@_free Just to clarify. Are you reporting an issue here or a solution you’ve discovered?
Any folders that OpenSeach need to write read should belong to OpenSearch user and group or user and group with ID 1000:1000.