Custom log path not working?

OpenSearch
1

Hi guys,

My first topic! :beers:

I am installing Opensearch 2.11.1 on AKS (Kubernetes) with a pretty custom setup and everything seems to be working relatively okay (small fires I am slowly putting out), except custom log files.

In my opensearch.yml file, I have set:
path.logs: /usr/share/persistent_file_share/opensearch/logs/node1

And in the STDOUT/STDERR I see this (I removed the custom cluster name):
For complete error details, refer to the log at /usr/share/persistent_file_share/opensearch/logs/node1/opensearch-cluster.log

Here is the interesting bit:

  • the Kubernetes mount is /usr/share/persistent_file_share
  • the directory opensearch/logs/node1 is being created (!)
  • there is no log file generated

Since the sub-directory is being created, both the path.logs setting and the rights on the mount must be correct.

Why is it not generating / writing the log file?

This is probably a really silly question. Be gentle. :smiley:

2

Nobody? Perhaps some insights from @pablo ?

3

Hi @Laszlo

Do you have any error or warning messages in the OpenSearch logs?

4

I do not know. There is no log file. :laughing:

Seriously though, I do get a lot in STDOUT and STDERR, so surely there should be details in the log file. That there is no log file is definitely an issue.

5

@Laszlo How did you set your log4j2.properties file?

The OpenSearch logs are controlled by the log4j2.properties file. By default, log4j2.properties has a console configured as an output.

Could you share your configuration?

1 Like
6

Ah, I see! So log files are not even configured by default! Okay, thank you. Now I know what to do.

By the way, log4j is still a thing? :wink:
Quite a fiasco that was.

I moved all my own code to Winston, which I find really amazing.

7

@Laszlo It’s still used by OpenSearch. You can always create a feature request in OpenSearch GitHub and suggest a different solution.

By the way, this configuration worked for me.

     status = error

     appender.console.type = Console
     appender.console.name = console
     appender.console.layout.type = PatternLayout
     appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n

     appender.rolling.type = RollingFile
     appender.rolling.name = rolling
     appender.rolling.fileName = ${sys:opensearch.logs.base_path}/opensearch.log
     appender.rolling.layout.type = PatternLayout
     appender.rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
     appender.rolling.filePattern = ${sys:opensearch.logs.base_path}/opensearch-%d{yyyy-MM-dd}.log.gz
     appender.rolling.policies.type = Policies
     appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
     appender.rolling.policies.time.interval = 1
     appender.rolling.policies.time.modulate = true
     appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
     appender.rolling.policies.size.size = 256MB

     rootLogger.level = info
     rootLogger.appenderRef.console.ref = console
     rootLogger.appenderRef.rolling.ref = rolling
8

Nice! Much appreciated, @pablo !