Performance-Analyzer-Logs not rotated

Plugins Performance Analyzer
1

Hi,

We are running out of space on /tmp on the elastic servers

-rw-r--r-- 1 elasticsearch elasticsearch 2305568768 Aug  3 12:36 performance_analyzer_agent_stats.log
-rw-r--r-- 1 elasticsearch elasticsearch   15613952 Aug  3 12:37 PerformanceAnalyzer.log

Especially the performance_analyzer_agent_stats.log uses currently 2.2GB (and not leaving anything left)

I looked into the file. The first statement ist from
EndTime=Wed, 29 Jan 2020 11:39:29 CET

So it seems the log is not rotated. Since the log lies in /tmp I assume that this should not be filled at all to this level.

What can I do to get rid of this log (it does not seem to contain any useful data for normal operations) or that this gets rotated/truncated appropriately

Regards Michael

1 Like
2

We are running into the same issue.

It seems more or less a bug.

3

Thanks for reporting the issue. We will look into this.

4

Hi,

We still have this problem and are in need to manually delete and restart the whole cluster.
Is there any effort or new information available? When will this be fixed? (or is there a configuration change possible)?

Regards Michael

5

would also like to bring attention to this issue, is there documentation somewhere that will help us configure automatic rotation of the performance analyzer logs?

kinda wondering if these are even needed if we have /dev/shm/performanceanalyzer

6

We have this exact same issue.

Is there really no info or acknowledgement of this issue? Or a potential good fix?

7

I changed /usr/share/elasticsearch/performance-analyzer-rca/pa_config/log4j2.xml to

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO">
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
        </Console>
        <RollingFile name="PerformanceAnalyzerLog" fileName="/tmp/PerformanceAnalyzer.log" filePattern="/tmp/PerformanceAnalyzer-%i.log.gz" immediateFlush="true" append="true">
            <PatternLayout pattern="%d{yyy-MM-dd HH:mm:ss.SSS} [PA:Reader] [%t] %-5level %logger{36} - %msg%n"/>
            <Policies>
                <SizeBasedTriggeringPolicy size="100 MB"/>
            </Policies>
            <DefaultRolloverStrategy max="5"/>
        </RollingFile>
        <File name="StatsLog" fileName="/tmp/performance_analyzer_agent_stats.log" immediateFlush="true" append="true">
        </File>
    </Appenders>
    <Loggers>
        <Logger name="stats_log" level="info" additivity="false">
            <AppenderRef ref="StatsLog"/>
        </Logger>
        <Root level="error">
            <AppenderRef ref="Console" />
            <AppenderRef ref="PerformanceAnalyzerLog"/>
        </Root>
    </Loggers>
</Configuration>

Logs now get rotated and zipped when they are 100M

# ls -rtlh /tmp/PerformanceAnalyzer*
-rw-r--r-- 1 elasticsearch elasticsearch 1.7M Jan 31 23:21 /tmp/PerformanceAnalyzer-1.log.gz
-rw-r--r-- 1 elasticsearch elasticsearch 1.7M Feb  1 03:24 /tmp/PerformanceAnalyzer-2.log.gz
-rw-r--r-- 1 elasticsearch elasticsearch 1.7M Feb  1 07:27 /tmp/PerformanceAnalyzer-3.log.gz
-rw-r--r-- 1 elasticsearch elasticsearch 1.7M Feb  1 11:31 /tmp/PerformanceAnalyzer-4.log.gz
-rw-r--r-- 1 elasticsearch elasticsearch 1.7M Feb  1 15:34 /tmp/PerformanceAnalyzer-5.log.gz
-rw-r--r-- 1 elasticsearch elasticsearch  11M Feb  1 15:59 /tmp/PerformanceAnalyzer.log
1 Like