Disable Logback log files in containers

by leelight · Published November 16, 2020 · Updated November 16, 2020

We found the spring boot project running containers will keep restarting after some time, maybe 10 days, maybe 5 days. The root cause is spring logback is writing log files into /tmp/ directory as “”spring.log in the containers and the log files are growing extremely fast. Finally the log files will kill the container.

I have tried many ways to disable the output log file.

1. 1. Logback configure

such as Max Log File Size, Total Size of Log Files, Log Files Max History, they did not help.

logging.file.max-size=1MB
logging.file.total-size-cap=10MB
logging.file.max-history=30
logging.file.clean-history-on-start=true

logging.file.max-size=1MB

logging.file.total-size-cap=10MB

logging.file.max-history=30

logging.file.clean-history-on-start=true

Also tried with properties enject via docker compose, does not help.

    test-app:
        image: test/image
        environment:
        - _JAVA_OPTIONS=-Xmx512m -Xms256m
        - LOGGING_FILE=/tmp/app.log
        - LOGGING_FILE_MAX_HISTORY=1
        - LOGGING_FILE_MAX_SIZE=1MB
        - LOGGING_FILE_TOTAL_SIZE_CAP=2MB
        - LOG_FILE_MAX_HISTORY=1
        - LOG_FILE_MAX_SIZE=1MB
        - LOG_FILE_TOTAL_SIZE_CAP=2MB

test-app:

image: test/image

environment:

- _JAVA_OPTIONS=-Xmx512m -Xms256m

- LOGGING_FILE=/tmp/app.log

- LOGGING_FILE_MAX_HISTORY=1

- LOGGING_FILE_MAX_SIZE=1MB

- LOGGING_FILE_TOTAL_SIZE_CAP=2MB

- LOG_FILE_MAX_HISTORY=1

- LOG_FILE_MAX_SIZE=1MB

- LOG_FILE_TOTAL_SIZE_CAP=2MB

Reference :

codejava.net/frameworks/spring-boot/logback-rolling-files-example

https://docs.spring.io/spring-boot/docs/2.1.4.RELEASE/reference/html/boot-features-logging.html

https://www.baeldung.com/spring-boot-logging

https://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/html/spring-boot-features.html#boot-features-logging

2. 2. Use Cron job in contaienr

Tried many ways but no success. Maybe because the docker image is openjdk:8-jre-alpine, and the cron can not be run in this image.

There are a lot of example here: https://stackoverflow.com/questions/37458287/how-to-run-a-cron-job-inside-a-docker-container

https://github.com/nehabhardwaj01/docker-cron

3. 3. Use Spring Scheduler

This does work of course, but it is not a clean way.

package com.test.scheduler;

public interface BacklogCleanScheduler {

    void cleanup();
}

package com.test.scheduler;

public interface BacklogCleanScheduler {

void cleanup();

}

package com.test.scheduler.impl;

import com.test.scheduler.BacklogCleanScheduler;
import lombok.SneakyThrows;
import org.apache.commons.io.FileUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

import java.io.File;
import java.nio.file.Files;
import java.time.Instant;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.util.List;

@Component
public class BacklogCleanSchedulerImpl implements BacklogCleanScheduler {

    private final Logger log = LoggerFactory.getLogger(BacklogCleanSchedulerImpl.class);

    @SneakyThrows
    @Override
    @Scheduled(cron = "1 * * ? * *")
    public void cleanup() {
        log.info("Logback cleanup start");
        findLogs("/tmp");
    }

    @SneakyThrows
    public void findLogs(String filePath) {
        File dir = new File(filePath);
        String[] extensions = new String[]{"gz"};
        List<File> files = (List<File>) FileUtils.listFiles(dir, extensions, true);
        log.debug("Logback cleanup size {}", files.size());
        for (File file : files) {
            if (isLogExpired(file)) {
                deleteFile(file);
            }
        }
    }

    public void deleteFile(File file) {
        try {
            Files.delete(file.toPath());
        } catch (Exception e) {
            log.error("Logback cleanup failed to delete {}", file);
        }
    }

    public boolean isLogExpired(File file) {
        ZonedDateTime fileDate = Instant.ofEpochMilli(file.lastModified()).atZone(ZoneId.of("Europe/Berlin"));
        ZonedDateTime oldDate = ZonedDateTime.now().minusDays(1L);
        return fileDate.isBefore(oldDate);
    }
}

package com.test.scheduler.impl;

import com.test.scheduler.BacklogCleanScheduler;

import lombok.SneakyThrows;

import org.apache.commons.io.FileUtils;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

import org.springframework.scheduling.annotation.Scheduled;

import org.springframework.stereotype.Component;

import java.io.File;

import java.nio.file.Files;

import java.time.Instant;

import java.time.ZoneId;

import java.time.ZonedDateTime;

import java.util.List;

@Component

public class BacklogCleanSchedulerImpl implements BacklogCleanScheduler {

private final Logger log = LoggerFactory.getLogger(BacklogCleanSchedulerImpl.class);

@SneakyThrows

@Override

@Scheduled(cron = "1 * * ? * *")

public void cleanup() {

log.info("Logback cleanup start");

findLogs("/tmp");

}

@SneakyThrows

public void findLogs(String filePath) {

File dir = new File(filePath);

String[] extensions = new String[]{"gz"};

List<File> files = (List<File>) FileUtils.listFiles(dir, extensions, true);

log.debug("Logback cleanup size {}", files.size());

for (File file : files) {

if (isLogExpired(file)) {

deleteFile(file);

}

public void deleteFile(File file) {

try {

Files.delete(file.toPath());

} catch (Exception e) {

log.error("Logback cleanup failed to delete {}", file);

}

public boolean isLogExpired(File file) {

ZonedDateTime fileDate = Instant.ofEpochMilli(file.lastModified()).atZone(ZoneId.of("Europe/Berlin"));

ZonedDateTime oldDate = ZonedDateTime.now().minusDays(1L);

return fileDate.isBefore(oldDate);

}

4. 4. Lockback Configuration (Best)

Very easy.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE configuration>

<configuration scan="true">
    <include resource="org/springframework/boot/logging/logback/defaults.xml"/>
    <include resource="org/springframework/boot/logging/logback/console-appender.xml"/>

    <root>
        <appender-ref ref="CONSOLE"/>
    </root>

    <!-- https://logback.qos.ch/manual/configuration.html#shutdownHook and https://jira.qos.ch/browse/LOGBACK-1090 -->
    <shutdownHook class="ch.qos.logback.core.hook.DelayingShutdownHook"/>

    <contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
        <resetJUL>true</resetJUL>
    </contextListener>

</configuration>

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE configuration>

<root>

<appender-ref ref="CONSOLE"/>

</root>

</contextListener>

</configuration>

Such configuration will set all log output to console. This will not affect the debugger when you using :

docker logs -f --tail=1000 container_id

1	docker logs -f --tail=1000 container_id

If you have configured the logstash to Kibana directly, not via Filebeat with /tmp/spring.log. Such change will not affect the Kibina.

Cookie	Duration	Description
cookielawinfo-checkbox-analytics	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional	11 months	The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy	11 months	The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.

Disable Logback log files in containers

You may also like...

Leave a Reply Cancel reply

Meta

Categories

Disable Logback log files in containers

1. 1. Logback configure

2. 2. Use Cron job in contaienr

3. 3. Use Spring Scheduler

4. 4. Lockback Configuration (Best)

You may also like...

[转]有赞百亿级日志系统是怎么设计出来的?

[总结]日志格式

[转]最佳日志实践 v2.0

Leave a Reply Cancel reply

Meta

Categories