logrotate: Tidying and archiving log files

In short: A tool that regularly renames, compresses and deletes old log files so they do not fill the disk.

In more detail: Programs constantly write messages to log files, for example /var/log/nginx/access.log. Without maintenance they grow without limit until the disk is full. logrotate “rotates” the files: the current file is renamed (access.log.1), a new one begins, older ones are compressed and deleted after a period.

In Depth

How it runs

Usually a systemd timer or a cron job starts logrotate once a day. Configuration is in /etc/logrotate.conf and in files under /etc/logrotate.d/ (one per application).

Example

/var/log/myapp/*.log {
    weekly
    rotate 8
    compress
    delaycompress
    missingok
    notifempty
    create 0640 myapp adm
    postrotate
        systemctl reload myapp > /dev/null 2>&1 || true
    endscript
}

weekly rotates weekly, rotate 8 keeps 8 old files, compress packs them with gzip, delaycompress compresses only the second-newest, missingok ignores missing files, notifempty skips empty ones. postrotate tells the program to open the new file.

copytruncate

If a program keeps the log file open permanently and does not know “reopen”, it would continue writing to the renamed file. With copytruncate logrotate copies the file and empties the original. A few lines can be lost in the process; it is a stopgap.

Testing and alternatives

sudo logrotate -d /etc/logrotate.d/myapp shows in a dry run what would happen; -f forces a rotation. The systemd journal limits its size itself (SystemMaxUse= in journald.conf). Limits by size (size 100M) instead of time are possible. For analysing large amounts of logs one uses central systems such as Loki or an Elastic suite.

See also: Logs and scheduling, Logging and log level, systemd