Logging/Log rotation does not work for catalina.out
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
tomcat10 (Ubuntu) |
New
|
Undecided
|
Unassigned | ||
tomcat9 (Debian) |
New
|
Unknown
|
|||
tomcat9 (Ubuntu) |
Fix Released
|
Undecided
|
Andreas Hasenack | ||
Focal |
Fix Released
|
Undecided
|
Andreas Hasenack | ||
Jammy |
Fix Released
|
Undecided
|
Andreas Hasenack |
Bug Description
[Impact]
Log handling in tomcat9 is broken in several ways:
a) logrotate fails to rotate the catalina.out log file
b) rsyslog is configured to chown the catalina.out log file to the tomcat user, but lacks the privileges to do so (in Ubuntu, rsyslog runs unprivileged)
c) even though on a fresh install tomcat9 is able to log to /var/log/
[Test Plan]
Create a container or VM for the ubuntu release under test. Here we will use lxc, and the commands and outputs below will be shown for jammy:
lxc launch ubuntu:jammy j-tomcat9-logging
lxc shell j-tomcat9-logging
apt update && apt install tomcat9
Observe that the /var/log/tomcat9 directory has permissions 02770 and that the catalina.out file in it is owned by syslog:adm:
root@j-
total 12
drwxrws--- 1 tomcat adm 188 Jul 20 18:32 .
drwxrwxr-x 1 root syslog 314 Jul 20 18:32 ..
-rw-r----- 1 tomcat adm 5994 Jul 20 18:32 catalina.
-rw-r----- 1 syslog adm 3522 Jul 20 18:32 catalina.out
-rw-r----- 1 tomcat adm 0 Jul 20 18:32 localhost.
-rw-r----- 1 tomcat adm 0 Jul 20 18:32 localhost_
But here the problems start, and these are the ones fixed by this SRU:
a) rsyslog is complaining that it can't change the ownership of catalina.out:
root@j-
Jul 20 18:32:22 j-tomcat9-logging rsyslogd: error during config processing: omfile: chown for file '/var/log/
b) logrotate fails:
root@j-
error: error opening /var/log/
And catalina.out remains unrotated:
root@j-
total 12
drwxrws--- 1 tomcat adm 188 Jul 20 18:32 .
drwxrwxr-x 1 root syslog 430 Jul 20 18:33 ..
-rw-r----- 1 tomcat adm 5994 Jul 20 18:32 catalina.
-rw-r----- 1 syslog adm 3522 Jul 20 18:32 catalina.out
-rw-r----- 1 tomcat adm 0 Jul 20 18:32 localhost.
-rw-r----- 1 tomcat adm 0 Jul 20 18:32 localhost_
c) if the package is reinstalled, or an update without this fix becomes available and is applied, the catalina.out file will have incorrect ownership and rsyslog won't be able to write to it anymore:
before reinstall:
root@j-
-rw-r----- 1 syslog adm 3523 Jul 20 18:49 catalina.out
after reinstall:
root@j-
Reading package lists... Done
(...)
Processing triggers for rsyslog (8.2112.
root@j-
-rw-r----- 1 tomcat adm 3797 Jul 20 18:49 catalina.out
And logging is broken:
root@j-
Jul 20 18:49:59 j-tomcat9-logging rsyslogd: file '/var/log/
Now install the tomcat9 package from proposed.
a) rsyslog won't complain anymore about failing to open or chown the file:
root@j-
root@j-
root@j-
root@j-
Jul 20 18:55:09 j-tomcat9-logging rsyslogd: imuxsock: Acquired UNIX socket '/run/systemd/
Jul 20 18:55:09 j-tomcat9-logging rsyslogd: imklog: cannot open kernel log (/proc/kmsg): Permission denied.
Jul 20 18:55:09 j-tomcat9-logging rsyslogd: activation of module imklog failed [v8.2112.0 try https:/
Jul 20 18:55:09 j-tomcat9-logging rsyslogd: rsyslogd's groupid changed to 111
Jul 20 18:55:09 j-tomcat9-logging rsyslogd: rsyslogd's userid changed to 104
Jul 20 18:55:09 j-tomcat9-logging rsyslogd: [origin software="rsyslogd" swVersion=
b) This time logrotate works, and the catalina.out file will be rotated:
root@j-
root@j-
total 24
drwxrws--- 1 tomcat adm 216 Jul 20 18:39 .
drwxrwxr-x 1 root syslog 612 Jul 20 18:39 ..
-rw-r----- 1 tomcat adm 12487 Jul 20 18:37 catalina.
-rw-r----- 1 syslog adm 0 Jul 20 18:39 catalina.out
-rw-r----- 1 syslog adm 7699 Jul 20 18:39 catalina.out.1
-rw-r----- 1 tomcat adm 0 Jul 20 18:32 localhost.
-rw-r----- 1 tomcat adm 0 Jul 20 18:32 localhost_
c) reinstalling the package won't break logging again:
root@j-
-rw-r----- 1 syslog adm 7974 Jul 20 19:10 /var/log/
root@j-
Reading package lists... Done
(...)
Processing triggers for rsyslog (8.2112.
root@j-
-rw-r----- 1 syslog adm 12152 Jul 20 19:11 /var/log/
[Where problems could occur]
These logging problems have been ongoing for quite some time, at least since Focal (20.04), so it's quite possible that users have made local configuration changes to avoid it. Part of the fix in this SRU is in the tomcat9.postinst maintainer script, which is difficult for local users to override, so it's possible that this update will undo, or conflict, with whatever local fixes were made.
It's hard to predict what it could be, and trying to be smart about it carries its own set of risks and complexities. I didn't go down that road, trying to keep the change simple and easy to understand.
[Other Info]
Older logging bug: https:/
MP proposing this fix for Kinetic, with some discussion and considerations: https:/
For focal, this SRU is also changing the permissions of /var/log/tomcat9 from 02750 to 02770. In jammy and later, it's 02770 already.
[Original Description]
In Ubuntu 20.04, with `tomcat9-
This could be fixed in a newer package but was not backported:
https:/
In Ubuntu 22.04, with `tomcat9-9.0.58-1` (latest) package, `logrotated` is not able to rotate `/var/log/
Because the `catalina.out` is created with `syslog:adm` ownerships. `syslog` user does not have enough permissions to change this.
This causes following error:
rsyslogd: error during config processing: omfile: chown for file '/var/log/
At the same time, the `/etc/logrotate
This causes logrotate to copy the contents of `/var/log/
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu79
Architecture: amd64
CasperMD5CheckR
DistroRelease: Ubuntu 22.04
InstallationDate: Installed on 2022-02-27 (18 days ago)
InstallationMedia: Ubuntu 22.04 LTS "Jammy Jellyfish" - Alpha amd64 (20220121)
Package: tomcat9 9.0.58-1
PackageArchitec
ProcVersionSign
RebootRequiredPkgs: Error: path contained symlinks.
Tags: jammy
Uname: Linux 5.15.0-18-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
Related branches
- Robie Basak: Approve (ubuntu-sru)
- Canonical Server Reporter: Pending requested
-
Diff: 58 lines (+19/-3)4 files modifieddebian/changelog (+12/-0)
debian/logrotate.template (+2/-1)
debian/tomcat9.postinst (+4/-1)
debian/tomcat9.tmpfile (+1/-1)
- Robie Basak: Approve (ubuntu-sru)
- Canonical Server Reporter: Pending requested
-
Diff: 74 lines (+21/-4)5 files modifieddebian/changelog (+13/-0)
debian/control (+2/-1)
debian/logrotate.template (+2/-2)
debian/rsyslog/tomcat9.conf (+1/-1)
debian/tomcat9.postinst (+3/-0)
- git-ubuntu bot: Approve
- Christian Ehrhardt (community): Approve
- Canonical Server Reporter: Pending requested
- Canonical Server: Pending requested
-
Diff: 74 lines (+21/-4)5 files modifieddebian/changelog (+13/-0)
debian/control (+2/-1)
debian/logrotate.template (+2/-2)
debian/rsyslog/tomcat9.conf (+1/-1)
debian/tomcat9.postinst (+3/-0)
description: | updated |
tags: | added: server-todo |
Changed in tomcat9 (Ubuntu): | |
assignee: | nobody → Andreas Hasenack (ahasenack) |
Changed in tomcat9 (Ubuntu Focal): | |
assignee: | nobody → Andreas Hasenack (ahasenack) |
Changed in tomcat9 (Ubuntu Jammy): | |
assignee: | nobody → Andreas Hasenack (ahasenack) |
Changed in tomcat9 (Debian): | |
status: | Unknown → New |
Changed in tomcat9 (Ubuntu Jammy): | |
status: | Confirmed → In Progress |
Changed in tomcat9 (Ubuntu Focal): | |
status: | Confirmed → In Progress |
description: | updated |
description: | updated |
description: | updated |
description: | updated |
description: | updated |
description: | updated |
description: | updated |
tags: |
added: verification-done removed: verification-needed |
no longer affects: | tomcat10 (Ubuntu Focal) |
no longer affects: | tomcat10 (Ubuntu Jammy) |
tags: | removed: server-todo |
Thank you for taking the time to report this bug and helping to make Ubuntu better. Please execute the following command only once, as it will automatically gather debugging information, in a terminal:
apport-collect 1964881
When reporting bugs in the future please use apport by using 'ubuntu-bug' and the name of the package affected. You can learn more about this functionality at https:/ /wiki.ubuntu. com/ReportingBu gs.