Bug#1092031: runit-services: duplicate rsyslog logrotate

[Dimitris T.]

Package: runit-services
Version: 0.8.1
Severity: normal

Hey,
not sure exactly when, but probably sometime in the last 2 versions of the package, a new /etc/logrotote.d/aa-rsyslog-runit file was introduced.
logrotate cron outputs some errors since. messages :

"/etc/cron.daily/logrotate:
error: rsyslog:1 duplicate log entry for /var/log/syslog
error: rsyslog:1 duplicate log entry for /var/log/mail.log
error: rsyslog:1 duplicate log entry for /var/log/kern.log
error: rsyslog:1 duplicate log entry for /var/log/auth.log
error: rsyslog:1 duplicate log entry for /var/log/user.log
error: rsyslog:1 duplicate log entry for /var/log/cron.log
error: found error in file rsyslog, skipping
run-parts: /etc/cron.daily/logrotate exited with return code 1"

there is also /etc/logrotate.d/rsyslog with the same logs to rotate, provided by rsyslog package.

probably related to bug #1079268 and this: https://salsa.debian.org/debian/runit-services/-/commit/6d5b38ff9eb15ee838da078a12a67dc15f9d3bca (?)
maybe this check is missing in .postinst?

thanks in advance,
d.

-- System Information:
Distributor ID: Devuan
Description: Devuan GNU/Linux 6 (excalibur/ceres)
Release: 6
Codename: excalibur ceres
Architecture: x86_64

Kernel: Linux 6.12.6-amd64 (SMP w/4 CPU threads; PREEMPT)
Locale: LANG=el_GR.UTF-8, LC_CTYPE=el_GR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: runit (via /run/runit.stopit)
LSM: AppArmor: enabled

Versions of packages runit-services depends on:
ii runit 2.1.2-60
ii runit-helper 2.16.4

Versions of packages runit-services recommends:
ii runit-init 2.1.2-60

Versions of packages runit-services suggests:
pn socklog <none>

-- no debconf information

[Andrew Bower]

Hi Dimitris,

Post by Dimitris T.
not sure exactly when, but probably sometime in the last 2 versions of the package, a new /etc/logrotote.d/aa-rsyslog-runit file was introduced.

I have the same problem and have for a while, too. I am currently on
0.8.1 and have runit/2.1.2-61~ from salsa next branch.

Post by Dimitris T.
https://salsa.debian.org/debian/runit-services/-/commit/6d5b38ff9eb15ee838da078a12a67dc15f9d3bca
(?) maybe this check is missing in .postinst?

You and Lorenzo obviously understand the background to this issue
better than I would. If you have a patch I will be happy to test
it out on my systems and report back to you both with feedback!

Andrew

[Lorenzo]

Control: tags -1 moreinfo

Hi Dimitris,

On Fri, 03 Jan 2025 22:44:57 +0200 "Dimitris T."

Post by Dimitris T.
Package: runit-services
Version: 0.8.1
Severity: normal
Hey,
not sure exactly when, but probably sometime in the last 2 versions
of the package, a new /etc/logrotote.d/aa-rsyslog-runit file was
introduced.

correct, it's in 0.8.0

Post by Dimitris T.
error: rsyslog:1 duplicate log entry for /var/log/syslog
error: rsyslog:1 duplicate log entry for /var/log/mail.log
error: rsyslog:1 duplicate log entry for /var/log/kern.log
error: rsyslog:1 duplicate log entry for /var/log/auth.log
error: rsyslog:1 duplicate log entry for /var/log/user.log
error: rsyslog:1 duplicate log entry for /var/log/cron.log
error: found error in file rsyslog, skipping
run-parts: /etc/cron.daily/logrotate exited with return code 1"
there is also /etc/logrotate.d/rsyslog with the same logs to rotate,
provided by rsyslog package.
probably related to bug #1079268 and

Yes, that's the bug

Post by Dimitris T.
this: https://salsa.debian.org/debian/runit-services/-/commit/6d5b38ff9eb15ee838da078a12a67dc15f9d3bca (?) maybe this check is missing in .postinst?

not really: the idea is that, since there are two logrotate files with
the same entries, the first processed "wins" and the second is skipped.
I agree is a poor solution, but without cooperation on rsyslog side I
don't see many alternatives.

I tested again and besides the error from logrotate, I don't see a real
issue here:

(according to my testing)

* logrotate emits an error but continues to process his files, included
those that are ordered after /etc/logrotate.d/rsyslog
(so no log rotation is skipped due to this error, aa-rsyslog-runit
simply replaces rsyslog)

* cron jobs are not affected by logrotate exit status, so every cron job
after logrotate is processed (I tested with anacron on daily job)

Could you please look again and see if logrotation and/or cron jobs are
actually skipped due to this error (look at the expected action in the
filesystem)?

Best Regards,
Lorenzo

[Andrew Bower]

Hi everyone,

Three thoughts come to mind, reading this. Two of them, terrible...

something relative i noticed.
if i set /etc/logrotate.d/rsyslog -> rsyslog.disabled (so only one
logrotation script for rsyslog ), then logrotate completes succesfully. 
but...
when some runit-services trigger is run following some package upgrade (eg.
rsyslog), then, aa-rsyslog-runit is also renamed to
aa-rsyslog-runit.disabled.. (?!)  and there is no (rsys)log rotation!!
===
$ doas mv /etc/logrotate.d/rsyslog /etc/logrotate.d/rsyslog.disabled
-rw-r--r--   1 root root   260 Dec  26 13:53 aa-rsyslog-runit
-rw-r--r--   1 root root   248 Feb  23  2023 rsyslog.disabled
$ doas apt install --reinstall rsyslog
...
Processing triggers for runit (2.1.2-61) ...
ok: run: rsyslog: (pid 31634) 0s
Processing triggers for orphan-sysvinit-scripts (0.17) ...
Processing triggers for runit-services (0.8.2) ...
-rw-r--r--   1 root root   260 Dec  26 13:53 aa-rsyslog-runit.disabled
-rw-r--r--   1 root root   248 Feb  23  2023 rsyslog.disabled

(1) Ah, looking at d/runit-services.postinst, that looks fixable.
runit-services is trying to make sure that if it adds this file and
rsyslog's rotation is already disabled, then the new file should be
installed as disabled, too.

The logic in the postinst script could be changed so it only does this
if it just added the logrotate file, and not if it previously had.

The other two, whacky ideas I had were:

(2) Install an 'inotifywatch' on the log files being moved by logrotated
and then perform sv hup. Yeah... that's excessive!

(3) Regularly perform 'sv hup' on rsyslogd as it is harmless to do so;
make sure to schedule it when we guess logrotate is finished. Also, this
is a terrible idea!

and if/when cooperation with rsyslog is restored (...?) then it could be
easily reverted... (i think).

The irony of the level of rsyslog cooperation with non-systemd users, is
that as rsyslog usage drops off a cliff due to it no longer being
installed by default, we will become an ever greater proportion of its
users!

https://qa.debian.org/popcon.php?package=rsyslog

[lorenzo]

Conreol: tags -1 -moreinfo

Hi Dimitris, hi Andrew,

On Fri, 14 Feb 2025 17:40:28 +0200 "Dimitris T."

Post by Lorenzo
Could you please look again and see if logrotation and/or cron jobs
are actually skipped due to this error (look at the expected action
in the filesystem)?

you're correct, apart from this annoying daily "spam" logrotate/cron
error, nothing else is affected.  also tested with daily anacron.

thanks for confirming this :)

(but in general, i would prefer it completely "fixed" = no
errors/spam :)  )

ok, I understand.

--
something relative i noticed.
if i set /etc/logrotate.d/rsyslog -> rsyslog.disabled (so only one
logrotation script for rsyslog ), then logrotate completes
succesfully. but...
when some runit-services trigger is run following some package
upgrade (eg. rsyslog), then, aa-rsyslog-runit is also renamed to
aa-rsyslog-runit.disabled.. (?!)  and there is no (rsys)log rotation!!
(1) Ah, looking at d/runit-services.postinst, that looks fixable.
runit-services is trying to make sure that if it adds this file and
rsyslog's rotation is already disabled, then the new file should be
installed as disabled, too.

from coding point of view this is fixable, the issue is more on the
social and policy side. in short, we risk to be forced to revert due to
an escalation from rsyslogd maintainer..
well, there is no reply in #1079270 so as long as it doesn't break usage
under systemd maybe he doesn't care.

I'll see what I can do..

(2) Install an 'inotifywatch' on the log files being moved by
logrotated and then perform sv hup. Yeah... that's excessive!

I confess that I considered this already for a second or so when had to
fix #1079268, but them, hum, better not..

Another solution that I considered is to have rsyslogd log to
stdout and catch logs with svlogd, similar to socklog-run, but the
problem is that
* it requires editing rsyslog config file on the fly, so it's brittle
* users will likely be surprised to find system logs in
/var/log/runit/rsyslogd
so I discarded that too.

Best,
Lorenzo

[Andrew Bower]

Post by lorenzo
On Fri, 14 Feb 2025 17:40:28 +0200 "Dimitris T."

if i set /etc/logrotate.d/rsyslog -> rsyslog.disabled (so only one
logrotation script for rsyslog ), then logrotate completes
succesfully. but...
when some runit-services trigger is run following some package
upgrade (eg. rsyslog), then, aa-rsyslog-runit is also renamed to
aa-rsyslog-runit.disabled.. (?!)  and there is no (rsys)log rotation!!
(1) Ah, looking at d/runit-services.postinst, that looks fixable.
runit-services is trying to make sure that if it adds this file and
rsyslog's rotation is already disabled, then the new file should be
installed as disabled, too.

from coding point of view this is fixable, the issue is more on the
social and policy side. in short, we risk to be forced to revert due to
an escalation from rsyslogd maintainer..
well, there is no reply in #1079270 so as long as it doesn't break usage
under systemd maybe he doesn't care.
I'll see what I can do..

I think *a* change would be ok - currently you make the runit state
identical to the non-runit state regardless. If you only do that at the
point that you first add the file, you are honouring the user's
preference.

So:

if not (runit.enabled or runit.disabled) then {
if standard.disabled
install runit script as disabled
else if standard.enabled
install runit script as enabled
}

However, this only solves this original problem in the case where the
user has deliberately disabled the standard script to suppress the
warnings. Users who just install things get the warnings. I would agree
that there is no obvious solution within the runit package to this that
wouldn't be problematic from a social/policy point of view or
disproportionately complicated from a technical point of view.

The logical place to fix this is in the rsyslogd package.

Perhaps we should team up with sysv people to solve this in a
sustainable way. I wonder if we should contribute some functionality to
logrotate to allow augmenting configs. This could potentially be framed
as support for read-only usr type modes. But I'm probably guilty of
looking for a technical solution to a political problem here!

[lorenzo]

On Tue, 18 Feb 2025 16:04:49 +0000

Post by Andrew Bower
I think *a* change would be ok - currently you make the runit state
identical to the non-runit state regardless. If you only do that at
the point that you first add the file, you are honouring the user's
preference.
if not (runit.enabled or runit.disabled) then {
if standard.disabled
install runit script as disabled
else if standard.enabled
install runit script as enabled
}

maybe I can just watch /usr/sbin/rsyslogd to decide whether to install
our logrotate file and to disable it (if it's there and the rsyslogd
bin is removed); this way it would be possible for the user to disable
logrotate file from rsyslog package and turn off the warning.

if [ -f /usr/sbin/rsyslogd ] && [ ! -f
/etc/logrotate.d/aa-rsyslog-runit ]; then
if [ -f /etc/logrotate.d/aa-rsyslog-runit.disabled ]; then
mv -f /etc/logrotate.d/aa-rsyslog-runit.disabled \
/etc/logrotate.d/aa-rsyslog-runit
else cp /usr/share/runit-services/logrotate.d/aa-rsyslog-runit \
/etc/logrotate.d/
fi
fi

if [ ! -f /usr/sbin/rsyslogd ] && [ -f /etc/logrotate.d/aa-rsyslog-runit ]; then
mv -f /etc/logrotate.d/aa-rsyslog-runit \
/etc/logrotate.d/aa-rsyslog-runit.disabled
fi

I'm afraid I will still get duplicates of this bug as this
reaches stable, maybe I should leave this bug open.

[Andrew Bower]

How about biting the bullet and doing a dpkg-divert like
orphan-sysvinit-scripts does?

[lorenzo]

On Thu, 27 Feb 2025 21:17:18 +0000

Post by Andrew Bower
How about biting the bullet and doing a dpkg-divert like
orphan-sysvinit-scripts does?

well, then we need to conflict[1] with orphan-sysvinit-scripts, only
for this. Since runit in part uses sysvinit scripts I prefer not to..

[1] not sure what happens with two diversions on the same file, I guess
we get the link flipping from one target to another depending on the
configure order..

[Lorenzo]

Hi

Post by lorenzo
On Thu, 27 Feb 2025 21:17:18 +0000

Post by Andrew Bower
How about biting the bullet and doing a dpkg-divert like
orphan-sysvinit-scripts does?

well, then we need to conflict[1] with orphan-sysvinit-scripts, only
for this. Since runit in part uses sysvinit scripts I prefer not to..

I remembered this somehow wrong, o-s-s diverts the file in
/usr/lib/rsyslog while we are acting on the logrotate file
in /etc/logrotate.d/

so diverting the logrotate file is possible but there are still problems

debian policy [2] says
"Do not attempt to divert a conffile, as dpkg does not handle it well."

but also, when doing a diversion there will be a copy of the diverted
file under /etc/logrotate.d/, so it could cause the same error as in
this bug unless is suffixed as .disabled

finally rsyslog package does some handling of the file in
/etc/logrotate.d/ to rename it with .disabled suffix when the
package is removed but not purged

[2]https://www.debian.org/doc/debian-policy/ap-pkg-diversions.html

Post by lorenzo
[1] not sure what happens with two diversions on the same file, I
guess we get the link flipping from one target to another depending
on the configure order..