Bug#800947: ACL for /var/log/journal not set for group adm

[Raphaël Halimi]

Package: systemd
Version: 226-4

Hi,

About persistent logging, README.Debian claims :

"systemd will add an ACL for read permissions for users in the "adm" group."

This is not working: after creating /var/log/journal with the "install"
command as instructed in the README.Debian, and even after several
reboots, the ACL is not set:

***@arche:~$ getfacl /var/log/journal/
getfacl : suppression du premier « / » des noms de chemins absolus
# file: var/log/journal/
# owner: root
# group: systemd-journal
# flags: -s-
user::rwx
group::r-x
other::r-x

Partition is ext4, so ACL are supported (I can set them manually as
instructed in the README.Debian file from Jessie).

Regards,

[Michael Biebl]

Post by Raphaël Halimi
Package: systemd
Version: 226-4
Hi,
"systemd will add an ACL for read permissions for users in the "adm" group."
This is not working: after creating /var/log/journal with the "install"
command as instructed in the README.Debian, and even after several
getfacl : suppression du premier « / » des noms de chemins absolus
# file: var/log/journal/
# owner: root
# group: systemd-journal
# flags: -s-
user::rwx
group::r-x
other::r-x

But the subdirectories of /var/log/journal have the correct ACL set, right?

[Raphaël Halimi]

Post by Michael Biebl
But the subdirectories of /var/log/journal have the correct ACL set, right?

Yes, you're right, I just noticed it; but using journalctl as a user
won't display system messages (only user messages), which is not the
expected behavior of adding a user in the "adm" group (pre-systemd).

Maybe it's because the system.journal file doesn't have the ACL set ?

***@arche:~$ getfacl -R /var/log/journal/
getfacl : suppression du premier « / » des noms de chemins absolus
# file: var/log/journal/
# owner: root
# group: systemd-journal
# flags: -s-
user::rwx
group::r-x
other::r-x

# file: var/log/journal//3deacfa10d0c169adfdeb36c50522bd6
# owner: root
# group: systemd-journal
# flags: -s-
user::rwx
group::r-x
group:adm:r-x
mask::r-x
other::r-x
default:user::rwx
default:group::r-x
default:group:adm:r-x
default:mask::r-x
default:other::r-x

# file: var/log/journal//3deacfa10d0c169adfdeb36c50522bd6/user-1000.journal
# owner: root
# group: root
user::rw-
user:raph:r--
group::r--
mask::r--
other::---

# file: var/log/journal//3deacfa10d0c169adfdeb36c50522bd6/system.journal
# owner: root
# group: root
user::rw-
group::r--
other::---

I admit I don't know ACLs very well, but aren't the "default:..." lines
supposed to mean that the files under there should have these
permissions too ?

Regards,

[Michael Biebl]

Post by Raphaël Halimi

Post by Michael Biebl
But the subdirectories of /var/log/journal have the correct ACL set, right?

Yes, you're right, I just noticed it; but using journalctl as a user
won't display system messages (only user messages), which is not the
expected behavior of adding a user in the "adm" group (pre-systemd).
Maybe it's because the system.journal file doesn't have the ACL set ?
getfacl : suppression du premier « / » des noms de chemins absolus
# file: var/log/journal/
# owner: root
# group: systemd-journal
# flags: -s-
user::rwx
group::r-x
other::r-x
# file: var/log/journal//3deacfa10d0c169adfdeb36c50522bd6
# owner: root
# group: systemd-journal
# flags: -s-
user::rwx
group::r-x
group:adm:r-x
mask::r-x
other::r-x
default:user::rwx
default:group::r-x
default:group:adm:r-x
default:mask::r-x
default:other::r-x
# file: var/log/journal//3deacfa10d0c169adfdeb36c50522bd6/user-1000.journal
# owner: root
# group: root
user::rw-
user:raph:r--
group::r--
mask::r--
other::---
# file: var/log/journal//3deacfa10d0c169adfdeb36c50522bd6/system.journal
# owner: root
# group: root
user::rw-
group::r--
other::---
I admit I don't know ACLs very well, but aren't the "default:..." lines
supposed to mean that the files under there should have these
permissions too ?

See
https://github.com/systemd/systemd/commit/8b258a645ae63dff3ab8dde6520d2e770e2a40f1

Apparently this was an intended change.

[Michael Biebl]

Post by Michael Biebl

Post by Raphaël Halimi

Post by Michael Biebl
But the subdirectories of /var/log/journal have the correct ACL set, right?

Yes, you're right, I just noticed it; but using journalctl as a user
won't display system messages (only user messages), which is not the
expected behavior of adding a user in the "adm" group (pre-systemd).
Maybe it's because the system.journal file doesn't have the ACL set ?
getfacl : suppression du premier « / » des noms de chemins absolus
# file: var/log/journal/
# owner: root
# group: systemd-journal
# flags: -s-
user::rwx
group::r-x
other::r-x
# file: var/log/journal//3deacfa10d0c169adfdeb36c50522bd6
# owner: root
# group: systemd-journal
# flags: -s-
user::rwx
group::r-x
group:adm:r-x
mask::r-x
other::r-x
default:user::rwx
default:group::r-x
default:group:adm:r-x
default:mask::r-x
default:other::r-x
# file: var/log/journal//3deacfa10d0c169adfdeb36c50522bd6/user-1000.journal
# owner: root
# group: root
user::rw-
user:raph:r--
group::r--
mask::r--
other::---
# file: var/log/journal//3deacfa10d0c169adfdeb36c50522bd6/system.journal
# owner: root
# group: root
user::rw-
group::r--
other::---
I admit I don't know ACLs very well, but aren't the "default:..." lines
supposed to mean that the files under there should have these
permissions too ?

See
https://github.com/systemd/systemd/commit/8b258a645ae63dff3ab8dde6520d2e770e2a40f1
Apparently this was an intended change.

Apparently the files were created before the ACLs have been set for
/var/log/journal/3deacfa10d0c169adfdeb36c50522bd6
so the journal files that were created did not inherit the correct ACLs
from the parent directory.

Possibly you created /var/log/journal or set Storage=persistent, but did
*not* reboot the system afterwards, which would trigger systemd-tmpfiles
to be run. And once you restart systemd-journald (which can happen by
systemd update), the journal files were created without the ACLs set.

On next reboot, the systemd.conf tmpfile did apply the ACL for the
directory, but it was too late at that point.

I wonder if we should fix the documentation to tell people to run
systemd-tmpfiles /usr/lib/tmpfiles/systemd.conf immediately after
enabling persistent journal.

[Michael Biebl]

Post by Michael Biebl
I wonder if we should fix the documentation to tell people to run
systemd-tmpfiles /usr/lib/tmpfiles/systemd.conf immediately after
enabling persistent journal.

We might also consider re-adding
https://anonscm.debian.org/cgit/pkg-systemd/systemd.git/commit/debian/systemd.postinst?id=03497b4ae5418470d64f2706af795122afe9645b

but maybe in a slightly modified way. Instead of running setfacl, we
could use
systemd-tmpfiles /usr/lib/tmpfiles.d/systemd.conf
to avoid re-adding a dependency on the "acl" package.

We could poke Lennart what he recommends.

[Raphaël Halimi]

Post by Michael Biebl
Apparently the files were created before the ACLs have been set for
/var/log/journal/3deacfa10d0c169adfdeb36c50522bd6
so the journal files that were created did not inherit the correct ACLs
from the parent directory.
Possibly you created /var/log/journal or set Storage=persistent, but did
*not* reboot the system afterwards, which would trigger systemd-tmpfiles
to be run. And once you restart systemd-journald (which can happen by
systemd update), the journal files were created without the ACLs set.
On next reboot, the systemd.conf tmpfile did apply the ACL for the
directory, but it was too late at that point.

No, I rebooted immediately after creating the directory.

Regards,

[Michael Biebl]

Post by Raphaël Halimi

Post by Michael Biebl
Apparently the files were created before the ACLs have been set for
/var/log/journal/3deacfa10d0c169adfdeb36c50522bd6
so the journal files that were created did not inherit the correct ACLs
from the parent directory.
Possibly you created /var/log/journal or set Storage=persistent, but did
*not* reboot the system afterwards, which would trigger systemd-tmpfiles
to be run. And once you restart systemd-journald (which can happen by
systemd update), the journal files were created without the ACLs set.
On next reboot, the systemd.conf tmpfile did apply the ACL for the
directory, but it was too late at that point.

No, I rebooted immediately after creating the directory.

Hm, right. There might be a race condition during boot, where
systemd-journald-flush.service is started before systemd-tmpfiles.service.
We could order systemd-journald-flush.service *after*
systemd-tmpfiles.service.

But, when using Storage=persistent, journald will create the directory
/var/log/journal/ itself. So this won't help in that case, unless
systemd-journald re-added the code to apply ACLs itself.


This change sucks from a user experience POV, as you basically now need
to make sure to apply the correct ACL yourself. I think the supplied ACL
rule in /usr/lib/tmpfiles.d/systemd.conf is pretty much useless.

Martin, any ideas?

Michael

[Michael Biebl]

Post by Michael Biebl
Hm, right. There might be a race condition during boot, where
systemd-journald-flush.service is started before systemd-tmpfiles.service.
We could order systemd-journald-flush.service *after*
systemd-tmpfiles.service.

Fwiw, this directly conflicts with what is used upstream in
https://github.com/systemd/systemd/blob/master/units/systemd-journal-flush.service.in#L15


[Unit]
Description=Flush Journal to Persistent Storage
...
Before=systemd-user-sessions.service systemd-tmpfiles-setup.service

[Felipe Sateler]

Post by Michael Biebl

Post by Raphaël Halimi

Post by Michael Biebl
Apparently the files were created before the ACLs have been set for
/var/log/journal/3deacfa10d0c169adfdeb36c50522bd6
so the journal files that were created did not inherit the correct ACLs
from the parent directory.
Possibly you created /var/log/journal or set Storage=persistent, but did
*not* reboot the system afterwards, which would trigger systemd-tmpfiles
to be run. And once you restart systemd-journald (which can happen by
systemd update), the journal files were created without the ACLs set.
On next reboot, the systemd.conf tmpfile did apply the ACL for the
directory, but it was too late at that point.

No, I rebooted immediately after creating the directory.

Hm, right. There might be a race condition during boot, where
systemd-journald-flush.service is started before systemd-tmpfiles.service.
We could order systemd-journald-flush.service *after*
systemd-tmpfiles.service.
But, when using Storage=persistent, journald will create the directory
/var/log/journal/ itself. So this won't help in that case, unless
systemd-journald re-added the code to apply ACLs itself.

That would be a bug in (upstream) systemd, I think. Journald appears
to set the ACL on new files but not on the /v/l/j directory.

Post by Michael Biebl
This change sucks from a user experience POV, as you basically now need
to make sure to apply the correct ACL yourself. I think the supplied ACL
rule in /usr/lib/tmpfiles.d/systemd.conf is pretty much useless.
Martin, any ideas?

I think a reasonable alternative is to ship using Storage=volatile by
default, and ship the directory in the package (or create it in
postinst).

[Michael Biebl]

Post by Felipe Sateler

Post by Michael Biebl
But, when using Storage=persistent, journald will create the directory
/var/log/journal/ itself. So this won't help in that case, unless
systemd-journald re-added the code to apply ACLs itself.

That would be a bug in (upstream) systemd, I think. Journald appears
to set the ACL on new files but not on the /v/l/j directory.

Are you sure? I thought systemd-journald removed that code and relies on
the file system to set them.

[Felipe Sateler]

Post by Michael Biebl

Post by Felipe Sateler

Post by Michael Biebl
But, when using Storage=persistent, journald will create the directory
/var/log/journal/ itself. So this won't help in that case, unless
systemd-journald re-added the code to apply ACLs itself.

That would be a bug in (upstream) systemd, I think. Journald appears
to set the ACL on new files but not on the /v/l/j directory.

Are you sure? I thought systemd-journald removed that code and relies on
the file system to set them.

It appears you are correct. I missed the `if (uid <= SYSTEM_UID_MAX)`
check, which makes sure journald will only set acl for regular users
(and thus, not the adm user), so the adm acl must be ensured by the
filesystem.

[Michael Biebl]

Post by Felipe Sateler
I think a reasonable alternative is to ship using Storage=volatile by
default, and ship the directory in the package (or create it in
postinst).

After thinking more about this, I think this is the only sane solution:
- Ship /var/log/journal in the systemd package
- Apply the ACL to /var/log/journal (not the subdirectory) in postinst
- Set the default from auto to volatile
- If a user had already created a /var/log/journal directory, check for
that in preinst and create a journald.conf.d snippet setting
Storage=persistent
- Update the instructions in README.Debian how to enable persistent
journal. Recommend to use a drop-in config in
/etc/systemd/journald.conf.d/ containing

[Journal]
Storage=persistent


I don't see a way how we can make Storage=auto work properly.

A nice side-effect of no-longer using Storage=auto would be, that we
could make systemd-container ship /var/log/journal/remote without problems.


Thoughts?

[Josh Triplett]

Post by Michael Biebl

Post by Felipe Sateler
I think a reasonable alternative is to ship using Storage=volatile by
default, and ship the directory in the package (or create it in
postinst).

- Ship /var/log/journal in the systemd package
- Apply the ACL to /var/log/journal (not the subdirectory) in postinst
- Set the default from auto to volatile
- If a user had already created a /var/log/journal directory, check for
that in preinst and create a journald.conf.d snippet setting
Storage=persistent
- Update the instructions in README.Debian how to enable persistent
journal. Recommend to use a drop-in config in
/etc/systemd/journald.conf.d/ containing
[Journal]
Storage=persistent
I don't see a way how we can make Storage=auto work properly.
A nice side-effect of no-longer using Storage=auto would be, that we
could make systemd-container ship /var/log/journal/remote without problems.
Thoughts?

This seems like the right answer. Would you also consider providing a
package ("systemd-journal-persistent") that 1) ships an
/etc/systemd/journald.conf.d/systemd-journal-persistent.conf with that
snippet, and 2) Provides system-log-daemon and linux-kernel-log-daemon,
just as syslog daemon packages do? That would make it much easier to
configure systems to use the journal as their primary log/syslog without
duplication.

- Josh Triplett

[Felipe Sateler]

Post by Josh Triplett

Post by Michael Biebl

Post by Felipe Sateler
I think a reasonable alternative is to ship using Storage=volatile by
default, and ship the directory in the package (or create it in
postinst).

- Ship /var/log/journal in the systemd package
- Apply the ACL to /var/log/journal (not the subdirectory) in postinst
- Set the default from auto to volatile
- If a user had already created a /var/log/journal directory, check for
that in preinst and create a journald.conf.d snippet setting
Storage=persistent
- Update the instructions in README.Debian how to enable persistent
journal. Recommend to use a drop-in config in
/etc/systemd/journald.conf.d/ containing
[Journal]
Storage=persistent
I don't see a way how we can make Storage=auto work properly.
A nice side-effect of no-longer using Storage=auto would be, that we
could make systemd-container ship /var/log/journal/remote without problems.
Thoughts?

This seems like the right answer. Would you also consider providing a
package ("systemd-journal-persistent") that 1) ships an
/etc/systemd/journald.conf.d/systemd-journal-persistent.conf with that
snippet, and 2) Provides system-log-daemon and linux-kernel-log-daemon,
just as syslog daemon packages do? That would make it much easier to
configure systems to use the journal as their primary log/syslog without
duplication.

I don't think system-log-daemon is an interface that promises
persistent logging. Otherwise the packages having Depends:
system-log-daemon (there are some) should have bugs filed against
them, as they would not be able to run in volatile systems.

[Josh Triplett]

Post by Felipe Sateler

Post by Josh Triplett

Post by Michael Biebl

Post by Felipe Sateler
I think a reasonable alternative is to ship using Storage=volatile by
default, and ship the directory in the package (or create it in
postinst).

- Ship /var/log/journal in the systemd package
- Apply the ACL to /var/log/journal (not the subdirectory) in postinst
- Set the default from auto to volatile
- If a user had already created a /var/log/journal directory, check for
that in preinst and create a journald.conf.d snippet setting
Storage=persistent
- Update the instructions in README.Debian how to enable persistent
journal. Recommend to use a drop-in config in
/etc/systemd/journald.conf.d/ containing
[Journal]
Storage=persistent
I don't see a way how we can make Storage=auto work properly.
A nice side-effect of no-longer using Storage=auto would be, that we
could make systemd-container ship /var/log/journal/remote without problems.
Thoughts?

This seems like the right answer. Would you also consider providing a
package ("systemd-journal-persistent") that 1) ships an
/etc/systemd/journald.conf.d/systemd-journal-persistent.conf with that
snippet, and 2) Provides system-log-daemon and linux-kernel-log-daemon,
just as syslog daemon packages do? That would make it much easier to
configure systems to use the journal as their primary log/syslog without
duplication.

I don't think system-log-daemon is an interface that promises
system-log-daemon (there are some) should have bugs filed against
them, as they would not be able to run in volatile systems.

systemd certainly doesn't tie the provision of syslog to persistent
logging, nor should it. And it should be possible to configure a system
to use the journal as syslog without persistence (useful for
diskless/stateless systems, for instance). However, Debian currently
ships with rsyslog by default, and in the past we've had the reasonable
concern of not wanting to log every syslog message to disk twice, once
in the journal and once in /var/log. So, if we want to provide a
package that makes it easy to enable the persistent journal, that
package should provide system-log-daemon.

- Josh Triplett

[Michael Biebl]

Control: tags -1 + pending

On Mon, 5 Oct 2015 12:26:02 +0200 =?UTF-8?Q?Rapha=c3=abl_Halimi?=

Post by Raphaël Halimi
Package: systemd
Version: 226-4
Hi,
"systemd will add an ACL for read permissions for users in the "adm" group."
This is not working: after creating /var/log/journal with the "install"
command as instructed in the README.Debian, and even after several

...

While the idea of shipping /var/log/journal pre-configured in the
package is still an option, I now decided to apply a different fix.

I've cherry-picked two upstream commits which also apply the ACLs to
/var/log/journal (so newly created files inherit them directly) and to
exisiting system.journal files.

I've also updated the instructions in README.Debian (which now match
what's in man systemd-journald(8):

mkdir -p /var/log/journal
systemd-tmpfiles --create --prefix /var/log/journal

Those two command are now sufficient to setup the persistent journal
with the correct permissions and ACLs.

This will be part of the upcoming 228-5 release.