Quantcast

[RFC] Fixing the machine id

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[RFC] Fixing the machine id

Lennart Poettering-2
heya,

one problem the D-Bus machine ID has for usage in various programs is
that it is not available during early boot. Kay and I have been thinking
about this for a while, and the obvious fix is of course to move it from
/var/lib to /etc which is guaranteed to be around right from the
beginning, and is the better place for it anyway, since it is everything
but variable.

Another small issue is that the generation of the id happens at boot
time instead of installation time. (This has the advantage though that
shipping the same OS image is easily doable, becuase they end up using
different is when booted.)

Fixing this second issue is more difficult in /etc, since that directory
is usually read-only during boot, hence we cannot generate the ID
automatically at boot anymore. Or can we?

Thinking about this Kay and I came to the following solution, and I'd
like to ask you for comments on this:

First of all we decided to move the generation of the ID into systemd,
for multiple reasons: it seems like a more appropriate place to maintain
it, given that it is machine and not bus specific; we already maintain
the hostname from /etc/hostname, and hence doing /etc/machine-id too is
only symmetric; we can do mount magic as described below much easier;
and we move the machine id out of the D-Bus context into a distro
context thus hopefully convincing more folks to use it instead of the
host name.

So, here's what systemd now does at boot: when it finds /etc/machine-id
populated, then everything is fine. If it is not populated, it tries to
initialize it from /var/lib/dbus/machine-id, with a fallback on
/dev/urandom. If that works, then everything is fine. If it doesn't but
the file exists and is suitable as mount point we instead store the id
we generated in a file in /dev/.systemd/ and then bind mount it over
/etc/machine-id. If that works, then everything is fine. If that doesn't
work or we cannot create the mount point then we simply fail and give
up.

Yes, on Linux you can bind mount not only directories, but also files.

At package installation time the same algorithm is used, to ensure the
validity of the machine id, at a time where we can actually write to
/etc. If people want to support state-less systems they can simply
create /etc/machine-id empty (so that it is useful as a mount point when
the system boots up and systemd wants to initialize the id) and it will
be mounted over at boot time with a dynamic machine id file

By initializing /etc/machine-id from /var/lib/machine-id we should make
upgrades reliable. On new installations the latter should probably just
be a symlink to the former, since both files use the same format.

What does this mean for D-Bus upstream? Very little, as it is mostly a
downstream decision to adopt systemd and then symlink
/var/lib/machine-id to /etc/machine-id, or not. Other OSes and systems
not using systemd will still continue to use /var/lib/machine-id, and
end up with the very same id. The only D-Bus change we might consider is
to one day teach it to check /etc/machine-id as a fallback or so.

In case you are interested to know what systemd is really doing, here's
the precise logic:

http://cgit.freedesktop.org/systemd/plain/src/machine-id-setup.c

With this in place there's very little left which stops us from spawning
D-Bus already during early boot: the system bus activation needs to be
moved to /usr. Or alternatively we just give up officialyl on seperate
/usr, in which case we don't need to. I'd be very much in favour of
that, but you know the politics. The bigger problem is of course the
system bus socket, which is in /var/run/dbus/. But we probably could
move that to an abstract socket or one in /dev/.dbus, relatively
easily.

Anyway, opinions?

Lennart

--
Lennart Poettering - Red Hat, Inc.
_______________________________________________
dbus mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/dbus
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Fixing the machine id

James-243
On Fri, 2011-03-04 at 22:40 +0100, Lennart Poettering wrote:

> heya,
>
> one problem the D-Bus machine ID has for usage in various programs is
> that it is not available during early boot. Kay and I have been thinking
> about this for a while, and the obvious fix is of course to move it from
> /var/lib to /etc which is guaranteed to be around right from the
> beginning, and is the better place for it anyway, since it is everything
> but variable.
>
> Another small issue is that the generation of the id happens at boot
> time instead of installation time. (This has the advantage though that
> shipping the same OS image is easily doable, becuase they end up using
> different is when booted.)
>
> Fixing this second issue is more difficult in /etc, since that directory
> is usually read-only during boot, hence we cannot generate the ID
> automatically at boot anymore. Or can we?
>
> Thinking about this Kay and I came to the following solution, and I'd
> like to ask you for comments on this:
>
> First of all we decided to move the generation of the ID into systemd,
> for multiple reasons: it seems like a more appropriate place to maintain
> it, given that it is machine and not bus specific; we already maintain
> the hostname from /etc/hostname, and hence doing /etc/machine-id too is
> only symmetric; we can do mount magic as described below much easier;
> and we move the machine id out of the D-Bus context into a distro
> context thus hopefully convincing more folks to use it instead of the
> host name.
>
> So, here's what systemd now does at boot: when it finds /etc/machine-id
> populated, then everything is fine. If it is not populated, it tries to
> initialize it from /var/lib/dbus/machine-id, with a fallback on
> /dev/urandom. If that works, then everything is fine. If it doesn't but
> the file exists and is suitable as mount point we instead store the id
> we generated in a file in /dev/.systemd/ and then bind mount it over
> /etc/machine-id. If that works, then everything is fine. If that doesn't
> work or we cannot create the mount point then we simply fail and give
> up.
The one thing that comes to mind; from my sysadmin point of view is that
files named /dev/.something pop up as warnings in daemons like rkhunter.
Maybe it would be sane to send them a patch so they know to expect these
new things. Maybe a small trivial point, but it's free.

>
> Yes, on Linux you can bind mount not only directories, but also files.
>
> At package installation time the same algorithm is used, to ensure the
> validity of the machine id, at a time where we can actually write to
> /etc. If people want to support state-less systems they can simply
> create /etc/machine-id empty (so that it is useful as a mount point when
> the system boots up and systemd wants to initialize the id) and it will
> be mounted over at boot time with a dynamic machine id file
>
> By initializing /etc/machine-id from /var/lib/machine-id we should make
> upgrades reliable. On new installations the latter should probably just
> be a symlink to the former, since both files use the same format.
>
> What does this mean for D-Bus upstream? Very little, as it is mostly a
> downstream decision to adopt systemd and then symlink
> /var/lib/machine-id to /etc/machine-id, or not. Other OSes and systems
> not using systemd will still continue to use /var/lib/machine-id, and
> end up with the very same id. The only D-Bus change we might consider is
> to one day teach it to check /etc/machine-id as a fallback or so.
>
> In case you are interested to know what systemd is really doing, here's
> the precise logic:
>
> http://cgit.freedesktop.org/systemd/plain/src/machine-id-setup.c
>
> With this in place there's very little left which stops us from spawning
> D-Bus already during early boot: the system bus activation needs to be
> moved to /usr. Or alternatively we just give up officialyl on seperate
> /usr, in which case we don't need to. I'd be very much in favour of
> that, but you know the politics. The bigger problem is of course the
> system bus socket, which is in /var/run/dbus/. But we probably could
> move that to an abstract socket or one in /dev/.dbus, relatively
> easily.
>
> Anyway, opinions?
>
> Lennart
>


_______________________________________________
dbus mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/dbus
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Fixing the machine id

Lennart Poettering-2
On Fri, 04.03.11 17:38, James ([hidden email]) wrote:

> > So, here's what systemd now does at boot: when it finds /etc/machine-id
> > populated, then everything is fine. If it is not populated, it tries to
> > initialize it from /var/lib/dbus/machine-id, with a fallback on
> > /dev/urandom. If that works, then everything is fine. If it doesn't but
> > the file exists and is suitable as mount point we instead store the id
> > we generated in a file in /dev/.systemd/ and then bind mount it over
> > /etc/machine-id. If that works, then everything is fine. If that doesn't
> > work or we cannot create the mount point then we simply fail and give
> > up.
>
> The one thing that comes to mind; from my sysadmin point of view is that
> files named /dev/.something pop up as warnings in daemons like rkhunter.
> Maybe it would be sane to send them a patch so they know to expect these
> new things. Maybe a small trivial point, but it's free.

Well, it's kinda common already: udev, MD and systemd have been doing
this for a while, and mount now too.

Lennart

--
Lennart Poettering - Red Hat, Inc.
_______________________________________________
dbus mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/dbus
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Fixing the machine id

Marcel Holtmann
In reply to this post by Lennart Poettering-2
Hi Lennart,

> With this in place there's very little left which stops us from spawning
> D-Bus already during early boot: the system bus activation needs to be
> moved to /usr. Or alternatively we just give up officialyl on seperate
> /usr, in which case we don't need to. I'd be very much in favour of
> that, but you know the politics. The bigger problem is of course the
> system bus socket, which is in /var/run/dbus/. But we probably could
> move that to an abstract socket or one in /dev/.dbus, relatively
> easily.

using the abstract socket is bad for security since you have no
permission checks anymore. And essentially anybody could start owning
that socket. You do not really want that.

Also the system socket address is pretty much hardcoded. At least when I
looked through that a few weeks ago. And then changing the address
becomes quickly big trouble. Only advantage with systemd would be that
you can easily set an environment variable for every process. So it
might work out for you. In general it is a big mess.

Regards

Marcel


_______________________________________________
dbus mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/dbus
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Fixing the machine id

Bugzilla from mbiebl@gmail.com
2011/3/5 Marcel Holtmann <[hidden email]>:

> Hi Lennart,
>
>> With this in place there's very little left which stops us from spawning
>> D-Bus already during early boot: the system bus activation needs to be
>> moved to /usr. Or alternatively we just give up officialyl on seperate
>> /usr, in which case we don't need to. I'd be very much in favour of
>> that, but you know the politics. The bigger problem is of course the
>> system bus socket, which is in /var/run/dbus/. But we probably could
>> move that to an abstract socket or one in /dev/.dbus, relatively
>> easily.
>
> using the abstract socket is bad for security since you have no
> permission checks anymore. And essentially anybody could start owning
> that socket. You do not really want that.
>
> Also the system socket address is pretty much hardcoded. At least when I
> looked through that a few weeks ago. And then changing the address
> becomes quickly big trouble. Only advantage with systemd would be that
> you can easily set an environment variable for every process. So it
> might work out for you. In general it is a big mess.

Given that systemd mounts a tmpfs on /var/run very early during boot,
is there actually need to change the location of the socket?
Which (D-Bus) service would you want to start before /var/run is writable?

Michael


--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
_______________________________________________
dbus mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/dbus
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Fixing the machine id

Kay Sievers
On Sat, Mar 5, 2011 at 03:09, Michael Biebl <[hidden email]> wrote:

> 2011/3/5 Marcel Holtmann <[hidden email]>:
>> Hi Lennart,
>>
>>> With this in place there's very little left which stops us from spawning
>>> D-Bus already during early boot: the system bus activation needs to be
>>> moved to /usr. Or alternatively we just give up officialyl on seperate
>>> /usr, in which case we don't need to. I'd be very much in favour of
>>> that, but you know the politics. The bigger problem is of course the
>>> system bus socket, which is in /var/run/dbus/. But we probably could
>>> move that to an abstract socket or one in /dev/.dbus, relatively
>>> easily.
>>
>> using the abstract socket is bad for security since you have no
>> permission checks anymore. And essentially anybody could start owning
>> that socket. You do not really want that.
>>
>> Also the system socket address is pretty much hardcoded. At least when I
>> looked through that a few weeks ago. And then changing the address
>> becomes quickly big trouble. Only advantage with systemd would be that
>> you can easily set an environment variable for every process. So it
>> might work out for you. In general it is a big mess.
>
> Given that systemd mounts a tmpfs on /var/run very early during boot,
> is there actually need to change the location of the socket?
> Which (D-Bus) service would you want to start before /var/run is writable?

/var might be on a different device, so we can't mount /var/run that
time, but we would like to have the socket available.

Kay
_______________________________________________
dbus mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/dbus
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Fixing the machine id

Lennart Poettering-2
In reply to this post by Marcel Holtmann
On Fri, 04.03.11 17:45, Marcel Holtmann ([hidden email]) wrote:

>
> Hi Lennart,
>
> > With this in place there's very little left which stops us from spawning
> > D-Bus already during early boot: the system bus activation needs to be
> > moved to /usr. Or alternatively we just give up officialyl on seperate
> > /usr, in which case we don't need to. I'd be very much in favour of
> > that, but you know the politics. The bigger problem is of course the
> > system bus socket, which is in /var/run/dbus/. But we probably could
> > move that to an abstract socket or one in /dev/.dbus, relatively
> > easily.
>
> using the abstract socket is bad for security since you have no
> permission checks anymore. And essentially anybody could start owning
> that socket. You do not really want that.

Well, the system bus socket is accessible by everybody anyway, and
authentication is based on SCM_CREDENTIALS mostly anyway. Note that on
Linux the session bus (i.e. the bus that is private to the user)
actaully does use a (random) abstract namespace socket.

I think D-Bus is one of the very few cases where fixed name abstract
namespace sockets are safe: after all we do this all to be able to start
it as early as possible, and that means at a time where unprivileged
code doesn't run yet. Also, D-Bus is not restartable. FInally, in a
socket based environment it could even be restartable, but the socket
could never be taken away from systemd and dbus.

> Also the system socket address is pretty much hardcoded. At least when I
> looked through that a few weeks ago. And then changing the address
> becomes quickly big trouble. Only advantage with systemd would be that
> you can easily set an environment variable for every process. So it
> might work out for you. In general it is a big mess.

Well, other options thinkable are:

a) bind mount the socket into /var/run as soon as that is available.

b) teach dbus to listen on two sockets, and not fail if it cannot create
one. Then, send it a sighup or so to retry to establish all sockets.

c) Do the passthru magic ubuntu is currently doing, which i find kinda
messy, but might be an option after all: during early boot already mount
/var/run and /var/lock, but into a fake /var. Then, after mounting the
real /var into a temporary mount point, bind mount both into them, and
finally atomically move the temporary mount point to the real place,
thus always guaranteeing that /var/run and /var/lock are available all
the time.

d) introduce /dev/.systemd/run and /dev/.systemd/lock, and make /var/run
and /var/lock just a bind mount alias of that. Or, do it even smarter:
when booting up, make /dev/.systemd/{run,lock} symlinks to
/var/{run,lock}, then mount the tmpfs on that so that the symlinks are
not visible anymore. Finally, when /var is up, just atomically move the
mount from /dev/.systemd/{run,lock} to /var/{run,lock}, revealing the
symlinks again. That way everybody in need of /var/run during early boot
can access it via /dev/.systemd/run. And it will stay accesible through
that all the time.

I kinda like the idea of d).

Lennart

--
Lennart Poettering - Red Hat, Inc.
_______________________________________________
dbus mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/dbus
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Fixing the machine id

David Zeuthen-4
Hi,

On Sat, Mar 5, 2011 at 9:46 AM, Lennart Poettering <[hidden email]> wrote:
>> using the abstract socket is bad for security since you have no
>> permission checks anymore. And essentially anybody could start owning
>> that socket. You do not really want that.
>
> Well, the system bus socket is accessible by everybody anyway, and
> authentication is based on SCM_CREDENTIALS mostly anyway.

Doesn't matter. The message bus client process does not check the
credentials of the message bus process (in libdbus-1 and GDBus anyway)
- only the message bus process checks the credentials of the client.
So, yes, it's entirely possible for any random process to take over
this abstract socket and pretend to be the system message bus. Of
course the system message bus would need to somehow crash and that
rarely happens.

We could add bilateral authentication, sure, but that's just not how
it works right now.

    David
_______________________________________________
dbus mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/dbus
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Fixing the machine id

Kay Sievers
On Sat, Mar 5, 2011 at 16:12, David Zeuthen <[hidden email]> wrote:

> On Sat, Mar 5, 2011 at 9:46 AM, Lennart Poettering <[hidden email]> wrote:
>>> using the abstract socket is bad for security since you have no
>>> permission checks anymore. And essentially anybody could start owning
>>> that socket. You do not really want that.
>>
>> Well, the system bus socket is accessible by everybody anyway, and
>> authentication is based on SCM_CREDENTIALS mostly anyway.
>
> Doesn't matter. The message bus client process does not check the
> credentials of the message bus process (in libdbus-1 and GDBus anyway)
> - only the message bus process checks the credentials of the client.
> So, yes, it's entirely possible for any random process to take over
> this abstract socket and pretend to be the system message bus. Of
> course the system message bus would need to somehow crash and that
> rarely happens.
>
> We could add bilateral authentication, sure, but that's just not how
> it works right now.

But we bind the socket with pid 1, and pass it to D-Bus at service
startup, don't we? Even if D-Bus crashes, it should not be a problem,
because no other process can ever take that over. Am I missing
something?

Kay
_______________________________________________
dbus mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/dbus
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Fixing the machine id

Colin Walters
In reply to this post by Lennart Poettering-2
On Fri, Mar 4, 2011 at 4:40 PM, Lennart Poettering <[hidden email]> wrote:
> heya,
>
> one problem the D-Bus machine ID has for usage in various programs is
> that it is not available during early boot. Kay and I have been thinking
> about this for a while, and the obvious fix is of course to move it from
> /var/lib to /etc which is guaranteed to be around right from the
> beginning, and is the better place for it anyway, since it is everything
> but variable.

Sounds great!  One comment:

> So, here's what systemd now does at boot: when it finds /etc/machine-id
> populated, then everything is fine. If it is not populated, it tries to
> initialize it from /var/lib/dbus/machine-id, with a fallback on
> /dev/urandom.

My one concern is that if systemd has a /dev/urandom fallback, and
it's silently used in the case of OS misconfiguration, it could make
consumers of it unreliable.  Someone posted a DBus patch to get rid of
the file and just autogenerate it each time, but one of the major
points of this ID *is* that it's persistent; if your app or library
needs to store machine-specific state, you really don't want it to be
lost on reboot.

Or is systemd going to create /etc/machine-id from /dev/urandom
persistently?  In that case, sounds good.
_______________________________________________
dbus mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/dbus
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Fixing the machine id

Lennart Poettering-2
On Wed, 09.03.11 13:25, Colin Walters ([hidden email]) wrote:

Heya,

> > So, here's what systemd now does at boot: when it finds /etc/machine-id
> > populated, then everything is fine. If it is not populated, it tries to
> > initialize it from /var/lib/dbus/machine-id, with a fallback on
> > /dev/urandom.
>
> My one concern is that if systemd has a /dev/urandom fallback, and
> it's silently used in the case of OS misconfiguration, it could make
> consumers of it unreliable.  Someone posted a DBus patch to get rid of
> the file and just autogenerate it each time, but one of the major
> points of this ID *is* that it's persistent; if your app or library
> needs to store machine-specific state, you really don't want it to be
> lost on reboot.
>
> Or is systemd going to create /etc/machine-id from /dev/urandom
> persistently?  In that case, sounds good.

The rule is: if there's a way to make it persistant, we make it
persistant. But usually there isn't if we boot from CDROM, for
example...

Lennart

--
Lennart Poettering - Red Hat, Inc.
_______________________________________________
dbus mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/dbus
Loading...