Document all events (was: cross-comment events and *.conf files)

Hi Mike,

Thanks for reporting this and be reassured that we are investigating this suggestion. Natty should ship with a few new man pages for upstart, including one currently entitled upstart-events. You can see a preview of it here:

  http://people.canonical.com/~jhunt/upstart/man/upstart-events.pdf

Caveat: this page is beta and displays poorly via PDF. If you want to view it properly:

  wget http://people.canonical.com/~jhunt/upstart/man/upstart-events.7
  tbl upstart-events.7 | nroff -man | less

The table summarizes all "well-known" events and should negate the need to embed references to man pages in .conf files.

Additionally the version of upstart shipping with natty should include basic "visualisation" of jobs such that upstart itself will be able to display the links between events and jobs.

I think the message here is that there should be a man page or alias to a man page for every event. As James says, that is being done with the upstart-events man page. So I think on that alone, this bug can be Confirmed and set to Wishlist. James, if there is another bug where you've been tracking the work, can you mark this one as a duplicate of it?

As far as the assertions, I think we have a method of making sure that required files are there in the packaging system. The package upstart, which owns /etc/init/rc-sysinit.conf, depends on mountall and ifupdown. Further, there may be situations where something starts on an event that is only generated by something it is not dependent on. For instance, you may only want to run task foo when a service with NEEDS_FOO=1 starts.

start on starting NEEDS_FOO=1

The loose coupling gained by *not* making things too rigid is I think part of the power of the system. The 'emits' keyword is sort of like assertions, but on the other side. Right now, there's no enforcement of emits, so they're prone to rotting over time. With the visualization stuff that James Hunt has been working on (its really, REALLY cool.. James, please get it out soon! :) You'll see that your event isn't pointed to by any 'emits', or that your job will point to an event that isn't consumed by anything.

On Fri, Feb 04, 2011 at 06:17:30PM -0000, Clint Byrum wrote:
> As far as the assertions, I think we have a method of making sure that
> required files are there in the packaging system. The package upstart,
> which owns /etc/init/rc-sysinit.conf, depends on mountall and ifupdown.
> Further, there may be situations where something starts on an event that
> is only generated by something it is not dependent on. For instance, you
> may only want to run task foo when a service with NEEDS_FOO=1 starts.

Let me respectfully disagree in the case where files _must_ be present
and events _must_ be asserted.

If a file is accidentally removed, renamed, or becomes unaccessible _and_
is necessary for a successful boot, I want /bin/init or whatever to scream
loud and long. The best place for that is in the *.conf stanza that depends
upon the event that the missing file is supposed to assert.

> The loose coupling gained by *not* making things too rigid is I think
> part of the power of the system.

Yes. Where events are optional, then we don't want hard checks.

However if there was a double check on the file existing _and_ the event
happening, then there would be an enforced documentation of both the event
and where it was expected to come from, as a "good programming practice".

More than one source of an event? Check for the existence of at least one
of the source files or commands.

> The 'emits' keyword is sort of like
> assertions, but on the other side. Right now, there's no enforcement of
> emits, so they're prone to rotting over time.

All the more reason to have a programmed means to couple the emit with the
event's sensitivity.

-

Let me say again that this whole mess makes me nervous. In the good old
days if I could come up with a strictly sequential order of
/etc/init.d/S[0-9][0-9]* scripts that worked reliably, I was done!

It would work every time because A would be followed by B etc. all the way
down to the end. Getting the strict relationship of all events and all
dependencies correct is (for now) proving to be non-trivial.

I suggest once more that having the _option_ of having a strictly sequential
boot sequence for those of us that are not sensitive to how long it takes
the machine to boot is _very_ desirable.

Why is it desirable? Because I have been burned repeatedly by upstart and
so I am still on Ubuntu 8.04. I understand /etc/init.d/S[0-9][0-9]* .
I don't yet understand /etc/init/*.conf .

Mike Bianchi

This is an interesting message and I think deserves broader discussion,
so I'm CC'ing upstart-devel.

On Fri, 2011-02-04 at 19:51 +0000, Mike Bianchi wrote:
> On Fri, Feb 04, 2011 at 06:17:30PM -0000, Clint Byrum wrote:
> > As far as the assertions, I think we have a method of making sure that
> > required files are there in the packaging system. The package upstart,
> > which owns /etc/init/rc-sysinit.conf, depends on mountall and ifupdown.
> > Further, there may be situations where something starts on an event that
> > is only generated by something it is not dependent on. For instance, you
> > may only want to run task foo when a service with NEEDS_FOO=1 starts.
>
> Let me respectfully disagree in the case where files _must_ be present
> and events _must_ be asserted.
>
> If a file is accidentally removed, renamed, or becomes unaccessible _and_
> is necessary for a successful boot, I want /bin/init or whatever to scream
> loud and long. The best place for that is in the *.conf stanza that depends
> upon the event that the missing file is supposed to assert.
>

Complaining loudly is different than asserting, IMO, so I may have just
misunderstood your intent. To me upstart is as much a library as it is a
program most of the time, so asserting and stopping everything /
crashing is a lot different than warning/crying about an obviously
broken system.

I'd be pretty happy if we had something that scans upstart's jobs, and
complains during postinst of a new upstart job:

"W: event X, used by job Y, has no emits clause"

I think James Hunt's visualization tool will add the right hooks to make
this possible.

>
> > The loose coupling gained by *not* making things too rigid is I think
> > part of the power of the system.
>
> Yes. Where events are optional, then we don't want hard checks.
>
> However if there was a double check on the file existing _and_ the event
> happening, then there would be an enforced documentation of both the event
> and where it was expected to come from, as a "good programming practice".
>
> More than one source of an event? Check for the existence of at least one
> of the source files or commands.
>

Agreed on principle that having something like this will be beneficial.
I'm coming around to the idea of a keyword

start on required foo

Which would at tell upstart to check for emits after parsing all
configs, and warn about the situation.

I still question the value of enforcing this. Rather, I don't know if
there are mechanics for it. What exactly are you going to refuse to do
if there's no emitter for an event? Are you going to just start the job
anyway? That could make things *worse* by not waiting for necessary
resources. Not start or stop at all? Well thats what happens now. Exit
and panic the system? That seems a bit counter-productive to me.

>
> > The 'emits' keyword is sort of like
> > assertions, but on the other side. Right now, there's no enforcement of
> > emits, so they're prone to rotting over time.
>
> All the more reason to have a programmed means to couple the emit with the
> event's sensitivity.
>

Right, so if we at least throw warnings when nothing claims to emit
something that is st...

So one interesting thing about this is that if you design a system
that complains about its configuration during boot, then you're
complaining at the wrong person, the user. The user has no idea how
the system boots, and should have no need to know.

The system should boot regardless, and it _should_ be possible to
design an Upstart boot sequence such that even if certain things don't
happen, other things happen instead. We obviously haven't done that
yet.

That being said, I'm all for an external tool to verify a
configuration, which operating systems could run as part of release
testing, etc. Distributions like Ubuntu could run it after an upgrade
to check for errors, and rollback the upgrade if the resulting
configuration would not boot.

Scott

On Fri, Feb 4, 2011 at 1:02 PM, Clint Byrum <email address hidden> wrote:
> This is an interesting message and I think deserves broader discussion,
> so I'm CC'ing upstart-devel.
>
> On Fri, 2011-02-04 at 19:51 +0000, Mike Bianchi wrote:
>> On Fri, Feb 04, 2011 at 06:17:30PM -0000, Clint Byrum wrote:
>> > As far as the assertions, I think we have a method of making sure that
>> > required files are there in the packaging system. The package upstart,
>> > which owns /etc/init/rc-sysinit.conf, depends on mountall and ifupdown

> Clint Byrum wrote in comment #4 ...
> Complaining loudly is different than asserting, IMO, so I may have just
> misunderstood your intent.

My intent is to have a trail of crumbs when the expected events fail to happen.
If the boot sequence gets to runlevel 2 that is very different from
  runlevel unknown . The OS may be up, at it could be argued that it is not
running properly. It would be nice if it said so.

Maybe there should be a final *.conf that says

        assert event foo has happened
        assert runlevel 2
        assert mount /home
                :

> I'm coming around to the idea of a keyword
>
> start on required foo
>
> Which would at tell upstart to check for emits after parsing all
> configs, and warn about the situation.
>
> I still question the value of enforcing this.

A console and/or log entry of the form
        /etc/init/glurp.conf: failed: no required event foo
would be very useful, whether that failure was severe or just unfortunate.

A further analysis of
        /etc/init/glurp.conf: required event foo: no emit found
would also help.

A stanza of the form
        start on required foo from /etc/init/whiz.conf
might yield
        /etc/init/glurp.conf: /etc/init/whiz.conf: not found
   or
        /etc/init/glurp.conf: /etc/init/whiz.conf: does not emit foo

> Scott James Remnant wrote in comment #5
> ... a system that complains about its configuration during boot, [is]
> complaining at the wrong person, the user.

Agreed. But at least it complains. The recent problems have been boot
sequences that _appear_ to be complete but in fact are not.

Hi Mike,

Hopefully the newly introduced show-config and check-config commands go some way towards providing what you are looking for:

https://wiki.ubuntu.com/NattyNarwhal/TechnicalOverviewUpstart#New_Initctl_Commands
http://upstart.ubuntu.com/cookbook/#initctl-show-config
http://upstart.ubuntu.com/cookbook/#initctl-check-config

Regards,

James.

Wow! After the long silence in this bug I figured nothing was happening.
It wasn't linked to anyplace where the ongoing discussion of Upstart's
evolution was clearly happening.

I have a lot of reading to do.

I would like acknowledgment of my contribution in the Cookbook.
I'm sure there are others who also deserve recognition.