init: support logging of job output

e.g. to log file, syslog, e-mail (cron support) also see bug #327823

Three bug reports that were started before this bug report have been marked as duplicates of this bug.

I understood that later bugs were marked as duplicates of earlier bugs; that way the activity log is kept in one place.

The three bug reports marked as duplicates of this bug are 69771, 98955 and 37254. This is noted here for ease of cross-reference in case the situation is changed subsequent to this comment.

There are actually three bugs reported in the bug reports marked as duplicates of this bug.

In date order:

1) There is no shutdown logging facility. 2006-11-01. https://bugs.launchpad.net/upstart/+bug/69771.
2) Upstart boot logging is broken. 2007-03-30. https://bugs.launchpad.net/upstart/+bug/98955 /sbin/logd is not started since Gutsy. 2009-02-13. https://bugs.launchpad.net/upstart/+bug/328881.
3) There is no boot logging mechanism. 2007-09-04. https://bugs.launchpad.net/ubuntu/+source/ubuntu-meta/+bug/137254.

Upstart boot logging has been broken since before 2007-03-30, more than 2 years ago. I guess it must be a combination of intractable and low priority. Given that situation, please can the maintainers consider supporting a workaround such as the popular but unsupported bootlogd_2.86.02 (http://launchpadlibrarian.net/11578525/bootlogd_2.86.02.tar.gz)? AIUI a relatively small change would allow it to provide shutdown logging to, for example. /var/log/shutdown.

Yes, Yes and again YES

many people are discussing on many places about this logging problem.
I also WILL to have /var/log/boot .. shutdown !!!

Can we make some kind of pressure ?

How ?

Where?

(sorry please my English)

Thanks to Sergei Ivanov, there is an unsupported workaround that provides boot logging. See https://bugs.launchpad.net/upstart/+bug/98955/comments/29 and later.

On Mon, 2009-08-10 at 06:58 +0000, kapetr wrote:

> Yes, Yes and again YES
>
> many people are discussing on many places about this logging problem.
> I also WILL to have /var/log/boot .. shutdown !!!
>
> Can we make some kind of pressure ?
>
No.

> How ?
>
> Where?
>
You could do the work yourself and contribute the necessary support in
the form of patches.

Scott
--
Scott James Remnant
<email address hidden>

Hello,

"do it yourself" - i would like, but it is out of my knowledge.

I had try to get binary package "sysvinit_2.86.ds1-61_i386.deb" from Dedian, I had extractet just the /sbin/bootlogd binary (ldd ... says: all OK) + I had created /etc/rcS.d/60bootlogd link to /etc/init.d/bootlogd.

Now is bootlogd running, bu unfortunately it do not write anything to /var/log/boot (or other with "-l" defind file) - it just create it.

I don't know why :-( Any Idea ? (/etc/default/bootlogd is set to Yes)

To compile above discussed "workaround" - Is seems to me to be too dangerous - well: the bootlogd is running with root privileges.

Hello kapetr :)

This could turn into a discussion that would be inappropriate in a bug report so I have started a thread on Linux Questions at http://www.linuxquestions.org/questions/linux-general-1/howto-workaround-broken-upstart-boot-logging-746468/#post3637957.

Best

Charles

So what is the reason for this being the master bug and not https://bugs.launchpad.net/upstart/+bug/98955
observing that bug 98955 is much older and has much more history comments, and many many more subscribers?

While I agree that #98955 seems like a better candidate for the master bug, there's three issues:
1) Scott has much more experience and authority than me, and he's the one who marked the duplicate
2) Scott opened #98955 in the first place
3) The issue is to solve the bug, marking duplicates and master bugs is secondary

@Dimitrios Symeonidis
The primary function of bug reports is to solve the problem but an important secondary function is to provide workaround information pending solution. In the case of long-standing bugs like this one, that secondary function assumes more importance.

On Mon, 2009-08-17 at 14:52 +0000, Charles Atkinson wrote:

> @Dimitrios Symeonidis
> The primary function of bug reports is to solve the problem but an important secondary function is to provide workaround information pending solution. In the case of long-standing bugs like this one, that secondary function assumes more importance.
>
Again your assertion is utterly incorrect. The ONLY purpose of a bug
report is to report a problem to developers, provide them enough
information for them to be able to replicate it, and provide a place for
developers to discuss any fixes and call for testing.

There is NO "secondary function" such as you suggest; in fact,
suggesting workarounds to people is the WORST thing you can do. It
means that those people can no longer help with the report, because
they're no longer affected. And it means that when a correct fix does
arrive, they will not be upgraded to it. In the worst situations, they
may be left with a broken system.

Scott
--
Scott James Remnant
<email address hidden>

Thank you, Scott!

As the proud owner of a buggy system, that often fails to boot, leaving me with a garbled and unreadable kernel panic message, this "Wishlist" item is not so trivial. As a struggling newbie, reading that I should go backwards and put sysvinit back in action, using instructions I don't understand, is less than confidence inspiring.

That this bug is considered a Wishlist item is ridiculous, upstart was clearly not ready for the job... I don't really understand what the getty is, but I am pretty sure nothing should be spewed all over it.

That the only option for anyone wishing to check the health of their startup or shutdown, whether they be newbie or elite, is to have to rig it, is absurd in the extreme.

In a perfect world, startup and shutdown messages would be optional, because the perfect system would never have problems and reading those messages would be for educational purposes only. It is not a perfect world, not a perfect operating system and those messages are NOT optional.

Erica

On Mon, 2009-08-24 at 15:28 +0000, HousieMousie2 wrote:

> That this bug is considered a Wishlist item is ridiculous, upstart was
> clearly not ready for the job... I don't really understand what the
> getty is, but I am pretty sure nothing should be spewed all over it.
>
It's a wishlist for Upstart; the bug of log messages being random and
lost during boot is unrelated to Upstart and affected sysvinit before we
introduced it.

Upstart is simply where we'll fix the bug.

Scott
--
Scott James Remnant
<email address hidden>

Thank you again, Scott! :-)

I had not seen anyone mention that the problem existed prior to Upstart, only that Upstart (more or less) replaced sysvinit, and that before Upstart log.d was working. My apologies if I simply missed it.

I wonder then why putting sysvinit back is suggested as a good workaround. Keep booting and hope that the right telltale message is logged?

May I ask why it is a Wishlist item though when these logs are terribly important?

Erica

sysvinit is the workaround because patches at least exist for it to do
logging. Upstart currently lacks the capability altogether. Being a feature
request (ie "please teach upstart how to log" as opposed to "upstart forgot
how to log") is what makes it Wishlist.

I just searched launchpad for "bootlogd" (without "misleading" to exclude reports about bootlogd config files being installed without bootlogd itself) and did not find anything about bootlogd being significantly broken. There were some references to it not working when launched from a terminal other than the console but I didn't find the actual report and suspect that is anyway a rare situation.

Boot logging is an important diagnosis tool; upstart boot logging cannot safely be enabled (so cannot be used to do any logging); and bootlogd is working satisfactorily for many of us. For these reasons I re-iterate: "Please, maintainers, consider supporting a workaround such as the popular but unsupported bootlogd_2.86.02".

Ah, okay, I get it. Thank you, Mackenzie!

On Mon, 2009-08-24 at 18:43 +0000, Charles Atkinson wrote:

> "Please, maintainers, consider supporting a workaround such as the
> popular but unsupported bootlogd_2.86.02".
>
What makes you think that this isn't getting attention?

Clearly the bug is open and not simply closed.

Clearly a developer is subscribed to it (hi!)

Fixing boot logging is on my TODO list, but it's about item 246 right
now and probably in the karmic+2 timeframe.

Rather than arguing about which bug should be a duplicate of which, and
the purpose of the bug tracking system, etc. you could actually help out
address the problem either by working to make bootlogd installable (and
work with usplash, which it never has!) or working on Upstart upstream
to contribute the necessary logging support

Scott
--
Scott James Remnant
<email address hidden>

@HousieMousie2 Please also remember that this is a bug on upstart. Your comments seem more appropriate on the OS you are using (probably ubuntu :-) )
There are bugs about missing boot log message in ubuntu, these however are not wishlists, because ubuntu have had that feature, please see bug 34165 (https://bugs.launchpad.net/ubuntu/+source/boot/+bug/34165). Nevertheless, this ubuntu bug doesn't seem to get much attention, too sad. Then again what attention should it get if the plan to fix it in ubuntu is to stay with upstart and fix it in upstart.

Jarl

In view of Jarl's comment I have commented on Scott's comment at https://bugs.launchpad.net/ubuntu/+source/boot/+bug/34165/comments/3

Note that bootlogd is now in the Ubuntu sysvinit-utils package, so the brave can have a go at running that. I'm still convinced it won't work though ;-) but if you can make it work, please do document it for others.

Scott James Remnant wrote in comment 19:
> What makes you think that this isn't getting attention?
> Clearly the bug is open and not simply closed.
> Clearly a developer is subscribed to it (hi!)

Are you also asigned the bug, if so, could you please update the assigned to field.

> Fixing boot logging is on my TODO list, but it's about item 246 right now and probably in the karmic+2 timeframe.

That's sad.

On Fri, 2009-11-27 at 12:56 +0000, Jarl wrote:

> Scott James Remnant wrote in comment 19:
> > What makes you think that this isn't getting attention?
> > Clearly the bug is open and not simply closed.
> > Clearly a developer is subscribed to it (hi!)
>
> Are you also asigned the bug, if so, could you please update the
> assigned to field.
>
I am not assigned to the bug - I tend to prefer not to mark bugs that
I'm not actively working on as assigned to me, because it discourages
anybody else from picking it up.

> > Fixing boot logging is on my TODO list, but it's about item 246 right
> now and probably in the karmic+2 timeframe.
>
> That's sad.
>
The way to fix that is for another developer (e.g. a community one) to
pick it up.

Scott
--
Have you ever, ever felt like this?
Had strange things happen? Are you going round the twist?

> The way to fix that is for another developer (e.g. a community one) to
> pick it up.

I'm so annoyed of not having boot logging that I might give it a shot at doing some C programming myself, but after "bzr branch lp:upstar", i didn't know where to start hacking :-). That might mean I'm not the proper person to attempt this, however if anyone can point out where I can start losing my time in the source code, I might get a little encouraged and try again :-)

One idea I had, for instance: I see that boot messages are sent to tty8, so my first idea would be to look at the code that does that and add functionality there to redirect a copy of this output to a file in /var/log/ or something like that. Maybe I'm completely clueless? Any pointers

For sanity's sake, I'm closing the Ubuntu tasks for upstream Upstart bugs. I've experimented with having both, but it is just making bugs hard to find now. Will use the policy whereby bugs on the Ubuntu package exist in the Ubuntu packaging or patches only, any bugs in the Upstart code are Upstream bugs.

This is a question from a third party software developer: I've to support a legacy application
package which used to add an entry to /etc/inittab on traditional Unix/Linux systems to
start (and respawn) a certain demon.

When Ubuntu moved to upstart I added a /etc/event.d/-file template. Later I was forced to
add an additional *.conf template for /etc/init directory. My question is: What do I have to put
into these files to get the stderr output of this demon into the syslog on all versions of
Ubuntu? (we have a lot of 8.04 LTS still in use in the field; moving slowly to 10.04 LTS)
For me it is very difficult to figure this out. Thanks in advance.
Is installing a backported upstart package a viable option? How much risk is that?

On Fri, 2011-03-11 at 07:12 +0000, Peter Funk wrote:
> This is a question from a third party software developer: I've to support a legacy application
> package which used to add an entry to /etc/inittab on traditional Unix/Linux systems to
> start (and respawn) a certain demon.
>
> When Ubuntu moved to upstart I added a /etc/event.d/-file template. Later I was forced to
> add an additional *.conf template for /etc/init directory. My question is: What do I have to put
> into these files to get the stderr output of this demon into the syslog on all versions of
> Ubuntu? (we have a lot of 8.04 LTS still in use in the field; moving slowly to 10.04 LTS)
> For me it is very difficult to figure this out. Thanks in advance.
> Is installing a backported upstart package a viable option? How much risk is that?
>

Peter, have you tried piping the output of the executed command to
logger?

As in

exec /usr/sbin/foobard | logger -t foobard

?

Clint Byrum wrote 11.03.2011 10:05:
> Peter, have you tried piping the output of the executed command to
> logger?
>
> As in
>
> exec /usr/sbin/foobard | logger -t foobard

Not yet: This is a clever idea. I will try this.
Thank you very much.

Best regards, Peter
--
Peter Funk, home: ✉Oldenburger Str.86, D-27777 Ganderkesee
mobile:+49-179-640-8878 phone:+49-421-20419-0 <http://www.artcom-gmbh.de/>
office: ArtCom GmbH, ✉Haferwende 2, D-28357 Bremen, Germany

Upstart 1.4 introduces "console log". See init(5) and:

  http://upstart.ubuntu.com/cookbook/#console-log

...and: http://upstart.ubuntu.com/cookbook/#stanzas-by-category

console log seems broken for me, here is some info...

using the following file saved as /etc/init/web-proxy.conf:

   description "web-proxy"
   author "Ankur Oberoi"

   start on (local-filesystems and net-device-up IFACE=eth0)
   stop on shutdown

   console log
   exec /usr/local/bin/node /srv/proxy/server.js

This file fails to show up when I run `sudo initctl reload-configuration | grep web` but if I comment out the console line, it does show up and moreover it works correctly.

I compiled from the source tarball for 1.4 on Oneiric 11.10 on x86_64. If there is any more helpful info I can provide, please let me know. I'd really like the console log feature!

@Ankur: Do you mean the job only starts if you comment out the "console log"? Have you manually created /var/log/upstart/ ?

upstart 1.4 has landed in Ubuntu precise.