upstart jobs do not respect /etc/security/limits.conf

Here, an additional test Test. A new / clean Ubuntu Lucid installation:

root@ts3:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 10.04.4 LTS
Release: 10.04
Codename: lucid

I set this limits in /etc/security/limits.conf
root hard nofile 65536
root soft nofile 65536

root@ts3:~# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 20
file size (blocks, -f) unlimited
pending signals (-i) 16382
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65536
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) unlimited
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

root@ts3:~# dpkg -l "cups*"
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Description
+++-==============-==============-============================================
ii cups 1.4.3-1ubuntu1 Common UNIX Printing System(tm) - server
un cups-bsd <none> (no description available)
ii cups-client 1.4.3-1ubuntu1 Common UNIX Printing System(tm) - client pro
ii cups-common 1.4.3-1ubuntu1 Common UNIX Printing System(tm) - common fil
ii cups-driver-gu 5.2.5-0ubuntu1 printer drivers for CUPS
un cups-pdf <none> (no description available)
un cups-ppdc <none> (no description available)
un cups-pt <none> (no description available)
un cupsddk <none> (no description available)
un cupsddk-driver <none> (no description available)
un cupsomatic-ppd <none> (no description available)
un cupsys <none> (no description available)
un cupsys-bsd <none> (no description available)
un cupsys-client <none> (no description available)
un cupsys-common <none> (no description available)
un cupsys-driver- <none> (no description available)

CPUS changes in /etc/cups/cupsd.conf:
LogLevel debug
MaxClients 32768

CUPS 1.4.3-1ubuntu1 works fine, like expected:

root@ts3:~# grep Max /var/log/cups/error_log
I [22/Feb/2012:18:17:23 +0100] MaxClients limited to 1/3 (21845) of the file descriptor limit (65536)...

then i change the /etc/apt/sources.list from lucid to maverick and reinstall CUPS
apt-get install cups
...
The following packages will be upgraded:
cups cups-client cups-common libcups2

root@ts3:~# dpkg -l "cups*"
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Description
+++-==============-==============-============================================
ii cups ...

Finaly, i found the solution for this problem.

Since Ubuntu Maverick CUPS will by startet by Upstart instead of init.d script. Unfortunately Upstart ignores the limits set in /etc/security/limits.conf. The solution for this problem is to set the limits in Upstart CUPS config file /etc/init/cups.conf e.g.:

limit nofile 4096 4096

After CUPS restart "/etc/init.d/cups restart" limits settings are back correctly:

grep Max /var/log/cups/error_log
I [23/Feb/2012:12:41:56 +0100] MaxClients limited to 1/3 (1365) of the file descriptor limit (4096)...

BR Pawel

While "limit" stanza is a solution for system jobs, it doesn't work for user jobs if I want to increase hard limits. So far I haven't found any way to increase hard limits for user jobs.
I think, when starting a user job, upstart should apply limits from limits.conf using PAM or directly.

It's by design that upstart does not look at /etc/security/limits.conf for system jobs. PAM settings are only applied to user sessions, not to system services.

As Alexander comments, we do want to apply PAM limits to user jobs, something which is currently not implemented; but that's not what this bug report is about, so closing.

The fix around this issue is a poor solution. We were just hit hard with this very issue running MySQL server on Ubuntu 12.04 LTS. We increased the size of the server and the open files limit and Ubuntu would not honor it. It took many many man hours to stuble onto a post describing the problem and the solution.
Details can be found here: http://askubuntu.com/questions/288471/mysql-cant-open-files-after-updating-server-errno-24

Excerpts from Van Stokes, Jr.'s message of 2013-04-30 16:08:24 UTC:
> The fix around this issue is a poor solution. We were just hit hard with this very issue running MySQL server on Ubuntu 12.04 LTS. We increased the size of the server and the open files limit and Ubuntu would not honor it. It took many many man hours to stuble onto a post describing the problem and the solution.
> Details can be found here: http://askubuntu.com/questions/288471/mysql-cant-open-files-after-updating-server-errno-24
>

I can understand your frustration, this is a confusing area. However,
PAM is intended as a user oriented library, and daemons are by definition
not users.

In man limits.conf, it is clearly stated:

       Also, please note that all limit settings are set per login. They
       are not global, nor are they permanent; existing only for the
       duration of the session.

mysqld does not have a session, and should not have a session. If that
file worked for daemons in the past, it was by accident because they
were started directly from user sessions. This is precisely why upstart
has the ability to specify ulimits for system daemons like mysqld.

Note that sysctl could also be used to raise the system wide open
files limit.

Just wasted an hour on this issue :(