xargs complains command line is too long, but kernel allows command lines of arbitrary length

Thanks for your report.

Please, can you post a testcase and tell us in details how to reproduce this issue.
This will greatly help us to track down your problem. Thanks.

xargs -0 touch < foo

Where foo is a file containing no NUL bytes that is longer than 128Kb. If the file is shorter, I get an error from touch instead, which seems reasonable (file name too long). But why does xargs complain?

In fact, I got the error message slightly wrong, it is:

"argument line too long"

confirming.

== testcase ==
for f in `seq 1 20000`;echo -n "xxxxxxxx "; done | xargs -0

Only happens with -0 which means the input line is processed as one long line.

Are you using a stock kernel or a customized one.

Please can you provide the output of uname -a

Thanks.

Stock kernel:

Linux canta 2.6.24-19-generic #1 SMP Wed Jun 18 14:43:41 UTC 2008 i686 GNU/Linux

In Ubuntu the maximum page allocation for env+arg is hard coded and limited to 32 pages of 4KB.

If you look at the include/linux/binfmts.h file, you will find the following near the top:

/*
 * MAX_ARG_PAGES defines the number of pages allocated for arguments
 * and envelope for the new program. 32 should suffice, this gives
 * a maximum env+arg of 128kB w/4KB pages!
 */
#define MAX_ARG_PAGES 32

This is not what I find. Instead, I have:

/*
 * These are the maximum length and maximum number of strings passed to the
 * execve() system call. MAX_ARG_STRLEN is essentially random but serves to
 * prevent the kernel from being unduly impacted by misaddressed pointers.
 * MAX_ARG_STRINGS is chosen to fit in a signed 32-bit integer.
 */
#define MAX_ARG_STRLEN (PAGE_SIZE * 32)
#define MAX_ARG_STRINGS 0x7FFFFFFF

so, it seems that there is a maximum string length, but not a maximum overall argument length. Still, you can close this bug.

It's defined a few lines below at line 32 on my version.

black:$ grep -n MAX_ARG_PAGES /usr/src/linux-headers-2.6.26-4/include/linux/binfmts.h
32:# define MAX_ARG_PAGES 32

You can safely increase this value and recompile the kernel to give more space to arguments.

Closing this report as requested.

Thanks for your time.

You're quoting from a non-stock (or at least, non-hardy) kernel. Is this limit re-established in later (Ubuntu?) kernels? If so, I'm deeply disappointed, as this was a great innovation in Linux. If it's Ubuntu-specific, I'd like to know, so I can file a bug against the relevant kernel package.

Well. I've been wrong on this one. I'm afraid that the kernel has nothing to do with this.

The limit for find and xargs is hardcoded in lib/buildcmd.c in bc_use_sensible_arg_max and can be at build time.

Here is the relevant part of the changelog and more details can be found in the README.

2007-07-31 Eric Blake <email address hidden>

 Allow choice of default arg size, Savannah bug #20594.
 * configure.ac (DEFAULT_ARG_SIZE): Check environment for a default
 size override.
 * lib/buildcmd.c (bc_use_sensible_arg_max): Use default size from
 configure, if requested.
 * README (DEFAULT_ARG_SIZE): Mention the ability to tune this at
 configure time.
 * NEWS: Document the change.

I'm marking this report "confirmed" .