xargs on qemu has bad ARG_MAX

confirmed that reconfiguring xargs with
  DEFAULT_ARG_SIZE="(127u * 1024u)"
instead of the default: (128u * 1024u)
fixes the problem.

patch to debian/rules :

--- rules.orig 2009-05-05 15:44:19.000000000 +0000
+++ rules 2009-05-05 10:54:23.000000000 +0000
@@ -36,7 +36,7 @@

        dh_testdir
        # Add here commands to configure the package.
- CFLAGS="$(CFLAGS)" ./configure $(confflags) \
+ DEFAULT_ARG_SIZE="(127u * 1024u)" CFLAGS="$(CFLAGS)" ./configure $(confflags) \
                --prefix=/usr --localstatedir=/var/cache/locate \
                --enable-d_type-optimisation \
                --libexecdir='$${prefix}/lib/locate' \

actually this still broke... but less often....

so I hit it with a bigger stick:
  DEFAULT_ARG_SIZE="(64u * 1024u)"

I can't seem to confirm this on my arm system (karmic) nor on my jaunty x86_64 system. Is there any reason why you believe this may be specific to arm?

I reported this on #ubuntu-arm (or similar) whilst compiling arm code for ubuntu/maemo/mer on the open build service; and someone specifically requested that I tag it 'arm' when I reported it.

The problem only manifests itself using qemu arm emulation ... OTOH I guess this is fairly important to arm as qemu/findutils is pretty crucial for arm build systems :)

As I said, xargs is hitting an internal hardcoded value and failing and I think that is a bug.
I also think there is a bug in qemu and that's being pursued too.

I suspect this is not specific to arm, but I'm going to leave it tagged as armel for the time being since the only machine where you seem to be able to reproduce it at the moment is qemu-arm. Thanks for your work on the analysis of the problem.

The default arg size used in xargs is further checked against the posix api reported limits. It is possible the upper limit is 128K on qemu/versatile/arm, but note that whatever value is selected is further validated against sysconf(_SC_ARG_MAX). However, as noted in sysconf itself, this does not account for space needed to pass the env variables for exec(), and this may be why setting a value of 127k may restrict the size enough not to immediately fail, but still fails sometimes, depending on what is in the env or what space is also needed for the file path for exec. However, before changing xargs, we should validate what sysconf(_SC_ARG_MAX) reports as the kernel may have wrong or different values than that actually supported by the QEMU emulation of versatile, which may be different from real hardware. Hence, lets make sure we are not fixing the wrong problem in the wrong place...

@David Sugar: So we should be computing xargs' max at runtime with sysconf instead of at build time?

Loic, we currently do both; we have a default maximum value hard coded, but we also check at runtime to make sure it is not larger than what sysconf would allow.

The example appears to work for me on my setup; that's an Ubuntu Maverick ARM chroot with qemu 0.12.3 running on an Ubuntu Lucid i386 host (the qemu is newer than Lucid).

sudo chroot /armroot /usr/bin/qemu-arm-static /bin/bash

find /usr -print0 | xargs -0r pwd
/bin/pwd: ignoring non-option arguments
/
/bin/pwd: ignoring non-option arguments
/
/bin/pwd: ignoring non-option arguments
/
/bin/pwd: ignoring non-option arguments
/
/bin/pwd: ignoring non-option arguments
/
/bin/pwd: ignoring non-option arguments
/

but the reporter does say it's very sensitive.

Dave

This was fixed in qemu in commit:

fd4d81dd04b4e606ce40a41d66368ba77c77c753

long since included in qemu.

Moving the now-"fix released" bug to the qemu-linaro Ubuntu package