Please raise file descriptor hard limit to 4096 (but keep soft limit at 1024)

Bug #663090 reported by Dan Kegel
62
This bug affects 10 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Fix Released
Medium
Tim Gardner
Lucid
Fix Released
Undecided
Scott Ritchie
Natty
Fix Released
Medium
Tim Gardner
linux-ti-omap4 (Ubuntu)
Fix Released
Undecided
Unassigned
Lucid
Invalid
Undecided
Tim Gardner
Natty
Fix Released
Undecided
Unassigned

Bug Description

select() has the well-known flaw that it has undefined behavior if
you try to use it with a file decriptor above 1023 (see
http://sourceware.org/bugzilla/show_bug.cgi?id=10352 ).
This prevents Ubuntu from raising the default ulimit -n;
it just wouldn't do for random apps to start breaking.

Application developers have known for years that select() has
this flaw, and I'm starting to see desktop apps that really do
need more file descriptors. It may be time for us to start purging
dangerous calls to select() from the ubuntu codebase, to make the
world safe for higher values of ulimit -n.

(That said, if one calls select() with no file descriptors at all, just
for its timeout behavior, that's totally safe; those calls can be safely left in place.)

I guess the next step is to audit Ubuntu's codebase to count the number
of possibly dangerous calls to select().

Revision history for this message
Fabio Marconi (fabiomarconi) wrote :

Thank you for taking the time to make Ubuntu better. Since what you submitted is not really a bug, or a problem, but rather an idea to improve Ubuntu, you are invited to post your idea in Ubuntu Brainstorm at http://brainstorm.ubuntu.com/ where it can be discussed, voted by the community and reviewed by developers. Thanks for taking the time to share your opinion!

Changed in ubuntu:
status: New → Invalid
Revision history for this message
Dan Kegel (dank) wrote :

It's a bug in that Ubuntu's wine can't run a number of Windows apps due to this. A few example apps:

Catia
http://appdb.winehq.org/objectManager.php?sClass=version&iId=16156&iTestingId=50899
Quicken 2010
http://appdb.winehq.org/objectManager.php?sClass=version&iId=18171
Stronghold Legends
http://appdb.winehq.org/objectManager.php?sClass=version&iId=11924&iTestingId=56413
Visual C++
http://forum.winehq.org/viewtopic.php?t=7500&sid=d10c93707f9764618e4bbcc3c66ecca0

It's not a bug in wine, and not a bug in the app, that they need lots of file descriptors.
Making the user raise the limit himself just for the one app is inconvenient for them.

Now, one could magically endow wine with the ability to raise its own ulimit -n with some sort of wrapper,
and maybe that's what will happen, but in the end select() must die, or we'll keep bumping into
these problems.

But hey, if you say this isn't a good match for the bug tracker, I'll go file it in brainstorm as well.

Revision history for this message
Dan Kegel (dank) wrote :
Revision history for this message
Scott Ritchie (scottritchie) wrote :

Fabian, this is a bug, it's just not against a specific app (yet). It's one of those launchpad bugs that could target quite a few apps (much like when we scanned through the archive for old uses of libindicator)

Changed in ubuntu:
status: Invalid → Confirmed
Revision history for this message
Dan Kegel (dank) wrote :

Alexandre Julliard points out that we can raise the default hard limit to 4096 or so
but leave the default soft limit at 1024. That would keep unsafe select() from
bothering most apps.

Revision history for this message
Scott Ritchie (scottritchie) wrote :

How would that be done? Is the setting in /etc a hard or default soft limit?

Revision history for this message
Dan Kegel (dank) wrote :

Both. Add the lines

* hard nofile 4096
* soft nofile 1024

to /etc/security/limits.conf. Then apply the patch
http://www.winehq.org/pipermail/wine-patches/2010-November/095464.html
to wine. Et voila, wine is happy without exposing the average user
to damage from unsafe use of select().

Revision history for this message
Dan Kegel (dank) wrote :

wine's smarter than I thought. It already does the setrlimit in its loader, so no patch is needed, we just need to raise the hard rlimit.

summary: - Why can't Johnny handle 1025 file descriptors?
+ Please raise file descriptor hard limit to 4096 (but keep soft limit at
+ 1024)
Revision history for this message
Scott Ritchie (scottritchie) wrote :

Moving to Linux package where it belongs, according to the mailing list: https://lists.ubuntu.com/archives/ubuntu-devel/2010-November/031946.html

affects: ubuntu → linux (Ubuntu)
Revision history for this message
Gursimran singh (simar) wrote :

Even if the due to the issue, many application does not run under wine(as stated in the bug comment #2). The setting for the file descriptor seems to be quite intentional. So i'm marking it a wishlist, and leave the decision with the person concerned..

Changed in linux (Ubuntu):
importance: Undecided → Wishlist
Revision history for this message
Dan Kegel (dank) wrote :

Please reconsider. An increasing number of apps are going to be hurt by this low hard limit.

Revision history for this message
Scott Ritchie (scottritchie) wrote :

I should note the Ubuntu-devel mailing list agrees with me:

http://<email address hidden>/msg00146.html

*namedrops Kees Cook and Steve Langasek* ;)

Revision history for this message
Charlie Schluting ☃ (cschluti) wrote :

+1
this low limit bites server users all the time..

Changed in linux (Ubuntu):
importance: Wishlist → Medium
Revision history for this message
Dan Kegel (dank) wrote :

Is this done for Natty yet? I'm installing League of Legends right now, and
watching the number of wineserver's open fd's creep towards 1024, at
which point it will probably explode.

Revision history for this message
Kelytha (kelytharun) wrote :

+1

Valve's games based on the Source engine also suffer from this bug.

Revision history for this message
Dan Kegel (dank) wrote :

$ dpkg-query -S /etc/security/limits.conf
libpam-modules: /etc/security/limits.conf

This bug should probably be against libpam-modules, since the fix is to add one or two lines to /etc/security/limits.conf.

affects: linux (Ubuntu) → pam (Ubuntu)
Revision history for this message
Steve Langasek (vorlon) wrote :

No. It is the policy of the pam package to not deviate from the kernel defaults for ulimits. pam is not the place to make this change; if the default ulimits are wrong in the kernel, they should be fixed in the kernel.

affects: pam (Ubuntu) → linux (Ubuntu)
Tim Gardner (timg-tpi)
Changed in linux (Ubuntu Natty):
assignee: nobody → Tim Gardner (timg-tpi)
status: Confirmed → Fix Committed
Revision history for this message
Dan Kegel (dank) wrote :

Can't wait to try it!

Revision history for this message
Susan Cragin (susancragin) wrote : Re: [Bug 663090] Re: Please raise file descriptor hard limit to 4096 (but keep soft limit at 1024)

Yippee!!!

Dan, question...
Today's git compile, is there a reportable error in the jack module?

make[1]: Entering directory `/home/susan/wine/dlls/winejack.drv'
gcc -m32 -c -I. -I. -I../../include -I../../include -D__WINESRC__ -D_REENTRANT -fPIC -Wall -pipe -fno-strict-aliasing -Wdeclaration-after-statement -Wstrict-prototypes -Wtype-limits -Wwrite-strings -Wpointer-arith -Wlogical-op -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -o audio.o audio.c
audio.c: In function ‘JACK_DriverProc’:
audio.c:2419:50: error: ‘RTLD_NOW’ undeclared (first use in this function)
audio.c:2419:50: note: each undeclared identifier is reported only once for each function it appears in
audio.c:2443:5: warning: implicit declaration of function ‘MessageBoxA’
audio.c:2443:78: error: ‘MB_OK’ undeclared (first use in this function)
make[1]: *** [audio.o] Error 1
make[1]: Leaving directory `/home/susan/wine/dlls/winejack.drv'
make: *** [dlls/winejack.drv] Error 2

-----Original Message-----
>From: Dan Kegel <email address hidden>
>Sent: Mar 29, 2011 7:51 PM
>To: <email address hidden>
>Subject: [Bug 663090] Re: Please raise file descriptor hard limit to 4096 (but keep soft limit at 1024)
>
>Can't wait to try it!
>
>--
>You received this bug notification because you are a direct subscriber
>of the bug.
>https://bugs.launchpad.net/bugs/663090
>
>Title:
> Please raise file descriptor hard limit to 4096 (but keep soft limit
> at 1024)
>
>To unsubscribe from this bug, go to:
>https://bugs.launchpad.net/ubuntu/+source/linux/+bug/663090/+subscribe

Revision history for this message
Susan Cragin (susancragin) wrote :

The above comment was posted in error. I thought it was a personal message and didn't check the recipient. Sorry.

Revision history for this message
Tim Gardner (timg-tpi) wrote :

Dan - you can test it out by pulling from https://launchpad.net/~kernel-ppa/+archive/pre-proposed?field.series_filter=natty (as soon as its done building)

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package linux - 2.6.38-8.40

---------------
linux (2.6.38-8.40) natty; urgency=low

  [ Brad Figg ]

  * [Config] Set CONFIG_NR_CPUS=256 for amd64 generic
    - LP: #737124

  [ Henrik Rydberg ]

  * SAUCE: HID: hid-ntrig: add support for 1b96:0006 model
  * SAUCE: HID: ntrig: fix suspend/resume on recent models

  [ Kees Cook ]

  * [Config] packaging: adjust perms on vmlinuz as well
  * SAUCE: nx-emu: further clarify dmesg reporting
    - LP: #745181

  [ Leann Ogasawara ]

  * rebase to v2.6.38.1
  * [Config] update configs after v2.6.38.1 rebase
  * rebase to v2.6.38.2

  [ Manoj Iyer ]

  * SAUCE: thinkpad-acpi: module autoloading for newer Lenovo ThinkPads.
    - LP: #745217

  [ Tim Gardner ]

  * SAUCE: INR_OPEN=4096
    - LP: #663090

  [ Upstream Kernel Changes ]

  * (drop after v2.6.38) HID: ntrig don't dereference unclaimed hidinput
  * (drop after v2.6.38) HID: ntrig: apply NO_INIT_REPORTS quirk
  * (drop after v2.6.38) HID: hid-ntrig: init settle and mode check
  * eeepc-wmi: add hotplug code for Eeepc 1000H
  * eeepc-wmi: serialize access to wmi method
  * eeepc-wmi: return proper error code in eeepc_rfkill_set()
  * eeepc-wmi: add an helper using simple return codes
  * eeepc-wmi: add hibernate/resume callbacks
  * eeepc-wmi: switch to platform_create_bundle()
  * eeepc-wmi: reorder defines
  * eeepc-wmi: use the presence bit correctly
  * eeepc-wmi: add camera and card reader support
  * eeepc-wmi: add wimax support
  * eeepc-wmi: set the right key code for 0xe9
  * eeepc-wmi: support backlight power (bl_power) attribute
  * eeepc-wmi: respect wireless_hotplug setting
  * eeepc-wmi: real touchpad led device id is 0x001000012
  * eeepc-wmi: comments keymap to clarify the meaning of some keys
  * eeepc-wmi: add touchpad sysfs file
  * eeepc-wmi: reorder device ids

  [ Major Kernel Changes ]

  * rebase from v2.6.38 to v2.6.38.1
    - LP: #735640, #735450
  * rebase from v2.6.38.1 to v2.6.38.2
    - LP: #733780
 -- Leann Ogasawara <email address hidden> Mon, 28 Mar 2011 06:20:13 -0700

Changed in linux (Ubuntu Natty):
status: Fix Committed → Fix Released
Revision history for this message
Steve Langasek (vorlon) wrote :

discussed on IRC, this patch raised both the hard and soft limits where we only want to raise the hard limit. Reopening.

Changed in linux (Ubuntu Natty):
status: Fix Released → In Progress
Revision history for this message
Tim Gardner (timg-tpi) wrote :

New patch pending that leaves the default soft limit at 1024 and raises the default hard limit to 4096

Changed in linux (Ubuntu Natty):
status: In Progress → Fix Committed
Revision history for this message
Tim Gardner (timg-tpi) wrote :
Revision history for this message
Dan Kegel (dank) wrote :
Revision history for this message
Dan Kegel (dank) wrote :

Upsteam bug report comment:

--- Comment #1 from Andrew Morton <email address hidden> 2011-03-31 20:24:33 ---
Send it to lkml, please. CC me and anyone else who might be interested.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package linux - 2.6.38-8.41

---------------
linux (2.6.38-8.41) natty; urgency=low

  [ Luke Yelavich ]

  * [Config] Disable CONFIG_CRASH_DUMP on 32-bit powerpc kernels
    - LP: #745358
  * [Config] Disable CONFIG_DRM_RADEON_KMS on powerpc kernels
  * [Config] Build some framebuffer drivers as modules for powerpc kernels.

  [ Seth Forshee ]

  * SAUCE: (drop after 2.6.38) eeepc-wmi: Add support for T101MT
    Home/Express Gate key

  [ Tim Gardner ]

  * SAUCE: Increase the default hard limit for open FDs to 4096
    - LP: #663090

  [ Upstream Kernel Changes ]

  * ALSA: pcm: fix infinite loop in snd_pcm_update_hw_ptr0()
  * Relax si_code check in rt_sigqueueinfo and rt_tgsigqueueinfo
 -- Leann Ogasawara <email address hidden> Tue, 05 Apr 2011 09:25:22 -0700

Changed in linux (Ubuntu Natty):
status: Fix Committed → Fix Released
Tim Gardner (timg-tpi)
Changed in linux-ti-omap4 (Ubuntu Natty):
status: New → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (38.0 KiB)

This bug was fixed in the package linux-ti-omap4 - 2.6.38-1209.15

---------------
linux-ti-omap4 (2.6.38-1209.15) natty-proposed; urgency=low

  * Release tracking bug
    - LP: #837761

  [ Paolo Pisati ]

  * [Config] Turn on CONFIG_USER_NS and DEVPTS_MULTIPLE_INSTANCES.
    - LP: #787749

  [ Tim Gardner ]

  * [Config] Add enic/fnic to nic-modules udeb, CVE-2011-1020
    - LP: #801610

  [ Upstream Kernel Changes ]

  * mpt2sas: prevent heap overflows and unchecked reads
    - LP: #780546
  * agp: fix arbitrary kernel memory writes
    - LP: #775809
  * can: add missing socket check in can/raw release
    - LP: #780546
  * agp: fix OOM and buffer overflow
    - LP: #775809
  * bonding: Incorrect TX queue offset, CVE-2011-1581
    - LP: #792312
    - CVE-2011-1581
  * fs/partitions/efi.c: corrupted GUID partition tables can cause kernel
    oops
    - LP: #795418
    - CVE-2011-1577
  * can: Add missing socket check in can/bcm release.
    - LP: #796502
    - CVE-2011-1598
  * USB: ehci: remove structure packing from ehci_def
    - LP: #791552
  * taskstats: don't allow duplicate entries in listener mode,
    CVE-2011-2484
    - LP: #806390
    - CVE-2011-2484
  * ext4: init timer earlier to avoid a kernel panic in __save_error_info,
    CVE-2011-2493
    - LP: #806929
    - CVE-2011-2493
  * dccp: handle invalid feature options length, CVE-2011-1770
    - LP: #806375
    - CVE-2011-1770
  * pagemap: close races with suid execve, CVE-2011-1020
    - LP: #813026
    - CVE-2011-1020
  * report errors in /proc/*/*map* sanely, CVE-2011-1020
    - LP: #813026
    - CVE-2011-1020
  * close race in /proc/*/environ, CVE-2011-1020
    - LP: #813026
    - CVE-2011-1020
  * auxv: require the target to be tracable (or yourself), CVE-2011-1020
    - LP: #813026
    - CVE-2011-1020
  * deal with races in /proc/*/{syscall, stack, personality}, CVE-2011-1020
    - LP: #813026
    - CVE-2011-1020
  * rose: Add length checks to CALL_REQUEST parsing, CVE-2011-1493
    - LP: #816550
    - CVE-2011-1493
  * GFS2: make sure fallocate bytes is a multiple of blksize, CVE-2011-2689
    - LP: #819572
    - CVE-2011-2689
  * Bluetooth: l2cap and rfcomm: fix 1 byte infoleak to userspace.
    - LP: #819569
    - CVE-2011-2492
  * Add mount option to check uid of device being mounted = expect uid,
    CVE-2011-1833
    - LP: #732628
    - CVE-2011-1833
  * ipv6: make fragment identifications less predictable, CVE-2011-2699
    - LP: #827685
    - CVE-2011-2699
  * perf: Fix software event overflow, CVE-2011-2918
    - LP: #834121
    - CVE-2011-2918
  * proc: fix oops on invalid /proc/<pid>/maps access, CVE-2011-1020
    - LP: #813026
    - CVE-2011-1020

linux-ti-omap4 (2.6.38-1209.13) natty; urgency=low

  [ Tim Gardner ]

  * Release Tracking Bug
    - LP: #772381

  [ Brad Figg ]

  * Ubuntu-2.6.38-9.43

  [ Bryan Wu ]

  * merge Ubuntu-2.6.38-9.43
  * cherry-pick 6 patches from u2 of 'for-ubuntu' branch
  * [Config] Sync up configs for 2.6.38.4

  [ Herton Ronaldo Krzesinski ]

  * SAUCE: Revert "x86, hibernate: Initialize mmu_cr4_features during boot"
    - LP: #764758

  [ Leann Ogasawara ]

  * [Config] updateconfigs for 2.6.38.4

  [ Paolo Pisati ]

  * [Conf...

Changed in linux-ti-omap4 (Ubuntu Natty):
status: Fix Committed → Fix Released
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "Increase the default FD hard limit to 4096" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-sponsors please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

tags: added: patch
Tim Gardner (timg-tpi)
Changed in linux-ti-omap4 (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Scott Ritchie (scottritchie) wrote :

I'm going to nominate this for Lucid, as I ran into this issue a few times in live environments. A quick web search for ubuntu nginx 1024 file will reveal a lot of similar people having the issue.

Tim Gardner (timg-tpi)
Changed in linux (Ubuntu Lucid):
assignee: nobody → Tim Gardner (timg-tpi)
status: New → In Progress
Changed in linux-ti-omap4 (Ubuntu Lucid):
assignee: nobody → Tim Gardner (timg-tpi)
status: New → In Progress
Tim Gardner (timg-tpi)
Changed in linux (Ubuntu Lucid):
status: In Progress → Fix Committed
Changed in linux-ti-omap4 (Ubuntu Lucid):
status: In Progress → Invalid
Changed in linux (Ubuntu Lucid):
assignee: Tim Gardner (timg-tpi) → Scott Ritchie (scottritchie)
Revision history for this message
Luis Henriques (henrix) wrote :

This bug is awaiting verification that the kernel for Lucid in -proposed solves the problem (2.6.32-43.97). Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-lucid' to 'verification-done-lucid'.

If verification is not done by one week from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-lucid
Revision history for this message
Tim Gardner (timg-tpi) wrote :

Marking verification-done-lucid since 'ulimit: raise default hard ulimit on number of files to 4096' is an upstream cherry-pick that has been successfully applied to subsequent kernels with no regression.

tags: added: verification-done-lucid
removed: verification-needed-lucid
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package linux - 2.6.32-43.97

---------------
linux (2.6.32-43.97) lucid-proposed; urgency=low

  [Luis Henriques]

  * Release Tracking Bug
    - LP: #1045405

  [ Upstream Kernel Changes ]

  * rds: set correct msg_namelen
    - LP: #1031112
    - CVE-2012-3430
  * eCryptfs: Initialize empty lower files when opening them
    - LP: #911507
  * net: Allow driver to limit number of GSO segments per skb
    - LP: #1037456
    - CVE-2012-3412
  * tcp: do not scale TSO segment size with reordering degree
    - LP: #1037456
    - CVE-2012-3412
  * tcp: Apply device TSO segment limit earlier
    - LP: #1037456
    - CVE-2012-3412
  * sfc: Replace some literal constants with EFX_PAGE_SIZE/EFX_BUF_SIZE
    - LP: #1037456
    - CVE-2012-3412
  * sfc: Fix maximum number of TSO segments and minimum TX queue size
    - LP: #1037456
    - CVE-2012-3412
  * mm: Hold a file reference in madvise_remove
    - LP: #1042447
    - CVE-2012-3511
  * ulimit: raise default hard ulimit on number of files to 4096
    - LP: #663090
 -- Luis Henriques <email address hidden> Wed, 05 Sep 2012 09:39:41 +0100

Changed in linux (Ubuntu Lucid):
status: Fix Committed → Fix Released
Revision history for this message
Adam Conrad (adconrad) wrote : Update Released

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.