Ubuntu
linux package

kernel oops on ssh through ipsec vpn

Bug #710371 reported by Roland Dreier on 2011-01-30

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	linux (Ubuntu)	Fix Released	High	Andy Whitcroft

Bug Description

I got the oops in the attached dmesg immediately after trying to ssh to a machine that is reached through an ipsec VPN (configured with the shrew ipsec client, packaged in ike/ike-qtgui). This wasn't happening with 2.6.37 kernels in natty; it only happened since the update to 2.6.38. I have no problems with ssh connections not through the VPN, either before or after the kernel update.

I'm downloading the daily mainline build and will try and see if it happens there too...

ProblemType: Bug
DistroRelease: Ubuntu 11.04
Package: linux-image-2.6.38-1-generic 2.6.38-1.28
Regression: Yes
Reproducible: No
ProcVersionSignature: Ubuntu 2.6.38-1.28-generic 2.6.38-rc2
Uname: Linux 2.6.38-1-generic x86_64
AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.23.
Architecture: amd64
ArecordDevices:
**** List of CAPTURE Hardware Devices ****
card 0: Intel [HDA Intel], device 0: CONEXANT Analog [CONEXANT Analog]
   Subdevices: 1/1
   Subdevice #0: subdevice #0
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: roland 2260 F.... pulseaudio
CRDA: Error: [Errno 2] No such file or directory
Card0.Amixer.info:
Card hw:0 'Intel'/'HDA Intel at 0xf2520000 irq 44'
   Mixer name : 'Intel IbexPeak HDMI'
   Components : 'HDA:14f15069,17aa21a4,00100302 HDA:80862804,17aa21b5,00100000'
   Controls : 12
   Simple ctrls : 6
Card29.Amixer.info:
Card hw:29 'ThinkPadEC'/'ThinkPad Console Audio Control at EC reg 0x30, fw 6UHT31WW-1.12'
   Mixer name : 'ThinkPad EC 6UHT31WW-1.12'
   Components : ''
   Controls : 1
   Simple ctrls : 1
Card29.Amixer.values:
Simple mixer control 'Console',0
   Capabilities: pswitch pswitch-joined penum
   Playback channels: Mono
   Mono: Playback [on]
Date: Sun Jan 30 14:04:27 2011
EcryptfsInUse: Yes
Frequency: This has only happened once.
HibernationDevice: RESUME=UUID=078baba7-9199-4437-ab85-971bade7d75d
InstallationMedia: Ubuntu 11.04 "Natty Narwhal" - Alpha amd64 (20101206)
MachineType: LENOVO 2901CTO
ProcEnviron:
LANGUAGE=en_US:en
PATH=(custom, user)
LANG=en_US.UTF-8
LC_MESSAGES=en_US.utf8
SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-2.6.38-1-generic root=UUID=ab1e5491-e40c-4113-bf20-22ecb604999a ro quiet splash vt.handoff=7
RelatedPackageVersions:
linux-restricted-modules-2.6.38-1-generic N/A
linux-backports-modules-2.6.38-1-generic N/A
linux-firmware 1.46
SourcePackage: linux
dmi.bios.date: 10/27/2010
dmi.bios.vendor: LENOVO
dmi.bios.version: 6UET61WW (1.41 )
dmi.board.name: 2901CTO
dmi.board.vendor: LENOVO
dmi.board.version: Not Available
dmi.chassis.asset.tag: No Asset Information
dmi.chassis.type: 10
dmi.chassis.vendor: LENOVO
dmi.chassis.version: Not Available
dmi.modalias: dmi:bvnLENOVO:bvr6UET61WW(1.41):bd10/27/2010:svnLENOVO:pn2901CTO:pvrThinkPadT410s:rvnLENOVO:rn2901CTO:rvrNotAvailable:cvnLENOVO:ct10:cvrNotAvailable:
dmi.product.name: 2901CTO
dmi.product.version: ThinkPad T410s
dmi.sys.vendor: LENOVO

Tags:

Revision history for this message

Roland Dreier (roland.dreier) wrote on 2011-01-30:

AcpiTables.txt Edit (339.0 KiB, text/plain; charset="utf-8")
AlsaDevices.txt Edit (587 bytes, text/plain; charset="utf-8")
AplayDevices.txt Edit (261 bytes, text/plain; charset="utf-8")
BootDmesg.txt Edit (54.5 KiB, text/plain; charset="utf-8")
Card0.Amixer.values.txt Edit (1.1 KiB, text/plain; charset="utf-8")
Card0.Codecs.codec.0.txt Edit (8.1 KiB, text/plain; charset="utf-8")
Card0.Codecs.codec.3.txt Edit (2.7 KiB, text/plain; charset="utf-8")
CurrentDmesg.txt Edit (5.7 KiB, text/plain; charset="utf-8")
Dependencies.txt Edit (1.8 KiB, text/plain; charset="utf-8")
IwConfig.txt Edit (594 bytes, text/plain; charset="utf-8")
Lspci.txt Edit (12.5 KiB, text/plain; charset="utf-8")
Lsusb.txt Edit (497 bytes, text/plain; charset="utf-8")
PciMultimedia.txt Edit (598 bytes, text/plain; charset="utf-8")
ProcCpuinfo.txt Edit (3.3 KiB, text/plain; charset="utf-8")
ProcCpuinfo_.txt Edit (3.3 KiB, text/plain; charset="utf-8")
ProcInterrupts.txt Edit (1.9 KiB, text/plain; charset="utf-8")
ProcModules.txt Edit (5.1 KiB, text/plain; charset="utf-8")
RfKill.txt Edit (182 bytes, text/plain; charset="utf-8")
UdevDb.txt Edit (118.1 KiB, text/plain; charset="utf-8")
UdevLog.txt Edit (280.4 KiB, text/plain; charset="utf-8")
WifiSyslog.txt Edit (92.1 KiB, text/plain; charset="utf-8")

Revision history for this message

Roland Dreier (roland.dreier) wrote on 2011-01-30:

Download full text (4.3 KiB)

With the upstream kernel build linux-image-2.6.38-999-generic version 2.6.38-999.201101281114 I get what looks to be the same oops:

[ 307.861225] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 307.865272] IP: [< (null)>] (null)
[ 307.868654] PGD 0
[ 307.871146] Oops: 0010 [#1] SMP
[ 307.873642] last sysfs file: /sys/devices/LNXSYSTM:00/device:00/PNP0A08:00/device:0b/PNP0C09:00/PNP0C0A:00/power_supply/BAT0/voltage_now
[ 307.876302] CPU 0
[ 307.876329] Modules linked in: hidp hid ip6table_filter deflate zlib_deflate ip6_tables ctr twofish_generic twofish_x86_64 twofish_common camellia serpent blowfish cast5 des_generic aesni_intel cryptd aes_x86_64 aes_generic xcbc rmd160 xt_CHECKSUM iptable_mangle sha512_generic binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack sha256_generic ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge sha1_generic stp crypto_null af_key joydev parport_pc dm_crypt ppdev snd_hda_codec_hdmi rfcomm snd_hda_codec_conexant arc4 sco bnep thinkpad_acpi l2cap iwlagn snd_hda_intel snd_hda_codec snd_hwdep snd_pcm iwlcore uvcvideo snd_seq_midi snd_rawmidi mac80211 snd_seq_midi_event videodev v4l2_compat_ioctl32 btusb snd_seq bluetooth cfg80211 snd_timer snd_seq_device psmouse serio_raw snd snd_page_alloc intel_ips soundcore nvram tpm_tis tpm tpm_bios lp parport i915 drm_kms_helper e1000e drm ahci libahci i2c_algo_bit video
[ 307.898289]
[ 307.901550] Pid: 3140, comm: ssh Not tainted 2.6.38-999-generic #201101281114 2901CTO/2901CTO
[ 307.904894] RIP: 0010:[<0000000000000000>] [< (null)>] (null)
[ 307.908301] RSP: 0018:ffff880231b13ad0 EFLAGS: 00010246
[ 307.911679] RAX: ffffffff81a643c0 RBX: 0000000000000000 RCX: ffff88023105b180
[ 307.915069] RDX: ffff88023105b180 RSI: ffffffff814e5a50 RDI: ffff88023105b180
[ 307.918484] RBP: ffff880231b13b88 R08: 0000000000000008 R09: ffff880231b13be8
[ 307.921901] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000008
[ 307.925304] R13: 0000000000000000 R14: ffff880231b13e58 R15: ffff88022fe223c0
[ 307.928756] FS: 00007fd52fccc7e0(0000) GS:ffff8800bb000000(0000) knlGS:0000000000000000
[ 307.932232] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 307.935709] CR2: 0000000000000000 CR3: 0000000204452000 CR4: 00000000000006f0
[ 307.939253] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 307.942752] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 307.946109] Process ssh (pid: 3140, threadinfo ffff880231b12000, task ffff88022dda5b00)
[ 307.949520] Stack:
[ 307.952869] ffffffff814e6347 ffff880231b13af8 0000000000000246 ffff88022fe224c8
[ 307.956302] ffff880231b13be8 0000003d31b13b18 ffff880231b13e98 ffffffff814e5a50
[ 307.959728] ffff88023105be00 ffff880231b13b48 ffffffff814defc3 ffff880231b13c08
[ 307.963144] Call Trace:
[ 307.966528] [<ffffffff814e6347>] ? ip_append_data+0x787/0xab0
[ 307.969967] [<ffffffff814e5a50>] ? ip_generic_getfrag+0x0/0xa0
[ 307.973414] [<ffffffff814defc3>] ? ipv4_dst_blackhole+0x1c3/0x210
[ 307.976864] [<ffffffff815a115e>] ? _raw_spin_lock_bh+0x1e/0x...

With the upstream kernel build linux-image-2.6.38-999-generic version 2.6.38-999.201101281114 I get what looks to be the same oops:

[  307.861225] BUG: unable to handle kernel NULL pointer dereference at           (null)
[  307.865272] IP: [<          (null)>]           (null)
[  307.868654] PGD 0 
[  307.871146] Oops: 0010 [#1] SMP 
[  307.873642] last sysfs file: /sys/devices/LNXSYSTM:00/device:00/PNP0A08:00/device:0b/PNP0C09:00/PNP0C0A:00/power_supply/BAT0/voltage_now
[  307.876302] CPU 0 
[  307.876329] Modules linked in: hidp hid ip6table_filter deflate zlib_deflate ip6_tables ctr twofish_generic twofish_x86_64 twofish_common camellia serpent blowfish cast5 des_generic aesni_intel cryptd aes_x86_64 aes_generic xcbc rmd160 xt_CHECKSUM iptable_mangle sha512_generic binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack sha256_generic ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge sha1_generic stp crypto_null af_key joydev parport_pc dm_crypt ppdev snd_hda_codec_hdmi rfcomm snd_hda_codec_conexant arc4 sco bnep thinkpad_acpi l2cap iwlagn snd_hda_intel snd_hda_codec snd_hwdep snd_pcm iwlcore uvcvideo snd_seq_midi snd_rawmidi mac80211 snd_seq_midi_event videodev v4l2_compat_ioctl32 btusb snd_seq bluetooth cfg80211 snd_timer snd_seq_device psmouse serio_raw snd snd_page_alloc intel_ips soundcore nvram tpm_tis tpm tpm_bios lp parport i915 drm_kms_helper e1000e drm ahci libahci i2c_algo_bit video
[  307.898289] 
[  307.901550] Pid: 3140, comm: ssh Not tainted 2.6.38-999-generic #201101281114 2901CTO/2901CTO
[  307.904894] RIP: 0010:[<0000000000000000>]  [<          (null)>]           (null)
[  307.908301] RSP: 0018:ffff880231b13ad0  EFLAGS: 00010246
[  307.911679] RAX: ffffffff81a643c0 RBX: 0000000000000000 RCX: ffff88023105b180
[  307.915069] RDX: ffff88023105b180 RSI: ffffffff814e5a50 RDI: ffff88023105b180
[  307.918484] RBP: ffff880231b13b88 R08: 0000000000000008 R09: ffff880231b13be8
[  307.921901] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000008
[  307.925304] R13: 0000000000000000 R14: ffff880231b13e58 R15: ffff88022fe223c0
[  307.928756] FS:  00007fd52fccc7e0(0000) GS:ffff8800bb000000(0000) knlGS:0000000000000000
[  307.932232] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  307.935709] CR2: 0000000000000000 CR3: 0000000204452000 CR4: 00000000000006f0
[  307.939253] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  307.942752] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  307.946109] Process ssh (pid: 3140, threadinfo ffff880231b12000, task ffff88022dda5b00)
[  307.949520] Stack:
[  307.952869]  ffffffff814e6347 ffff880231b13af8 0000000000000246 ffff88022fe224c8
[  307.956302]  ffff880231b13be8 0000003d31b13b18 ffff880231b13e98 ffffffff814e5a50
[  307.959728]  ffff88023105be00 ffff880231b13b48 ffffffff814defc3 ffff880231b13c08
[  307.963144] Call Trace:
[  307.966528]  [<ffffffff814e6347>] ? ip_append_data+0x787/0xab0
[  307.969967]  [<ffffffff814e5a50>] ? ip_generic_getfrag+0x0/0xa0
[  307.973414]  [<ffffffff814defc3>] ? ipv4_dst_blackhole+0x1c3/0x210
[  307.976864]  [<ffffffff815a115e>] ? _raw_spin_lock_bh+0x1e/0x30
[  307.980287]  [<ffffffff8150bd09>] udp_sendmsg+0x389/0x790
[  307.983732]  [<ffffffff815143d9>] inet_sendmsg+0x69/0xc0
[  307.987138]  [<ffffffff8149cb3a>] sock_sendmsg+0x10a/0x120
[  307.990552]  [<ffffffff8107e663>] ? queue_delayed_work_on+0xb3/0x110
[  307.994033]  [<ffffffff815a1099>] ? _raw_spin_unlock_bh+0x19/0x20
[  307.997455]  [<ffffffff814b8260>] ? __dst_free+0x50/0xa0
[  308.000868]  [<ffffffff815a1099>] ? _raw_spin_unlock_bh+0x19/0x20
[  308.004274]  [<ffffffff8150adcb>] ? udp_lib_rehash+0x11b/0x140
[  308.007675]  [<ffffffff8150b2ef>] ? udp_v4_rehash+0x2f/0x40
[  308.011108]  [<ffffffff8149adaa>] ? sockfd_lookup_light+0x4a/0x80
[  308.014548]  [<ffffffff8149d75a>] sys_sendto+0x13a/0x190
[  308.017982]  [<ffffffff815a0e0e>] ? _raw_spin_lock+0xe/0x20
[  308.021395]  [<ffffffff81156c30>] ? fd_install+0x60/0x70
[  308.024770]  [<ffffffff8100c002>] system_call_fastpath+0x16/0x1b
[  308.028141] Code:  Bad RIP value.
[  308.031547] RIP  [<          (null)>]           (null)
[  308.034952]  RSP <ffff880231b13ad0>
[  308.038330] CR2: 0000000000000000
[  308.041823] ---[ end trace 12c863de1da69f98 ]---

so this looks to be both easily reproducible and present upstream.

Jeremy Foshee (jeremyfoshee) on 2011-01-31

tags:	removed: needs-upstream-testing
Changed in linux (Ubuntu):
status:	New → Triaged

Brian Murray (brian-murray) on 2011-01-31

Changed in linux (Ubuntu):
importance:	Undecided → High

Revision history for this message

Roland Dreier (roland.dreier) wrote on 2011-01-31:

Add missing default_mtu methods to blackhole dst_ops Edit (2.2 KiB, text/plain)

I'm attaching a patch that I also just sent to netdev that (I think) is the right fix for this crash.

Revision history for this message

Roland Dreier (roland.dreier) wrote on 2011-02-01:

Fix is now in Dave Miller's tree pending merge to Linus:
http://git.kernel.org/?p=linux/kernel/git/davem/net-2.6.git;a=commitdiff;h=ec831ea72ee5d7d473899e27a86bd659482c4d0d

Andy Whitcroft (apw) on 2011-02-01

Changed in linux (Ubuntu):
assignee:	nobody → Andy Whitcroft (apw)

Revision history for this message

Roland Dreier (roland.dreier) wrote on 2011-02-04:

Fix is now upstream in Linus's tree as ec831ea72ee5 ("net: Add default_mtu() methods to blackhole dst_ops").

Revision history for this message

Andy Whitcroft (apw) wrote on 2011-02-16:

This commit was in v2.6.38-rc4, which was released in 2.6.38-3.30 in Natty. Closing Fix Released.

Changed in linux (Ubuntu):
status:	Triaged → Fix Released

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Patches

Add missing default_mtu methods to blackhole dst_ops Edit

Add patch

Bug attachments

Add attachment

Remote bug watches

Bug watches keep track of this bug in other bug trackers.

Ubuntulinux package

kernel oops on ssh through ipsec vpn

Bug Description

Other bug subscribers

Patches

Bug attachments

Remote bug watches

Ubuntu
linux package