systemd-resolved crashed with SIGSEGV in dns_packet_is_reply_for()

Bug #1621396 reported by Daniele Dellafiore
394
This bug affects 9 people
Affects Status Importance Assigned to Milestone
systemd (Ubuntu)
Fix Released
Medium
Dimitri John Ledkov
Xenial
Fix Released
Low
Unassigned
Zesty
Fix Released
Low
Unassigned

Bug Description

[Impact]

 * Null-pointer dereference in resolved, results in resolved crash and reports on launchpad and errors.

[Test Case]

 * Unknown steps to reproduce
 * Monitor the drop off in crashes on errors.ubuntu.com:
https://errors.ubuntu.com/problem/ea90aefe098653f44b46e56d72e2cc05ff980465

[Regression Potential]

 * The behavior is similar, instead of crashing resolved it returns an error in the relevant function. Whilst this may not result in correct dns resolution for the affected dns packets, it should not result in resolved crashes.

[Original Bug Report]

This is one of background errors that happens without any active app being involved.
For the records, I had open: Firefox, Slack, Franz and the Terminal

The Ubuntu Error Tracker has been receiving reports about a problem regarding systemd. This problem was most recently seen with package version 233-6ubuntu2, the problem page at https://errors.ubuntu.com/problem/ea90aefe098653f44b46e56d72e2cc05ff980465 contains more details, including versions of packages affected, stacktrace or traceback, and individual crash reports.
If you do not have access to the Ubuntu Error Tracker you can request it at http://forms.canonical.com/reports/.

ProblemType: Crash
DistroRelease: Ubuntu 16.10
Package: systemd 231-5
ProcVersionSignature: Ubuntu 4.4.0-9136.55-generic 4.4.16
Uname: Linux 4.4.0-9136-generic x86_64
ApportVersion: 2.20.3-0ubuntu7
Architecture: amd64
CrashCounter: 1
Date: Thu Sep 8 09:33:55 2016
ExecutablePath: /lib/systemd/systemd-resolved
InstallationDate: Installed on 2013-06-06 (1189 days ago)
InstallationMedia: Ubuntu 13.04 "Raring Ringtail" - Release amd64 (20130424)
MachineType: Dell Inc. Dell System XPS L322X
ProcCmdline: /lib/systemd/systemd-resolved
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-9136-generic root=UUID=2850be62-a05e-4ab9-af2b-5f1fd159ce5d ro quiet splash vt.handoff=7
Signal: 11
SourcePackage: systemd
StacktraceTop:
 ?? ()
 ?? ()
 ?? () from /lib/systemd/libsystemd-shared-231.so
 sd_event_dispatch () from /lib/systemd/libsystemd-shared-231.so
 sd_event_run () from /lib/systemd/libsystemd-shared-231.so
Title: systemd-resolved crashed with SIGSEGV in sd_event_dispatch()
UpgradeStatus: Upgraded to yakkety on 2016-09-03 (4 days ago)
UserGroups:

dmi.bios.date: 04/18/2013
dmi.bios.vendor: Dell Inc.
dmi.bios.version: A08
dmi.board.name: 0PJHXN
dmi.board.vendor: Dell Inc.
dmi.board.version: A00
dmi.chassis.type: 8
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: 0.1
dmi.modalias: dmi:bvnDellInc.:bvrA08:bd04/18/2013:svnDellInc.:pnDellSystemXPSL322X:pvr:rvnDellInc.:rn0PJHXN:rvrA00:cvnDellInc.:ct8:cvr0.1:
dmi.product.name: Dell System XPS L322X
dmi.sys.vendor: Dell Inc.

CVE References

Revision history for this message
Daniele Dellafiore (ildella) wrote :
Revision history for this message
Apport retracing service (apport) wrote :

StacktraceTop:
 dns_packet_is_reply_for (key=0x558fe28027b0, p=0x558fe27fdb50) at ../src/resolve/resolved-dns-packet.c:2267
 dns_transaction_process_reply (t=0x558fe28027e0, p=0x558fe27fdb50) at ../src/resolve/resolved-dns-transaction.c:1010
 on_dns_packet.lto_priv.85 (s=<optimized out>, fd=<optimized out>, revents=<optimized out>, userdata=0x558fe28027e0) at ../src/resolve/resolved-dns-transaction.c:1107
 source_dispatch.lto_priv.92 (s=0x558fe27f2ea0) at ../src/libsystemd/sd-event/sd-event.c:2267
 sd_event_dispatch (e=e@entry=0x558fe2790280) at ../src/libsystemd/sd-event/sd-event.c:2626

Revision history for this message
Apport retracing service (apport) wrote : Stacktrace.txt
Revision history for this message
Apport retracing service (apport) wrote : StacktraceSource.txt
Revision history for this message
Apport retracing service (apport) wrote : ThreadStacktrace.txt
Changed in systemd (Ubuntu):
importance: Undecided → Medium
summary: - systemd-resolved crashed with SIGSEGV in sd_event_dispatch()
+ systemd-resolved crashed with SIGSEGV in dns_packet_is_reply_for()
tags: removed: need-amd64-retrace
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in systemd (Ubuntu):
status: New → Confirmed
Revision history for this message
Steve Langasek (vorlon) wrote :

Marking as a security bug, since this shows a crash in the packet parsing code that can potentially be triggered remotely by a hostile DNS server or spoofed responses.

information type: Private → Private Security
tags: added: zesty
Changed in systemd (Ubuntu):
assignee: nobody → Dimitri John Ledkov (xnox)
tags: added: bugpattern-needed
tags: added: artful
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Without understanding at all how the dns_packet structures work, it seems possible that a packet can pass DNS_PACKET_QR==1 check, yet when processed by dns_packet_extract fail the DNS_PACKET_QDCOUNT(p)>0, and hence end up with packet->question remain as NULL, resulting in bombing out as NULL pointer dereference.

Downloading a core dump to assert that above analysis is true, would be nice for sending this to upstream.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :
information type: Private Security → Public Security
tags: added: patch
Changed in systemd (Ubuntu):
status: Confirmed → Fix Committed
Changed in systemd (Ubuntu Zesty):
status: New → Fix Committed
Revision history for this message
Dimitri John Ledkov (xnox) wrote :
Revision history for this message
Dimitri John Ledkov (xnox) wrote :
Revision history for this message
Dimitri John Ledkov (xnox) wrote :
Changed in systemd (Ubuntu Zesty):
status: Fix Committed → Confirmed
Changed in systemd (Ubuntu Yakkety):
status: New → Confirmed
Changed in systemd (Ubuntu Xenial):
status: New → Confirmed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 233-6ubuntu3

---------------
systemd (233-6ubuntu3) artful; urgency=medium

  * resolved: fix null pointer dereference crash (LP: #1621396)

 -- Dimitri John Ledkov <email address hidden> Mon, 22 May 2017 09:29:22 +0100

Changed in systemd (Ubuntu):
status: Fix Committed → Fix Released
description: updated
Revision history for this message
Tyler Hicks (tyhicks) wrote :

I've requested a CVE from MITRE for this issue.

tags: added: apport-request-retrace
Revision history for this message
Tyler Hicks (tyhicks) wrote :

Lennart pointed out in the upstream pull request that systemd-resolved is respawned after crashing. Therefore, Ubuntu Security considers this security issue to be a low priority. To reduce the risk of regressions in security updates, our general rule is to only perform security updates that fix a medium or higher issue or wait until around five low issues have accumulated. The fix is simple and low risk but there's always inherent risk in building/publishing/installing new binaries. We'll include the fix in a future security update if there are new issues discovered in systemd.

Changed in systemd (Ubuntu Xenial):
importance: Undecided → Low
Changed in systemd (Ubuntu Yakkety):
importance: Undecided → Low
Changed in systemd (Ubuntu Zesty):
importance: Undecided → Low
Revision history for this message
Tyler Hicks (tyhicks) wrote :

@xnox you previously mentioned that you had some systemd SRUs to prepare. Feel free to include this fix in those SRUs to address the error tracker reports. It just doesn't quite make sense to do a standalone security update of the init daemon for a low priority security issue.

This bug is an interesting corner case of not being quite important enough to warrant a security update yet being enough of an annoyance that it warrants an SRU. My apologies for not noticing this fact earlier.

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Thank you for uploading this stable release update! To ease the SRU review process and later package validation, could you please update the bug description to include the relevant SRU information [1]? Especially the Regression Potential field that's missing here.

[1] https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template

description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Daniele, or anyone else affected,

Accepted systemd into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/232-21ubuntu4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in systemd (Ubuntu Zesty):
status: Confirmed → Fix Committed
tags: added: verification-needed
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

There is now one crash report in proposed. I'm not sure if this is because resolved was running, and has not been restarted yet. Or there is still this genuine crash present in the proposed package.....

_systemctl try-restart systemd-resolved.service || true

is called in the postinst, so the daemon should have been restarted.

Revision history for this message
Brian Murray (brian-murray) wrote :

Its worth noting that apport creates an initial .crash file without much information in it, after the user chooses to send the crash to the Error Tracker then some information gathering is performed. This information gathering stage includes adding the version of the package. Looking at the particular instance, https://errors.ubuntu.com/oops/3680877e-5046-11e7-89b7-fa163e54c21f, with the version of the package from -proposed a couple of things stand out.

Date: Thu Jun 8 12:35:23 2017

This is before the package was accepted into -proposed. Additionally there is this:

UnreportableReason:
Неполадка произошла с программой /lib/systemd/systemd-resolved, в которую были внесены изменения с момента её аварийного завершения работы.

So this is not a genuine crash with the version of the package from -proposed.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

After checking the tracker, there are no new crashes reported for the proposed version of the package (systemd amd64 232-21ubuntu4) marking as verification done for zesty.

tags: added: verification-done
removed: verification-needed
tags: added: verification-done-zesty
Revision history for this message
Apport retracing service (apport) wrote : Updated stack trace from duplicate bug 1699527

Package: systemd 232-21ubuntu3
ProcCmdline: /lib/systemd/systemd-resolved

Revision history for this message
Apport retracing service (apport) wrote : Dependencies.txt
Revision history for this message
Apport retracing service (apport) wrote : ProcMaps.txt
Revision history for this message
Apport retracing service (apport) wrote : Stacktrace.txt
Revision history for this message
Apport retracing service (apport) wrote : ThreadStacktrace.txt
tags: removed: apport-request-retrace
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 232-21ubuntu4

---------------
systemd (232-21ubuntu4) zesty; urgency=medium

  * Cherrypick upstream commit to enable system use kernel maximum limit for
    RLIMIT_NOFILE isntead of hard-coded (low) limit of 65536. (LP: #1686361)
  * debian/tests/root-unittests: disable execute and seccomp tests on arm
    test-seccomp and test-execute fail on arm64 kernels. Marking both tests as
    expected failures. An upstream bug report is filed to resolve these.
    (LP: #1672499)
  * Cherrypick upstream patch for platform predictable interface names.
    (LP: #1686784)
  * resolved: fix null pointer dereference crash (LP: #1621396)
  * Cherrypick core/timer downgrade message about random time addition
    (LP: #1692136)

 -- Dimitri John Ledkov <email address hidden> Wed, 24 May 2017 16:26:16 +0100

Changed in systemd (Ubuntu Zesty):
status: Fix Committed → Fix Released
Revision history for this message
Steve Langasek (vorlon) wrote : Update Released

The verification of the Stable Release Update for systemd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Changed in systemd (Ubuntu Xenial):
milestone: none → ubuntu-16.04.3
Changed in systemd (Ubuntu Yakkety):
status: Confirmed → In Progress
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Daniele, or anyone else affected,

Accepted systemd into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/229-4ubuntu18 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in systemd (Ubuntu Xenial):
status: Confirmed → Fix Committed
tags: added: verification-needed verification-needed-xenial
removed: verification-done
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

As per tracker, we have not had any reports of this crash in resolved in 16.04. Thus this fix is mostly advisory / precautionary one for xenial. Marking as verified. There are now even less chances for resolved to crash on systems that use resolved

tags: added: verification-done-xenial
removed: verification-needed verification-needed-xenial
Revision history for this message
Steve Langasek (vorlon) wrote :

LP: #1704677 is reported as a regression in this SRU. Marking verification failed pending resolution.

tags: added: verification-failed-xenial
removed: verification-done-xenial
Revision history for this message
Adam Conrad (adconrad) wrote :

Hello Daniele, or anyone else affected,

Accepted systemd into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/229-4ubuntu19 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-needed verification-needed-xenial
removed: verification-failed-xenial
Revision history for this message
Dimitri John Ledkov (xnox) wrote :
tags: added: verification-done verification-done-xenial
removed: verification-needed verification-needed-xenial
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 229-4ubuntu19

---------------
systemd (229-4ubuntu19) xenial; urgency=medium

  * debian/extra/units/systemd-resolved.service.d/resolvconf.conf: partially
    revert, by removing ExecStart|StopPost lines, as these are not needed on
    xenial and generate warnings in the journal. (LP: #1704677)

systemd (229-4ubuntu18) xenial; urgency=medium

  * debian/extra/units/systemd-resolved.service.d/resolvconf.conf: if resolved
    is going to be started, make sure this blocks network-online.target.
    (LP: #1673860)
  * networkd: cherry-pick support for setting bridge port's priority
    (LP: #1668347)
  * Cherrypick upstream commit to enable system use kernel maximum limit for
    RLIMIT_NOFILE isntead of hard-coded (low) limit of 65536. (LP: #1686361)
  * Cherrypick upstream patch for platform predictable interface names.
    (LP: #1686784)
  * resolved: fix null pointer dereference crash (LP: #1621396)
  * Cherrypick core/timer downgrade message about random time addition
    (LP: #1692136)
  * SECURITY UPDATE: Out-of-bounds write in systemd-resolved (LP: #1695546)
    - CVE-2017-9445
  * Cherry-pick subset of patches to introduce infinity value in logind.conf
    for UserTasksMax (LP: #1651518)

 -- Dimitri John Ledkov <email address hidden> Mon, 17 Jul 2017 17:00:42 +0100

Changed in systemd (Ubuntu Xenial):
status: Fix Committed → Fix Released
no longer affects: systemd (Ubuntu Yakkety)
To post a comment you must log in.