nginx crashes with dump because of segfault

Thank you for taking the time to file a bug report.

Please try to obtain a backtrace following the instructions at http://wiki.ubuntu.com/DebuggingProgramCrash and upload the backtrace (as an attachment) to the bug report. This will greatly help us in tracking down your problem.

Since there is not enough information in your report to begin triage or to
differentiate between a local configuration problem and a bug in Ubuntu, I
am marking this bug as "Incomplete". We would be grateful if you would:
provide a more complete description of the problem, explain why you
believe this is a bug in Ubuntu rather than a problem specific to your
system, and then change the bug status back to "New".

For local configuration issues, you can find assistance here:
http://www.ubuntu.com/support/community

Hi,
i'm trying hard to create that backtrace, but it doesn't work:

 ...
dynamically loaded /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.10 needs package libxml2, queueing
Installing extra package nginx-core to get ExecutablePath
Get:1 [3029 kB]
Fetched 3029 kB in 0s (0 B/s)
Extracting downloaded debs...
dpkg-source: info: extracting nginx in nginx-1.18.0
dpkg-source: info: unpacking nginx_1.18.0.orig.tar.gz
dpkg-source: info: unpacking nginx_1.18.0-0ubuntu1.2.debian.tar.xz
dpkg-source: info: using patch list from debian/patches/series
dpkg-source: info: applying 0002-Make-sure-signature-stays-the-same-in-all-nginx-buil.patch
dpkg-source: info: applying 0003-define_gnu_source-on-other-glibc-based-platforms.patch
dpkg-source: info: applying ubuntu-branding.patch
dpkg-source: info: applying nginx-fix-pidfile.patch
dpkg-source: info: applying CVE-2021-23017-1.patch
dpkg-source: info: applying CVE-2021-23017-2.patch
W: Der Download wird als root und nicht Sandbox-geschützt durchgeführt, da auf die Datei »nginx_1.18.0-0ubuntu1.2.dsc« durch den Benutzer »_apt« nicht zugegriffen werden kann. - pkgAcquire::Run (13: Permission denied)
--- stack trace ---
#0 0x00007fbf59548593 in ?? ()
No symbol table info available.
#1 0x0000000000000000 in ?? ()
No symbol table info available.
--- source code stack trace ---
#0 0x00007fbf59548593 in ?? ()
#1 0x0000000000000000 in ?? ()

It seems the symbols aren't there. Can you help me here with this problem or do i have to ask that anywhere else ?
Sorry, i'm completely new to filing a bug in Ubuntu.

Thanks,

Bernd

Hi Bernd,

In the web page I linked in my first comment [1] you can find a section telling you multiple ways to get the debug symbols in place. Did you try any of the methods listed there?

[1] https://wiki.ubuntu.com/DebuggingProgramCrash

Hi,

Please note that you will need to install libperl's debuginfo as well as nginx's. Thanks.

Hi,
i'm trying hard to install the debug symbols. When i succeed i'll come back to you.
And i will install the debug symbols from libperl. Thanks for your help.
Bernd

My stack trace still looks poor:
--- stack trace ---
#0 0x00007fbf59548593 in ?? ()
No symbol table info available.
#1 0x0000000000000000 in ?? ()
No symbol table info available
--- source code stack trace ---
#0 0x00007fbf59548593 in ?? ()
#1 0x0000000000000000 in ?? ()

I installed all dbgsym packages via:
find-dbgsym-packages (pidof nginx) 2>/dev/null |xargs aptitude install -y.
It installed all dbgsym packages except one because i had problems with dependencies:
It said it would need libx11-6=2:1.6.9-2ubuntu1.3 for libx11-6-dbgsym, but 2:1.6.9-2ubuntu1.2 is installed.

Additionally i didn't find debug-symbols for libperl5.30:
root@documents-oo:/var/crash/syslog# aptitude search libperl5.30-dbgsym
root@documents-oo:/var/crash/syslog# aptitude search libperl5.30-dbg
no hit.
I installed libperl5i-perl-dbgsym but that didn't help.
I googled for libperl5.30-dbg, but didn't find anything.

Also "find-dbgsym-packages /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0" didn't find anything.

I'm running out of ideas.

Bernd

If i understand https://wiki.ubuntu.com/DebuggingProgramCrash correctly,
this sequence should download all necessary debug symbols automatically:
apport-retrace -s -S system -v /var/crash/_usr_sbin_nginx.0.crash -C /var/crash/nginx

But also this creates a poor stacktrace:
--- stack trace ---
#0 0x00007fbf59548593 in ?? ()
No symbol table info available.
#1 0x0000000000000000 in ?? ()
No symbol table info available.
--- source code stack trace ---
#0 0x00007fbf59548593 in ?? ()
#1 0x0000000000000000 in ?? ()

I get this warning:
"W: Der Download wird als root und nicht Sandbox-geschützt durchgeführt, da auf die Datei »nginx_1.18.0-0ubuntu1.2.dsc« durch den Benutzer »_apt« nicht zugegriffen werden kann. - pkgAcquire::Run (13: Permission denied)"
which means translated
"W: Download is performed unsandboxed as root as file »nginx_1.18.0-0ubuntu1.2.dsc' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)"
I googled that and found it's just a warning, no problem.

Bernd

"We would be grateful if you would:
provide a more complete description of the problem, explain why you
believe this is a bug in Ubuntu rather than a problem specific to your
system, and then change the bug status back to "New"."

OK.
Nginx created twice a segfault.
/var/log/syslog:
Oct 31 00:02:06 documents-oo kernel: [1340433.341063] nginx[433202]: segfault at 10 ip 00007f6a15e08593 sp 00007ffeef04e220 error 4 in libperl.so.5.30.0[7f6a15da0000+166000]
Oct 31 00:02:06 documents-oo kernel: [1340433.355995] Code: 48 89 43 10 48 83 c4 18 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 1f 40 00 0f b6 7f 30 48 c1 e8 03 48 29 f8 48 89 c3 74 89 48 8b 02 <4c> 8b 68 10 4d 85 ed 0f 84 28 01 00 00 0f b6 40 30 49 c1 ed 03 49
the second segfault looks the same.
I think this is a bug and not a configuration problem because i changed the configuration just a little bit.
And segfaults are normally bugs in the program code.

Bernd

Hi Bernd,

For your issues with apport-retrace, yes it should download the files it needs, but in your case seems to fail for permission issues. Could you mabe just try it with sudo and rebuild instead of a direct binary path like:
  $ sudo apport-retrace --sandbox system --rebuild-package-info --stdout /var/crash/_usr_sbin_nginx.0.crash
Let us know if that creates a more useful output.

I agree that a crash is a problem either way. But we need to understand and/or recreate it to find if it is in upstream code or any Ubuntu modification/integration to then discuss/work/resolve the problem in the right place.

Therefore even if (hopefully not) your crash continues to refuse to give you proper symbols - did you try the approach of e.g. taking a clean Ubuntu VM, and installing/configuring it the way you did on the system this happened? This might clarify how related (or not) your config changes are and the same for logrotate that you expected to be related.
If you manage to recreate the issue please let us know the steps to reproduce - with those the debugging can happen on our side and everyone can e.g. compare different versions of nginx and/or releases of Ubuntu against this config.

Incomplete again (for now) as just with the addresses and no steps to recreate no one can help yet. Please continue the discussion and the work to get better debug output and (even better) a repro.

I was happy for a second as I thought I found a related case, but this is the very same case on askubuntu [1], so no more insights from there.

[1]: https://askubuntu.com/questions/1377700/ubuntu-20-04-3-nginx-crashes-with-segfault

Unfortunately it still does not work:

apport-retrace -v --sandbox system --rebuild-package-info --stdout /var/crash/_usr_sbin_nginx.0.crash
 ...

W: Der Download wird als root und nicht Sandbox-geschützt durchgeführt, da auf die Datei »nginx_1.18.0-0ubuntu1.2.dsc« durch den Benutzer »_apt« nicht zugegriffen werden kann. - pkgAcquire::Run (13: Permission denied)
--- stack trace ---
#0 0x00007f7d0133e593 in ?? ()
No symbol table info available.
#1 0x0000000000000000 in ?? ()
No symbol table info available.
--- source code stack trace ---
#0 0x00007f7d0133e593 in ?? ()
#1 0x0000000000000000 in ?? ()

But i'm wondering now if it's really a nginx bug.
The error said "Oct 31 00:02:06 documents-oo kernel: [1340433.341063] nginx[433202]: segfault at 10 ip 00007f6a15e08593 sp 00007ffeef04e220 error 4 in libperl.so.5.30."
So libperl.so.5.30 caused the segfault, isn't it ?
And that file belongs to libperl5.30.
root@documents-oo:/proc/4108842/fd# dpkg-query -S /usr/lib/x86_64-linux-gnu/libperl.so.5.30
libperl5.30:amd64: /usr/lib/x86_64-linux-gnu/libperl.so.5.30

So is it really a nginx bug ?

Bernd

Bernd

[Expired for nginx (Ubuntu) because there has been no activity for 60 days.]

Hi Bernd,

Are you still observing this crash?

It seems you are still facing permission issues, as Christian pointed out in his comment above. It would be nice if you could work around those permission issues to get the proper debuginfo to proceed with this bug.

Since this bug was automatically set to expired due to lack of activity after you requested more information, I will set it back to incomplete again, until you can provide more detailed information so we can move forward with this one.

[Expired for nginx (Ubuntu) because there has been no activity for 60 days.]