proftpd service fails to restart (including via logrotate)

Bug #1246245 reported by Goldhead
366
This bug affects 80 people
Affects Status Importance Assigned to Milestone
proftpd-dfsg (Debian)
Fix Released
Unknown
proftpd-dfsg (Ubuntu)
Fix Released
Medium
Brian Morton
Trusty
Fix Released
Medium
Unassigned

Bug Description

SRU request for Trusty

[Impact]

 * ProFTPd exits with signal 15 when restarted via init script.
   Worse yet, init script returns success.

 * Since logrotate runs via cron and performs a restart, this causes
   FTP service to stop and results in unexpected service downtime.

 * Final version of upstream patch at https://anonscm.debian.org/git/pkg-proftpd/proftpd-dfsg.git/commit/?id=23c5e44b3c30a082a0e15279961c3c4c3d7393fd

 * Only trusty and prior are affected

[Test Case]

 * bmorton@maui:~$ sudo service proftpd restart
   * Stopping ftp server proftpd [ OK ]
   * Starting ftp server proftpd [ OK ]
   bmorton@maui:~$ sudo service proftpd status
   ProFTPD is started in standalone mode, currently not running.

 * After a restart, status should say "currently running"

[Regression Potential]

 * Failure to start/stop/restart proftpd service

 * Patched package currently running on production host without issues

[Other Info]

 * This is a very old bug affecting a substantial number of users

 * Early AM nagios notifications about FTP service are undesirable

-------------------------------------------------------------------------

proftpd-basic 1.3.5~rc3-2 from Ubuntu 13.10
proftpd-basic 1.3.5~rc3-2.1ubuntu2 from Ubuntu 14.04

Init script from proftpd-basic package contains the BUG: when you run /etc/init.d/proftpd restart it fails because of there is the race between pidfile removal and start() which checks pidfile existency:

---
ProFTPD is started in standalone mode, currently running.
root@aa:~# /etc/init.d/proftpd restart
 * Stopping ftp server proftpd [ OK ]
 * Starting ftp server proftpd [ OK ]
root@aa:~# /etc/init.d/proftpd status
ProFTPD is started in standalone mode, currently not running.
---

the next workaround helps:

---
--- /etc/init.d/proftpd.orig 2013-10-30 13:52:46.791265726 +0400
+++ /etc/init.d/proftpd 2013-10-30 13:52:57.456265698 +0400
@@ -107,6 +107,7 @@
     fi
     if [ -f "$PIDFILE" ]; then
        start-stop-daemon --stop --signal $SIGNAL --quiet --pidfile "$PIDFILE"
+ sleep 1
         if [ $? = 0 ]; then
                log_end_msg 0
        else
---

Please, fix.

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in proftpd-dfsg (Ubuntu):
status: New → Confirmed
Revision history for this message
Cyrille Lapersonne (cyrille-lapersonne-apy-info) wrote :

Thanks Goldhead.
Your patch works perfectly.

Revision history for this message
Merlijn Hofstra (mhofstra) wrote :

This bug is also still present in 14.04

I tend to think that "sleep 1" is somewhat ugly in the init script, this solution works for me:

@@ -106,7 +106,7 @@
  fi
     fi
     if [ -f "$PIDFILE" ]; then
- start-stop-daemon --stop --signal $SIGNAL --quiet --pidfile "$PIDFILE"
+ start-stop-daemon --stop --signal $SIGNAL --retry 1 --quiet --pidfile "$PIDFILE"
      if [ $? = 0 ]; then
          log_end_msg 0
      else

Revision history for this message
joelparkerhenderson (joelparkerhenderson) wrote :

Thanks Merlijn, your patch works. IMHO your patch is a good way to really fix the race condition.

Revision history for this message
Francesco Paolo Lovergine (frankie) wrote : Re: [Bug 1246245] Re: proftpd service failed to restart

On Wed, Mar 19, 2014 at 02:11:36AM -0000, joelparkerhenderson wrote:
> Thanks Merlijn, your patch works. IMHO your patch is a good way to
> really fix the race condition.
>

Sorry, not. Adding sleeping seconds is the wrong way of fixing initscripts.
You should instead use the provided interface in start-stop-daemon via --retry.

--
Francesco P. Lovergine

Revision history for this message
Merlijn Hofstra (mhofstra) wrote : Re: proftpd service failed to restart

Hello Francesco,

That's why I submitted a revised patch that does use --retry rather than a sleep. Could you please merge this into the release for 14.04 as this bug is causing me some headaches? :-)

Revision history for this message
PRISMA Computer GmbH (j-infr-9) wrote :

This solution worked also for me (except I had to edit start-stop-daemon line by hand, I had problems with the diff file itself). But it is not merged in package until 2014-04-07.

Very sad. This bug leads to logrotate is killing proftpd.

description: updated
Revision history for this message
Francesco Paolo Lovergine (frankie) wrote : Re: [Bug 1246245] Re: proftpd service failed to restart

On Tue, May 06, 2014 at 12:00:46PM -0000, Alexander Birkner wrote:
> - fi
> - if [ -f "$PIDFILE" ]; then
> - start-stop-daemon --stop --signal $SIGNAL --quiet --pidfile "$PIDFILE"
> +      fi
> +      if [ -f "$PIDFILE" ]; then
> +         start-stop-daemon --stop --signal $SIGNAL --quiet --pidfile "$PIDFILE"
> + sleep 1

Unfortunately this is non-solution because timing depends on current load. It
just mitigates the issue, not solves it. And it is a non sense with systemd,
which probably is able to manage better the whole process

> - if [ $? = 0 ]; then
> - log_end_msg 0
> - else
> +          if [ $? = 0 ]; then
> +                 log_end_msg 0
> +         else
> ---
>
> Please, fix.
>

--
Francesco P. Lovergine

Revision history for this message
Benpro (benpro82) wrote : Re: proftpd service failed to restart
Revision history for this message
Benpro (benpro82) wrote :
Revision history for this message
Joel (joel-forsberg) wrote :

I just had logrotate trigger this bug on 14.04 (since it supposedly cannot just reload)

Added a re-worked patch, please review.

tags: added: patch
Revision history for this message
Joel (joel-forsberg) wrote :

I had a mistake, new patch.

Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "proftp.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

Revision history for this message
Joel (joel-forsberg) wrote :

I should probably read up on how to make a patch to the right codebase.

tags: removed: patch
Revision history for this message
Joel (joel-forsberg) wrote :

Upstream already had changed the init script, but I don't understand it fully.

(looking at master, commit b63ec2128f324610404efa991456b6206095531d File: debian/proftp-basic.init )
My questions:
1) Why exit if $2 is 0? (row 118)
2) Is it necessary to call start-stop-daemon twice if the first it exits with nonzero return value? (row 111 and 115)
3) row 123 looks wrong, attaching simple patch.
4) what is missing to close this ticket and make a corrected .deb? I'm willing to help but know little.

Revision history for this message
Joel (joel-forsberg) wrote :
Revision history for this message
danb1974 (danb1974) wrote :

Seems it's not that simple to do a proper fix, since the same signal() function is used for stop (TERM) and reload (HUP)

If you just add --retry to the first call to start-stop-daemon you will break reload, since proftpd will be killed (--retry changes the behaviour, now it's mission is to terminate the process, sending a TERM/KILL schedule until pid file is gone)

Looks like the clean approach would be to separate stop and reload and get rid of all the if/then/else dance and cascaded start-stop-daemon calls

Revision history for this message
danb1974 (danb1974) wrote :

My first attempt to cleanup the stop/reload mess, by creating a stop() function which has --retry schedule and leave a simple and clean signal() function. Feel free to point out if I'm totally wrong.

Revision history for this message
halfgaar (wiebe-halfgaar) wrote :

I'm having it too an 14.04. Why is it not fixed after 1.5 years? The Debian bug report on it is even older.

Revision history for this message
quentindemetz (quentindemetz) wrote :

Looks like it hasn't been backported to 14.04 - even though that is an LTS version...

Revision history for this message
bademux (bademux) wrote :

yes, the problem stays for latest ubuntu 14.04

Revision history for this message
mdsf (mydiffstuff) wrote :

Problem is still present even in 14.04.3 (!!!) :(

Think it will be in future LTS (16.x, 18.x, ...) too.

Revision history for this message
Pedro Braconnot (pbraconnot) wrote :

Yep Still there 14.04.3

Revision history for this message
Jiri Hoogeveen (wica128) wrote :

When will this be fixit?
It is still present in Ubuntu 14.04.3 Long Term Support!

Revision history for this message
Alexander Birkner (tyrola) wrote :

Yes, please merge the commit. Really frustrating on a LTS release...

Richard Moe (richardmoe)
no longer affects: proftpd-dfsg
Revision history for this message
Erik van Luijk (itserik) wrote :

When will this be merged?

Revision history for this message
Robert Everts (j-robert-4) wrote :

Problem still present. This way the Long Term Support seems like a bogus claim to me...

Revision history for this message
Poil (poil) wrote :

It's a long term bug ...

Revision history for this message
Alexander Birkner (tyrola) wrote :

Just 5 months to go until the next LTS Release. Maybe, who knows it's fixed in 16.04.

*ba dum tss*

Revision history for this message
Freddy (fslomka) wrote :

I cannot believe it is still not fixed.

On every new install I need to manually patch....

Josip Rodin (joy)
summary: - proftpd service failed to restart
+ proftpd service fails to restart (including via logrotate)
Revision history for this message
iblue (iblue) wrote :

I can confirm this bug. I just ran into it on a production machine. The bugfix is documented in this thread and here: http://stackoverflow.com/questions/23666697/proftpd-killed-signal-15-error-how-to-fix-logrotate-restart-error

It's killing FTP-Servers since 2013. And it's a one-line fix. If I had write access, I would patch it myself.

It would be totally awesome if anyone could fix it soon.

Revision history for this message
iblue (iblue) wrote :

This bug still affects us and I had to deploy a workaround. Is there a timeframe when this will get fixed? If I can somehow support this process, please let me know.

Revision history for this message
iblue (iblue) wrote :

Good morning. I see this bug is still unassigned. Is there a way to increase the chance of getting this fixed?

Revision history for this message
iblue (iblue) wrote :

Hi, any updates on this? It's just a one-liner, that shouldn't take long. If there is anything you need to get this fixed, please let me know.

Revision history for this message
iblue (iblue) wrote :

Good morning. This bug is still open and totally trival to fix. Could you please please fix it? That would be totally awesome.

Revision history for this message
andrea (a-zorzetto) wrote :

This bug is still present in 14.04.4

Is there anyone able to fix it please?

Revision history for this message
iblue (iblue) wrote :

Hi. The bug is still open. Could anyone please fix it?

Revision history for this message
iblue (iblue) wrote :

Good morning. The bug is still open. I wrote to the project admin, but didn't get an answer yet.

Revision history for this message
iblue (iblue) wrote :

Posting since ONE WEEK now.

Any progress on this? Could you fix it please? Is anyone reading this? Hellooooo... http://www.reactiongifs.com/r/weeds.gif

Revision history for this message
Luca Lesinigo (luca404) wrote :

I am reading this and I can confirm that the bug is still present on the latest 14.04 updates.
I can also confirm that the "sleep 1" workaround is enough to make it work perfectly.

Revision history for this message
wolf (l-wolf) wrote :

Checking in, can confirm too. I am subscribed to this bug for more than a year now. Really annoying and easy to fix.

Changed in proftpd-dfsg (Ubuntu):
assignee: nobody → wolf (l-wolf)
assignee: wolf (l-wolf) → nobody
Revision history for this message
Eugene Trotsan (eugene-trotsan-o) wrote :

Still happening on 16.04 - and really annoying having to manually fix this every update...

Revision history for this message
vagk (vagk-p) wrote :

This patch worked for me.
Taken changes from /etc/init.d/proftpd in 16.04

service proftpd restart
ok

service proftpd reload
ok

logrotate -f /etc/logrotate.d/proftpd-basic
ok

Can anyone else try this as well?

Revision history for this message
LucaChiampo (luca-chiampo) wrote :

The patch proposed from vagk-p in comment #43 works for me also!

Revision history for this message
Alexander Birkner (tyrola) wrote :

How many years would it take to create and deploy a new package, to solve this already-fixed very annoying issue finally?

Revision history for this message
Moses Moore (moses-ubuntu) wrote :

Got bitten by this bug today. Yikes.

Revision history for this message
LucaChiampo (luca-chiampo) wrote :

@ Moses Moore:
Apply the patch proposed from vagk-p in comment #43 it works like a charm!

Changed in proftpd-dfsg (Ubuntu):
assignee: nobody → Brian Morton (rokclimb15)
Revision history for this message
Brian Morton (rokclimb15) wrote :

Attaching debdiff for trusty. Editing bug to add SRU justification.

description: updated
description: updated
Revision history for this message
Robie Basak (racb) wrote :

Trusty task added. Please complete https://wiki.ubuntu.com/StableReleaseUpdates#Procedure and then subscribe ~ubuntu-sponsors.

Revision history for this message
Brian Morton (rokclimb15) wrote :

Revised debdiff with correct changelog message.

Mathew Hodson (mhodson)
Changed in proftpd-dfsg (Ubuntu):
importance: Undecided → Medium
Changed in proftpd-dfsg (Ubuntu Trusty):
importance: Undecided → Medium
tags: added: patch
Changed in proftpd-dfsg (Ubuntu Trusty):
status: New → Triaged
Revision history for this message
Brian Murray (brian-murray) wrote :

I've uploaded this to the Trusty SRU queue for review by the SRU team.

Changed in proftpd-dfsg (Ubuntu):
status: Confirmed → Fix Released
Changed in proftpd-dfsg (Debian):
status: Unknown → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Goldhead, or anyone else affected,

Accepted proftpd-dfsg into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/proftpd-dfsg/1.3.5~rc3-2.1ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in proftpd-dfsg (Ubuntu Trusty):
status: Triaged → Fix Committed
tags: added: verification-needed
Revision history for this message
Brian Morton (rokclimb15) wrote :

bmorton@maui:~$ sudo apt-get install proftpd-basic/trusty-proposed
Reading package lists... Done
Building dependency tree
Reading state information... Done
Selected version '1.3.5~rc3-2.1ubuntu2.2' (Ubuntu:14.04/trusty-proposed [amd64]) for 'proftpd-basic'
Suggested packages:
  openbsd-inetd inet-superserver proftpd-doc proftpd-mod-ldap
  proftpd-mod-mysql proftpd-mod-odbc proftpd-mod-pgsql proftpd-mod-sqlite
  proftpd-mod-geoip
The following packages will be upgraded:
  proftpd-basic
1 upgraded, 0 newly installed, 0 to remove and 13 not upgraded.
Need to get 1,967 kB of archives.
After this operation, 1,024 B of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ trusty-proposed/universe proftpd-basic amd64 1.3.5~rc3-2.1ubuntu2.2 [1,967 kB]
Fetched 1,967 kB in 3s (557 kB/s)
Preconfiguring packages ...
(Reading database ... 1144721 files and directories currently installed.)
Preparing to unpack .../proftpd-basic_1.3.5~rc3-2.1ubuntu2.2_amd64.deb ...
 * Stopping ftp server proftpd [ OK ]
Unpacking proftpd-basic (1.3.5~rc3-2.1ubuntu2.2) over (1.3.5~rc3-2.1ubuntu2.1) ...
Processing triggers for ureadahead (0.100.0-16) ...
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
Setting up proftpd-basic (1.3.5~rc3-2.1ubuntu2.2) ...
Installing new version of config file /etc/init.d/proftpd ...
 * Starting ftp server proftpd [ OK ]
Processing triggers for ureadahead (0.100.0-16) ...
bmorton@maui:~$ sudo nano /etc/apt/sources.list
bmorton@maui:~$ sudo service proftpd restart
 * Stopping ftp server proftpd [ OK ]
 * Starting ftp server proftpd [ OK ]
bmorton@maui:~$ sudo service proftpd status
ProFTPD is started in standalone mode, currently running.

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package proftpd-dfsg - 1.3.5~rc3-2.1ubuntu2.2

---------------
proftpd-dfsg (1.3.5~rc3-2.1ubuntu2.2) trusty; urgency=medium

  * Fix silent signal 15 exit on init script restart (LP: #1246245)

 -- Brian Morton <email address hidden> Wed, 15 Feb 2017 15:08:13 -0800

Changed in proftpd-dfsg (Ubuntu Trusty):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for proftpd-dfsg has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.