tftpd-hpa fails to start when the NIC is not available at startup time

Status changed to 'Confirmed' because the bug affects multiple users.

Just to confirm that I am seeing this on 14.10.

Confirmed for 14.04 LTS server flavor as described also here:
http://ubuntuforums.org/showthread.php?t=2254382

Confirmed for 14.04 LTS.

Solved by modifying /etc/init/tftpd-hpa.conf

with:
start on (filesystem and net-device-up IFACE!=lo)

Cf: http://upstart.ubuntu.com/cookbook/#start-on

Beware, there is a bug in report #1 attachement: modified.conffile..etc.init.tftpd.hpa.conf.txt
There is a syntax error in "start on runlevel (filesystem and net-device-up IFACE!=lo)"

The proposed fix (from #1 as well as the corrected version from #5) does not solve the problem for me. Actually, when I look at what time the system tries to start the tftp-hpa process, it makes it even worse: The process gets started earlier (around ~11s vs ~13s)! Also, the relevant errors look somewhat different (IPv6 resolve error). Fwiw, I use /etc/network/interface to configure eth1 statically.

Default start script
...
Jan 5 14:38:30 testseat2 in.tftpd[1052]: cannot resolve local IPv6 bind address: ::(Name or service not known)
Jan 5 14:38:30 testseat2 kernel: [ 13.111093] init: tftpd-hpa main process (1052) terminated with status 66
Jan 5 14:38:30 testseat2 kernel: [ 13.111102] init: tftpd-hpa main process ended, respawning
...

Altered start script:
....
Jan 5 14:30:59 testseat2 kernel: [ 11.365352] init: tftpd-hpa main process (708) terminated with status 66
Jan 5 14:30:59 testseat2 kernel: [ 11.365360] init: tftpd-hpa main process ended, respawning
...

A workaround for my case was to bind to the IPv4 address explicitly:

In /etc/default/tftpd-hpa:

TFTP_ADDRESS="0.0.0.0:69"

@falstaff

Your workaround works also for my configuration, just changed TFTP_ADDRESS to 0.0.0.0:69 and rebooted

TFTP_ADDRESS="0.0.0.0:69"

worked for me on 14.04. The other workaround of: "start on (filesystem and net-device-up IFACE!=lo)" did not help at all.

Same experience as jay-o. Please fix :/

I did NOT need to edit TFTP_ADDRESS in /etc/default/tftpd-hpa. I only needed to change the 'start on' condition in /etc/init/tftpd-hpa.conf to:

start on (filesystem and net-device-up IFACE!=lo)

The service started successfully after I made this change, and rebooted.

As with comment #10:

TFTP_ADDRESS="0.0.0.0:69" worked for me on 14.04 (server edition). The other workaround of: "start on (filesystem and net-device-up IFACE!=lo)" did not help at all. Even tried: "start on (filesystem and started network-services)"

Maybe something to do with fact that I have 2 network interfaces?

Why is the status still reported as "start/running" when the process clearly is not running? This is, I suppose, why the respawning of the job by upstart does not get done even though the respawn stanza is present in the conf file?

The IPv6 definition of localhost (::1) is clearly wrong in the configuration file.

root@pkvmci808:~# cat /etc/default/tftpd-hpa
...
TFTP_ADDRESS="[::]:69"
...
root@pkvmci808:~# systemctl start tftpd-hpa.service
Job for tftpd-hpa.service failed because the control process exited with error code. See "systemctl status tftpd-hpa.service" and "journalctl -xe" for details.
ubuntu@pkvmci808:~$ cat /var/log/syslog
...
Jun 27 17:56:51 pkvmci808 in.tftpd[21463]: cannot resolve local IPv6 bind address: ::(Name or service not known)
...

root@pkvmci808:~# vi /etc/hosts
...
TFTP_ADDRESS="[::1]:69"
...
root@pkvmci808:~# systemctl start tftpd-hpa.service

Just tested here and confirmed that tftpd-hpa installs and runs perfectly fine on both a clean Ubuntu 14.04 and Ubuntu 16.04 system with its default configuration.

Changing from "[::]" to "0.0.0.0" is completely wrong as that turns off IPv6 support for tftpd-hpa which people do need.

My best guess as to why some folks need to do that change is because they somehow turned off IPv6 on their system, which then causes any daemon that's configured to bind both IPv4 and IPv6 to fail.

While it'd be nice if tftpd-hpa dealt with such setups a bit nicer, turning off IPv6 support on Ubuntu isn't something that we actively support (understand, that it's at your own risk) and such regressions are absolutely to be expected in such cases.

I've confirmed the following:
 - An Ubuntu 14.04 system with IPv6 enabled and no IPv6 address works fine
 - An Ubuntu 16.04 system with IPv6 enabled and no IPv6 address works fine
 - An Ubuntu 14.04 system with IPv6 disabled fails as described in this bug report
 - An Ubuntu 16.04 system with IPv6 disabled fails as described in this bug report

And for good measure:
 - An Ubuntu 14.04 system with IPv6 enabled and an IPv6 address works fine too
 - An Ubuntu 16.04 system with IPv6 enabled and an IPv6 address works fine too

There may still be some kind of race in tftpd-hpa which I'm not hitting here and that would cause it to only work after a little while (similar to the other bug which was referenced here).
But if it's not working at all and only works after you change it to bind 0.0.0.0, that's most likely because you reconfigured your system's networking in a way we don't support.

Thanks Stéphane.

Let's mark this Incomplete until someone provides exact steps to reproduce. If the steps are to disable IPv6, then this bug will be Invalid or Won't Fix. If there are some other steps that can be followed to reproduce this, then we can look again.

To those affected: please carefully read Stéphane's comment above. If you have steps to reproduce this problem that do not involve disabling IPv6 and you believe that those steps demonstrate a bug in vsftpd or vsftpd packaging that leads to this failure, then please post your steps here and change the bug status back to New.

[Expired for tftp-hpa (Ubuntu) because there has been no activity for 60 days.]

Hello,

I may have found a reproducer and the explanation for this bug.

Reproducer
==========
1) Set a trusty VM with one NIC
2) Install tftp-hpa
3) From the VM console :
$ sudo -s
# service tftpd-hpa status
tftpd-hpa start/running, process 867
# service tftpd-hpa stop
tftpd-hpa stop/waiting
# ps aux | grep tftp | grep -v grep
#
# ifdown eth0
# service tftpd-hpa start
# ps aux | grep tftp | grep -v grep
#

If the NIC is down, the tftpd daemon will not start and the following will appear in /var/log/syslog :
Jan 9 16:12:26 TrustyS in.tftpd[1114]: cannot resolve local IPv6 bind address: ::(Name or service not known)

Starting tftpd while the network interface is still down, while rare, may happen is situation where the NIC is slow to come up or, for instance, for people using tftpd on laptop with Wifi only connectivity.

This is caused by the fact that tftpd will do a call to getaddrinfo() if an IPv6 host address is set, which is the case with [::]:69. This call fails if the NIC is not configured.

If no IPv6 address is set and :69 is used, the codepath reverts to INADDR_ANY which does not do the getaddrinfo(). The tftpd daemon binds correctly to the interface even if it is down as we can see :

# service tftpd-hpa stop
# ifdown eth0
# /usr/sbin/in.tftpd --listen --user tftp --address :69 --secure /var/lib/tftpboot
# ps aux | grep tftp | grep -v grep
root 1712 0.0 0.0 15120 152 ? Ss 16:25 0:00 /usr/sbin/in.tftpd --listen --user tftp --address :69 --secure /var/lib/tftpboot
# lsof -i | grep tftp
in.tftpd 1712 root 4u IPv4 10675 0t0 UDP *:tftp
in.tftpd 1712 root 5u IPv6 10676 0t0 UDP *:tftp

Debian is currently debating the same issue in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=771441 and the current option is to change the default to :69.

I suggest to fix this bug by using the :69 value that will eventually also be used in debian.

This bug was fixed in the package tftp-hpa - 5.2+20150808-1ubuntu2

---------------
tftp-hpa (5.2+20150808-1ubuntu2) zesty; urgency=medium

  * Replace the default value of TFTP_ADDRESS to :69 instead of [::]:69.
    The previous default caused a failure to start when the NIC is not
    available at startup time (LP: #1342580)

 -- Louis Bouchard <email address hidden> Thu, 09 Feb 2017 18:13:54 +0100

+ # Replace previous dual-stack syntax
+ # that fails to work if NIC is not available
+ # (LP: #1342580)
+ if [ "${TFTP_ADDRESS}" = "[::]:69" ]; then
+ TFTP_ADDRESS=":69"
        fi

Please include a version guard in this, so that if the user insists on setting the address to [::]:69, we aren't constantly rewriting it. Compare the block of code immediately above:

        # Move from IPv4-only to dual-stack
        if [ "${TFTP_ADDRESS}" = "0.0.0.0:69" ] && \
         dpkg --compare-versions $2 lt 5.2-7ubuntu3; then
[...]

The change is ok other than that, but a version check should be included as part of this SRU.

Rejecting the current upload; please reupload with the version check.

An upload of tftp-hpa to xenial-proposed has been rejected from the upload queue for the following reason: "needs a version check for the rewrite".

Accepted the yakkety package to -proposed - unfortunately the review script crashed mid-process and didn't manage to write the comment about it.

Please test the yakkety-proposed 5.2+20150808-1ubuntu1.16.10.1 version of the package and mark it as verification-done if the bug is fixed and no regressions found. Thank you!

Hello Oleksij, or anyone else affected,

Accepted tftp-hpa into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/tftp-hpa/5.2+20150808-1ubuntu1.16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Oleksij, or anyone else affected,

Accepted tftp-hpa into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/tftp-hpa/5.2-7ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

FYI, test was performed for T, X & Y, upgrading using proposed with and without a pre-set value in /etc/default/tftpd-hpa.

Value was set to :69 unless previously modified.

This bug was fixed in the package tftp-hpa - 5.2-7ubuntu3.1

---------------
tftp-hpa (5.2-7ubuntu3.1) trusty; urgency=medium

  * Replace the default value of TFTP_ADDRESS to :69 instead of [::]:69.
    The previous default caused a failure to start when the NIC is not
    available at startup time (LP: #1342580)

 -- Louis Bouchard <email address hidden> Thu, 09 Feb 2017 18:13:54 +0100

The verification of the Stable Release Update for tftp-hpa has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package tftp-hpa - 5.2+20150808-1ubuntu1.16.04.1

---------------
tftp-hpa (5.2+20150808-1ubuntu1.16.04.1) xenial; urgency=medium

  * Replace the default value of TFTP_ADDRESS to :69 instead of [::]:69.
    The previous default caused a failure to start when the NIC is not
    available at startup time (LP: #1342580)

 -- Louis Bouchard <email address hidden> Tue, 07 Mar 2017 12:00:08 +0100

This bug was fixed in the package tftp-hpa - 5.2+20150808-1ubuntu1.16.10.1

---------------
tftp-hpa (5.2+20150808-1ubuntu1.16.10.1) yakkety; urgency=medium

  * Replace the default value of TFTP_ADDRESS to :69 instead of [::]:69.
    The previous default caused a failure to start when the NIC is not
    available at startup time (LP: #1342580)

 -- Louis Bouchard <email address hidden> Tue, 07 Mar 2017 12:01:25 +0100