No network connectivity (NIC unmanaged) after upgrade

Status changed to 'Confirmed' because the bug affects multiple users.

Triaged/Critical: This will affect upgrades from Xenial to 18.04. It needs to be addressed properly.

Currently, upgrade fails this way because the postinst for network-manager is looking for a lower version than what is now in xenial-updates to gate the "migration" to netplan and writing an override file.

I'm reasonably certain that I can fix this in NM directly; for now I've uploaded a new version of network-manager for yakkety to my PPA (ppa:cyphermox/sru) to do the upgrade testing from Xenial to Yakkety. It involves first making sure xenial is fully updated to network-manager 1.2.6.

Any suggestions on how those of us affected by this can retroactively fix our systems? I am using "sudo dhclient p1p1" after every boot to get the ethernet up, but some programs such as "evolution" still do not have internet access.

Info I've gathered about this issue so far:

The netplan should be written to this location:

/etc/netplan/01-network-manager-all.yaml

And should contain:

# Let NetworkManager manage all devices on this system
network:
  version: 2
  renderer: NetworkManager

See this reference:

http://bazaar.launchpad.net/~ubuntu-core-dev/livecd-rootfs/trunk/view/head:/live-build/auto/build#L196

If the file is missing, it can be manually added.

Ubuntu transitioned to netplan right before version 1.2.2-0ubuntu8. See here:

https://lists.ubuntu.com/archives/ubuntu-devel/2016-July/039464.html

Additionally, this is related to these 2 bugs:

https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1638842
https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1658921

When 1.2.2 was released, a globally managed file was created (/etc/NetworkManager/conf.d/10-globally-managed-devices.conf), and a canceling file was also created (/etc/NetworkManager/conf.d/10-globally-managed-devices.conf).

https://git.launchpad.net/network-manager/tree/debian/network-manager.postinst#n48

For some reason, either the override file or the netplan file is missing on upgrades. Between the two of them, users are left without fully managed Ethernet and networking when upgrading from 16.04.

That's indeed what seems to be the case, and this has already been debugged. The issue is that the code that does the override on upgrade looks for the wrong version of NetworkManager, since it has been updated in Xenial.

This change has been in the process of being tested; I'll upload to the release proper as soon as I know that it really does work.

The netplan code itself should be created only on new installs; for an upgrade, it's not there, and thus we need to explicitly write an override for NM via NM postinst.

Ah, so this code:

 if dpkg --compare-versions "$2" le-nl "1.2.2-0ubuntu4"; then
  mkdir -p /etc/NetworkManager/conf.d || true
  # for old versions, override the global config with a null config
  touch /etc/NetworkManager/conf.d/10-globally-managed-devices.conf

Should have ge-nl rather than le-nl? Seems like it's backwards and that all new installs need the override, rather than old installs.

Also, why the decision to only do a netplan on install?

https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1671873 might be a duplicate (or related).

Could this bug get its description updated to include the SRU details? (as per https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template). We can't really reliably review this without it.
Thanks!

Hello Brian, or anyone else affected,

Accepted network-manager into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/network-manager/1.2.6-0ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

I upgraded a Xenial system to Yakkety, then enabled -proposed and installed the new version of network-manager.

bdmurray@clean-xenial-amd64:~$ apt-cache policy network-manager
network-manager:
  Installed: 1.2.6-0ubuntu1
  Candidate: 1.2.6-0ubuntu1.1
  Version table:
     1.2.6-0ubuntu1.1 500
        500 http://192.168.10.7/ubuntu yakkety-proposed/main amd64 Packages
 *** 1.2.6-0ubuntu1 500
        500 http://192.168.10.7/ubuntu yakkety-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     1.2.4-0ubuntu1 500
        500 http://192.168.10.7/ubuntu yakkety/main amd64 Packages
bdmurray@clean-xenial-amd64:~$ sudo apt-get install network-manager
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
  network-manager
1 upgraded, 0 newly installed, 0 to remove and 52 not upgraded.
Need to get 1,996 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://192.168.10.7/ubuntu yakkety-proposed/main amd64 network-manager amd64 1.2.6-0ubuntu1.1 [1,996 kB]
Fetched 1,996 kB in 0s (82.5 MB/s)
(Reading database ... 188256 files and directories currently installed.)
Preparing to unpack .../network-manager_1.2.6-0ubuntu1.1_amd64.deb ...
Unpacking network-manager (1.2.6-0ubuntu1.1) over (1.2.6-0ubuntu1) ...
Processing triggers for ureadahead (0.100.0-19) ...
Setting up network-manager (1.2.6-0ubuntu1.1) ...
Processing triggers for systemd (231-9ubuntu4) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for dbus (1.10.10-1ubuntu1.1) ...
bdmurray@clean-xenial-amd64:~$ sudo shutdown -r now
Connection to clean-xenial-amd64.local closed by remote host.
Connection to clean-xenial-amd64.local closed.

After the reboot I had network connectivity.

 $ ssh clean-xenial-amd64.local
Welcome to Ubuntu 16.10 (GNU/Linux 4.8.0-49-generic x86_64)

Setting to verification-done.

Verified network-manager 1.2.6-0ubuntu1.1 in yakkety-proposed. Upgrades with and without nplan installed appear to work as expecte.d

This bug was fixed in the package network-manager - 1.2.6-0ubuntu1.1

---------------
network-manager (1.2.6-0ubuntu1.1) yakkety; urgency=medium

  * debian/network-manager.postinst: drop in an empty override file for
    NetworkManager to manage all devices for upgrade from any version, as long
    as there is no netplan configuration yet. (LP: #1676547)

 -- Mathieu Trudel-Lapierre <email address hidden> Mon, 27 Mar 2017 17:02:27 -0400

The verification of the Stable Release Update for network-manager has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

The problem persists if you upgrade from 16.04.3 directly to 17.04 ( as described https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1658921/comments/23 )

> The problem persists if you upgrade from 16.04.3 directly to 17.04 ( as described https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1658921/comments/23 )

I can confirm this. My upgrade path was xenial -> zesty -> artful and I had to run "touch /etc/NetworkManager/conf.d/10-globally-managed-devices.conf" so network-manager could handle my enp2s0 device.

Possible duplicate https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1716034

Another possible duplicate: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1687262

Hello Brian, or anyone else affected,

Accepted network-manager into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/network-manager/1.4.4-1ubuntu3.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

I've verified this is fixed with the version of network-manager in zesty-proposed, using both the testcases (with and without nplan installed).

This bug was fixed in the package network-manager - 1.8.2-1ubuntu8

---------------
network-manager (1.8.2-1ubuntu8) artful; urgency=medium

  * debian/network-manager.postinst: drop in an empty override file for
    NetworkManager to manage all devices for upgrade from any version, as long
    as there is no netplan configuration yet. (LP: #1676547)

 -- Brian Murray <email address hidden> Mon, 11 Sep 2017 10:20:18 -0700

This bug was fixed in the package network-manager - 1.4.4-1ubuntu3.2

---------------
network-manager (1.4.4-1ubuntu3.2) zesty; urgency=medium

  * debian/network-manager.postinst: drop in an empty override file for
    NetworkManager to manage all devices for upgrade from any version, as long
    as there is no netplan configuration yet. (LP: #1676547)

 -- Brian Murray <email address hidden> Fri, 08 Sep 2017 11:29:32 -0700

I am truly sorry to say that I have tried many of the recommended solutions for this problem but to no avail. I am running Ubuntu 16.04 on a AMD 8-core desktop machine and if I ever ever ever even try to give a command like sudo apt-get update and apt-get upgrade I inevitably find myself with an operating system without ethernet connection. I have been unable to fix that no matter what I have tried so far. I have spent DAYS reading the blogs and trying to get a fix for this bug. In the end the only solution I could find so far was to use the original ssd back up disk to get back to the original system, before the sudo apt-get update etc commands. Can anyone in this forum or elsewhere explain in a simple and clear manner how to fix this major issue with ubuntu upgrades?

Also not to mention that even when I use my 16.04 without ever using the sudo apt-get update command, the wifi network is also not detected by ubuntu.

I just had the same problem...

I installed Ubuntu server 17.10 on a freshly formatted disk. During the install I was asked to configure the wired interface. Ok

After that I installed the meta-package ubuntu-desktop and re-booted. No wired interface available in NetworkManager. Wireless is ok.

I had to change "/etc/netplan/01-netcfg.yaml" from:

# This file describes the network interfaces available on your system
# For more information, see netplan(5).
network:
  version: 2
  renderer: networkd
  ethernets:
    enp3s0:
      dhcp4: yes

to:

# Let NetworkManager manage all devices on this system
network:
  version: 2
  renderer: NetworkManager

All is well after a reboot.

I do not have /etc/netplan directory on my ubuntu 14.04? Marco

This issue effects a manual (debootstrap) installation of 18.04.1. Ethernet devices are unmanaged by NetworkManager until a netplan configuration is added.

TJ - this isn't exactly the same issue if I understand correctly what you are doing. I ran into what I think is your issue last week. I installed using the 18.04.1 alternate server installation and then installed the 'ubuntu-desktop' meta package which pulled in network-manager. My interface was working, but not managed by network-manager until I added it as the renderer in the netplan yaml file.

Is that the same issue you are reporting?

Brian, I did something similar to what you just described and got hit by that. Is that a different issue?

On Thu, Aug 16, 2018 at 04:45:21PM -0000, Shahar Or wrote:
> Brian, I did something similar to what you just described and got hit by
> that. Is that a different issue?

Yes, the original bug report here was about upgrading Ubuntu desktop
systems.

--
Brian Murray

Ubuntu Server 18.04.3 fresh install.
Ethernet Unmanaged in Network Manager.
SOLVED. FIXED, etc.
-
I created file /etc/netplan/01-network-manager-all.yaml

Within I entered:
# Let NetworkManager manage all devices on this system
network:
  version: 2
  renderer: NetworkManager
--
I edited /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg

Within I ensured it only say:

network: {config: disabled}
---
I edit /etc/netplan/01-network-manager-all.yaml (If you don't nave then create)
within it I entered:
# Let NetworkManager manage all devices on this system
network:
  version: 2
  renderer: NetworkManager
----
rebooted
-----

Wooho, It works!
Thanks