Network Manager doesn't set metric for local networks any more, causing connection issues

Status changed to 'Confirmed' because the bug affects multiple users.

Please:
  1. Report to <https://bugzilla.gnome.org/>.
  2. Paste the new report URL here.
  3. Set this bug status back to "confirmed".

Thank you.

Thank you.

NM-APPLET Returns different error code sometime 5, 7 ,9,11 etc can not connect to internet without error. This is critical cause can not work on my computer on the internet. The system works outside internet. works fine with Ubuntu 14.4.2 and Ubuntu 14.10 This bug is in 15.04 only !

This bug was fixed in the package network-manager - 0.9.10.0-4ubuntu16

---------------
network-manager (0.9.10.0-4ubuntu16) wily; urgency=medium

  * debian/patches/fix_default_routes.patch: rework routing priorities and how
    default routes are added for devices and VPNs so that we can successfully
    run multiple devices on the same subnet without packet loss. (LP: #1436330)

 -- Mathieu Trudel-Lapierre <email address hidden> Fri, 08 May 2015 17:23:28 -0400

Hello Jason, or anyone else affected,

Accepted network-manager into vivid-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/network-manager/0.9.10.0-4ubuntu15.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

After installing the update, my ethernet and wireless routes still have the same metric:

$ ip route
default via 192.168.100.1 dev eth0 proto static metric 100
169.254.0.0/16 dev eth0 scope link metric 1000
192.168.100.0/24 dev eth0 proto kernel scope link src 192.168.100.150
192.168.100.0/24 dev wlan0 proto kernel scope link src 192.168.100.151
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1

I think this is a verification-failed for now.

After installing Ubuntu 15.04 my routing table looked like:

$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.10.100.3 0.0.0.0 UG 1024 0 0 eth0
10.10.96.0 0.0.0.0 255.255.248.0 U 0 0 0 eth0
10.10.96.0 0.0.0.0 255.255.248.0 U 0 0 0 wlan0
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 wlan0

After installing network-manager_0.9.10.0-4ubuntu16_amd64.deb and libnm-util2_0.9.10.0-4ubuntu16_amd64.deb my routing table looked like:

$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.10.100.3 0.0.0.0 UG 100 0 0 eth0
0.0.0.0 10.10.100.3 0.0.0.0 UG 400 0 0 wlan0
10.10.96.0 0.0.0.0 255.255.248.0 U 0 0 0 wlan0
10.10.96.0 0.0.0.0 255.255.248.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 wlan0

However, my DNS servers are:

$ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.10.100.11
nameserver 10.10.100.12

So, if I understand correctly, even with version 0.9.10.0-4ubuntu16 of the network manager, there is still a metric issue which could cause the system to attempt to connect to my DNS servers through the wireless connection even when perfectly connected through the ethernet cable.

For what it's worth, network-manager 0.9.10.0-4ubuntu15.2 seems to fix the problem for me.

`route -n` with *ubuntu15.1 with both WiFi and Ethernet connections:

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.17.75.1 0.0.0.0 UG 1024 0 0 eth0
10.17.75.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan0
10.17.75.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 wlan0

And with *ubuntu15.2:

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.17.75.1 0.0.0.0 UG 100 0 0 eth0
0.0.0.0 10.17.75.1 0.0.0.0 UG 400 0 0 wlan0
10.17.75.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan0
10.17.75.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 wlan0

And in both cases for me /etc/resolve.conf has always been:

nameserver 127.0.1.1

I get the correct Metric with
sudo apt-get install network-manager=0.9.10.0-4ubuntu15.2 libnm-util2=0.9.10.0-4ubuntu15.2
now:

route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.10.0.254 0.0.0.0 UG 100 0 0 eth1
0.0.0.0 10.10.0.254 0.0.0.0 UG 400 0 0 wlan0
10.10.0.0 0.0.0.0 255.255.0.0 U 0 0 0 wlan0
10.10.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 wlan0

@jderose: it don't think it did fix the problem for you:

10.17.75.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan0
10.17.75.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

Both your wlan0 and eth0 devices have the same metric.

@tagrain:

10.10.0.0 0.0.0.0 255.255.0.0 U 0 0 0 wlan0
10.10.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1

both your devices have the same metric also.

Ah you're correct. I was looking at the wrong rows.

I get the correct metrics on:

0.0.0.0 10.10.0.254 0.0.0.0 UG 100 0 0 eth1
0.0.0.0 10.10.0.254 0.0.0.0 UG 400 0 0 wlan0

But the wrong metrics on the ones that count.

10.10.0.0 0.0.0.0 255.255.0.0 U 0 0 0 wlan0
10.10.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1

The problem connecting to websites etc with wifi on still manifests.

Verify Failed

@jderose, @tagrain, as mdeslaur mentioned in post #12, the metric is correctly set for the gateway route, but it is incorrectly set for the local area network route.

Installed 0.9.10.0-4ubuntu15.2 from proposed and its not working, same behavior

pat@samsung930X3G:~$ ip route
default via 10.0.1.1 dev eth0 proto static metric 100
default via 10.0.1.1 dev wlan0 proto static metric 400
10.0.1.0/24 dev wlan0 proto kernel scope link src 10.0.1.26
10.0.1.0/24 dev eth0 proto kernel scope link src 10.0.1.127
169.254.0.0/16 dev wlan0 scope link metric 1000

I discussed this with Mathieu this morning on #ubuntu-devel, he commented that to properly fix, we need to add additional logic that includes the new metrics when creating the default routes. This will require a backport from NM 1.x, however as it won't merge cleanly, there's some rework involved in order to get it working on our older version of NM.

I tried silo-027 and it seems better (I think).

The packet loss problem remains though: thre's a ~20s break in traffic when the
new interface comes up or goes down though - for example:

laney@nightingale> ping -I wlan0 orangesquash.org.uk ~
PING orangesquash.org.uk (85.119.82.37) from 192.168.1.135 wlan0: 56(84) bytes of data.
64 bytes from cripps.orangesquash.org.uk (85.119.82.37): icmp_seq=1 ttl=56 time=12.0 ms
[ … ]
64 bytes from cripps.orangesquash.org.uk (85.119.82.37): icmp_seq=20 ttl=56 time=10.6 ms
64 bytes from cripps.orangesquash.org.uk (85.119.82.37): icmp_seq=21 ttl=56 time=14.8 ms
64 bytes from cripps.orangesquash.org.uk (85.119.82.37): icmp_seq=22 ttl=56 time=13.9 ms
64 bytes from cripps.orangesquash.org.uk (85.119.82.37): icmp_seq=23 ttl=56 time=14.8 ms
64 bytes from cripps.orangesquash.org.uk (85.119.82.37): icmp_seq=24 ttl=56 time=13.3 ms
64 bytes from cripps.orangesquash.org.uk (85.119.82.37): icmp_seq=25 ttl=56 time=15.4 ms
64 bytes from cripps.orangesquash.org.uk (85.119.82.37): icmp_seq=26 ttl=56 time=14.3 ms
64 bytes from cripps.orangesquash.org.uk (85.119.82.37): icmp_seq=27 ttl=56 time=12.6 ms
*** there's a gap here ****
64 bytes from cripps.orangesquash.org.uk (85.119.82.37): icmp_seq=47 ttl=56 time=25.5 ms
64 bytes from cripps.orangesquash.org.uk (85.119.82.37): icmp_seq=48 ttl=56 time=17.3 ms
64 bytes from cripps.orangesquash.org.uk (85.119.82.37): icmp_seq=49 ttl=56 time=23.9 ms

with both interfaces up

laney@nightingale> ip ro
default via 192.168.1.1 dev enx0023563c3054 proto static metric 100
default via 192.168.1.1 dev wlan0 proto static metric 400
169.254.0.0/16 dev wlan0 scope link metric 1000
192.168.1.0/24 dev enx0023563c3054 proto kernel scope link metric 100
192.168.1.0/24 dev wlan0 proto kernel scope link metric 400

My understanding was that there would be packet loss all the time without the patches applied, due to packets being sent in a round-robin fashion between two links to the same default gateway.

This said, I'm able to reproduce the apparent "packet loss", but I say apparent here because it seems to be an issue with ping (or, the kernel does something special, doesn't understand packets coming in to an interface it doesn't expect perhaps), not with the actual packets.

I've been able to observe traffic going out to my default gateway and coming back, with no interruption, using tcpdump:
sudo tcpdump -i any icmp

I don't think at this point we're working with anything different from how it was in Trusty, but just to be on the safe side I'll test that out to confirm we did get the same behavior.

Le 2015-07-03 00:09, Mathieu Trudel-Lapierre a écrit :
> My understanding was that there would be packet loss all the time
> without the patches applied, due to packets being sent in a round-robin
> fashion between two links to the same default gateway.
>
> This said, I'm able to reproduce the apparent "packet loss", but I say
> apparent here because it seems to be an issue with ping (or, the kernel
> does something special, doesn't understand packets coming in to an
> interface it doesn't expect perhaps), not with the actual packets.
>
> I've been able to observe traffic going out to my default gateway and coming back, with no interruption, using tcpdump:
> sudo tcpdump -i any icmp
>
> I don't think at this point we're working with anything different from
> how it was in Trusty, but just to be on the safe side I'll test that out
> to confirm we did get the same behavior.
>
I actually use pppoe now as a walk around the problem and my internet
connection works fines without network-manager so i remove it.

Ghislain Laframboise

I tested silo-027 on my old Thinkpad 410s running vivid. I know see the proper metrics on both the default and network routes for both eth0 and wlan0.

Tested on mako ( devel-proposed/ubuntu #250) and also verified that the proper metrics appear for default and network routes for both mobile data ( rmnet_usb0 ) and WiFi ( wlan0 ). There doesn't seem to be any adverse affects otherwise. Note, on touch, it's pretty rare for the WiFi and mobile data networks to be the same.

Tested krillin ( devel-proposed/krillin.en #137 ). It also appears to work OK. Note, the network setup on krillin is slightly different as it's rild always configures a connection with IP address and gateway the same, so only a default route is created for mobile data. When WiFi is enabled, I see two default routes with the correct metrics, and a network route for WiFi.

I still would like to do some more testing early next week, but results look promising.

After some more testing with mako, one thing I did notice that's a bit odd, is that when the phone is booted with WiFi enabled, only a single default route is created. If I disable WiFi, then re-enable, I see two default routes. This doesn't seem to have any adverse affect on things, but it's worth noting.

I tested on arale (ubuntu-touch/devel-proposed/meizu.en #58). Generally its working fine but discovered the same as Tony said in his last comment: When booting with WiFi enabled just a default-route for WiFi is added. When switching WiFi off a new default route gets added for mobile data.

This bug was fixed in the package network-manager - 0.9.10.0-4ubuntu18

---------------
network-manager (0.9.10.0-4ubuntu18) wily; urgency=medium

  * debian/tests/nm, debian/tests/wpa-dhclient: 802.11a appears to no longer
    have any supported channels for AP mode: mark all four 802.11a tests as
    expectedFailures while I dig into hostapd/mac80211_hwsim to get them to
    work again. Expected failure means we'll be notified as soon as they start
    to succeed again.

 -- Mathieu Trudel-Lapierre <email address hidden> Tue, 14 Jul 2015 18:11:58 -0400

Any idea if this will be released for Ubuntu 15.04? A month later and
latest nm version available is still 0.9.10.0-4ubuntu15.1

Luis Pabón

On 15 July 2015 at 00:37, Launchpad Bug Tracker <email address hidden>
wrote:

> This bug was fixed in the package network-manager - 0.9.10.0-4ubuntu18
>
> ---------------
> network-manager (0.9.10.0-4ubuntu18) wily; urgency=medium
>
> * debian/tests/nm, debian/tests/wpa-dhclient: 802.11a appears to no
> longer
> have any supported channels for AP mode: mark all four 802.11a tests as
> expectedFailures while I dig into hostapd/mac80211_hwsim to get them to
> work again. Expected failure means we'll be notified as soon as they
> start
> to succeed again.
>
> -- Mathieu Trudel-Lapierre <email address hidden> Tue, 14 Jul 2015
> 18:11:58 -0400
>
> ** Changed in: network-manager (Ubuntu)
> Status: In Progress => Fix Released
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1454568).
> https://bugs.launchpad.net/bugs/1436330
>
> Title:
> Network Manager doesn't set metric for local networks any more,
> causing connection issues
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/network-manager/+bug/1436330/+subscriptions
>

Three more weeks , and still no fix for the current 15.04 release?

I think you're trying and failing to do some automagic knick-knack, instead of giving the user the obvious solution to define his own metrics for every network connection.

Do i really have to just uninstall this bloody network manager and go back to /etc/network/interfaces ?

Karl,

FYI, network-manager should ignore any interfaces defined in /etc/network/interfaces.

Which is not to say this bug isn't frustrating, just that you should be able to manually configure things as you wish without removing network-manager :D

Is there a way that we can get hold of 0.9.10.0-4ubuntu18 without waiting for the official release?

Yes, this version of NM is the current one on 15.10, I was able to download and install the package without any issues from here:

https://launchpad.net/ubuntu/+source/network-manager/0.9.10.0-4ubuntu18/+build/7652985

Indeed it fixes this issue.

There's however a newer version:

https://launchpad.net/ubuntu/wily/amd64/network-manager/1.0.4-0ubuntu2

Haven't tried it. I will remain on 0.9.10.0-4ubuntu18 for the time being.

Verified fixes the issue! Thanks Luis. I needed two packages from that URL https://launchpad.net/ubuntu/+source/network-manager/0.9.10.0-4ubuntu18/+build/7652985

- libnm-util2 0.9.10.0-4ubuntu18
- network-manager 0.9.10.0-4ubuntu18

Hello Jason, or anyone else affected,

Accepted network-manager into vivid-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/network-manager/0.9.10.0-4ubuntu15.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Fix seems solid.

`route -n` with up-to-date Vivid:

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.17.76.1 0.0.0.0 UG 1024 0 0 eth0
10.17.75.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan0
10.17.76.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 eth0

`route -n` after enabling proposed, updating packages, and rebooting:

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.17.76.1 0.0.0.0 UG 100 0 0 eth0
10.17.75.0 0.0.0.0 255.255.255.0 U 400 0 0 wlan0
10.17.76.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 eth0

Confirm fix works.

$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.254 0.0.0.0 UG 1024 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 wlan1
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan1
$ sudo dpkg -i network-manager_0.9.10.0-4ubuntu18_amd64.deb libnm-util2_0.9.10.0-4ubuntu18_amd64.deb
...
$ sudo systemctl restart NetworkManager
$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.254 0.0.0.0 UG 100 0 0 eth1
0.0.0.0 192.168.0.254 0.0.0.0 UG 400 0 0 wlan1
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 wlan1
192.168.0.0 0.0.0.0 255.255.255.0 U 100 0 0 eth1
192.168.0.0 0.0.0.0 255.255.255.0 U 400 0 0 wlan1

Routes are now correct, and my ssh sessions to other hosts aren't timing out all the time anymore.

This bug was fixed in the package network-manager - 0.9.10.0-4ubuntu15.3

---------------
network-manager (0.9.10.0-4ubuntu15.3) vivid; urgency=medium

  * debian/patches/git_route_fixes_part1_207ab013.patch,
    debian/patches/git_route_fixes_part2_529591d8.patch,
    debian/patches/git_avoid_conflict_reinstall_dev_route_e439478c.patch:
    really fix routing: both the default gateways and the network routes get
    to be installed with a device-type-based priority so that we can get
    multiple routes installed without conficting. (LP: #1436330)

network-manager (0.9.10.0-4ubuntu15.2) vivid; urgency=medium

  * debian/patches/fix_default_routes.patch: rework routing priorities and how
    default routes are added for devices and VPNs so that we can successfully
    run multiple devices on the same subnet without packet loss. (LP: #1436330)

 -- Mathieu Trudel-Lapierre <email address hidden> Tue, 15 Sep 2015 10:10:50 -0400

The verification of the Stable Release Update for network-manager has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

I added a comment in bug 1446689 which should have been better here
So copying it here:
---------------------------

I upgraded vivid to wily yesterday and my networking is broken
Not sure but it looked like this bug?

Machine with vivid upgraded to wily: network not working

$ ip -d route
unicast 117.195.32.1 dev ppp0 proto kernel scope link src 117.195.47.122
unicast 169.254.0.0/16 dev ppp0 proto boot scope link metric 1000 [/code]

By contrast 14.10 booted from usb and pppoe by hand -- network working

$ ip -d route
unicast default dev ppp0 proto boot scope link
unicast 117.195.32.1 dev ppp0 proto kernel scope link src 117.195.40.157
unicast 192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.2 metric 1

Some evident issues

- The absence of default is an obvious problem
- The large metric is another problem
- eth0 has become ppp0

As suggested earlier in this thread I downloaded pppoe package and installed it by hand (since apt is down with not networking)
But its not helped

Please stop spamming, admins please do something, this is a bug reporting system, not a spam collector, it's not the first time that spam comments came from this user.