Fuel for OpenStack

VMs in 192.168.122.0/24 subnet have broken networking

Bug #1437410 reported by Erik Swanson
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
Medium
Denis Egorenko
Fuel for OpenStack 6.1
6.0.x
Won't Fix
Medium
MOS Maintenance
Fuel for OpenStack 6.0-updates
6.1.x
Won't Fix
Medium
MOS Maintenance
Fuel for OpenStack 6.1-updates
7.0.x
Fix Released
Medium
Denis Egorenko
Fuel for OpenStack 7.0

Bug Description

If I create a 192.168.122.0/24 subnet, VMs in that subnet are unable to reach the external world.

Symptoms:

As seen while pinging an external IP, there is an unwanted/incorrect SNAT applied to the packets before they leave the compute node. (When observing the private network with tcpdump, the pings will have a source IP that is *not* the VM.)
This causes the packets to not match the right (or any) SNAT rule inside the qrouter on the controller.

Other subnets with identical router and security group configurations work fine.

Suspected cause:

Failure of Fuel to remove a presumably-unneeded default libvirt network. (Observation/suggestion by "Sam-I-Am" in #openstack, who was helping me diagnose this.)

Revision history for this message
Erik Swanson (erik-swanson) wrote :

Cluster details: Fuel 6.0, HA, on Ubuntu. Networking: Neutron with VLANs.

Nastya Urlapova (aurlapova)
Changed in fuel:
milestone: none → 6.1
assignee: nobody → Fuel Library Team (fuel-library)
Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

Erik, can you choose other network than this one? This is a very specific issue that can be easily worked around.

Changed in fuel:
status: New → Confirmed
importance: Undecided → Medium
Vladimir Kuklin (vkuklin)
Changed in fuel:
status: Confirmed → Won't Fix
Nastya Urlapova (aurlapova)
tags: added: qa-agree-7.0 release-notes
Revision history for this message
Dmitry Ilyin (idv1985) wrote :

do you have these values in sysctl?
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1

Try to set them to 0

Revision history for this message
Dmitry Ilyin (idv1985) wrote :

On your host node

Revision history for this message
Erik Swanson (erik-swanson) wrote :

My cluster has 16 compute nodes and 3 controllers.

Is this change you want me to try safe for a production environment?
If so, is there a way to have the change rolled out automatically to the appropriate nodes?

Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

The issue here is pretty simple - there is a default network created by libvirt. Workaround is also pretty simple - if you want to use this network, just undefine default libvirt network on all compute nodes.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/209508

Changed in fuel:
status: Won't Fix → In Progress
OpenStack Infra (hudson-openstack)
Changed in fuel:
assignee: Alexey Deryugin (velovec) → Denis Egorenko (degorenko)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/209508
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=4e5e4cfe3a7be1d0b4c9830865bb7b6eeee9f0a3
Submitter: Jenkins
Branch: master

commit 4e5e4cfe3a7be1d0b4c9830865bb7b6eeee9f0a3
Author: Alexey Deryugin <email address hidden>
Date: Wed Aug 5 16:04:48 2015 +0300

    Delete default libvirt network definition

    Remove default libvirt network due to possible using
    the same network as VMs subnet in production cluster.

    Co-Authored-By: Denis Egorenko <email address hidden>

    Change-Id: Ic8720b37e924c4fbbe243f7a5ea9a94113d054fd
    Closes-Bug: #1437410

Changed in fuel:
status: In Progress → Fix Committed
Alexander Adamov (aadamov)
tags: added: release-notes-done
removed: release-notes
Olena Logvinova (ologvinova)
tags: added: rn7.0
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-docs (master)

Reviewed: https://review.openstack.org/222658
Committed: https://git.openstack.org/cgit/stackforge/fuel-docs/commit/?id=18e039a858e9e7d8846f55dc0bf5ff193d1d8ac2
Submitter: Jenkins
Branch: master

commit 18e039a858e9e7d8846f55dc0bf5ff193d1d8ac2
Author: Alexander Adamov <email address hidden>
Date: Fri Sep 11 18:02:12 2015 +0300

    [RN 7.0]Fuel install&deploy issues

    Adds resolved and known issues:
    LP1491725, LP1437410,
    LP1477903

    Change-Id: I87fcb333d632de6faa7071713f88e8519bccf8d7
    Related-Bug: #1491725
    Related-Bug: #1437410
    Related-Bug: #1477903

Dmitriy Kruglov (dkruglov)
tags: added: on-verification
Revision history for this message
Dmitriy Kruglov (dkruglov) wrote :

Verified on MOS 7.0, custom ISO. The issue is not reproduced.

ISO info:
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "7.0"
  openstack_version: "2015.1.0-7.0"
  api: "1.0"
  build_number: "1260"
  build_id: "2015-10-09_12-02-12"
  nailgun_sha: "edbae54d510edbaa1d379e9523febe5a0e5acd41"
  python-fuelclient_sha: "486bde57cda1badb68f915f66c61b544108606f3"
  fuel-agent_sha: "50e90af6e3d560e9085ff71d2950cfbcca91af67"
  fuel-nailgun-agent_sha: "d7027952870a35db8dc52f185bb1158cdd3d1ebd"
  astute_sha: "6c5b73f93e24cc781c809db9159927655ced5012"
  fuel-library_sha: "713698e88c6e1e4ed9ebad759a21266890898d57"
  fuel-ostf_sha: "2cd967dccd66cfc3a0abd6af9f31e5b4d150a11c"
  fuelmain_sha: "a65d453215edb0284a2e4761be7a156bb5627677"

tags: removed: on-verification
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.