IPv6 Tempest tests fail with DVR

http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiTWlzbWF0Y2hFcnJvclwiIGFuZCBtZXNzYWdlOlwidGVzdF9kaGNwNl9zdGF0ZWxlc3NfZnJvbV9vc1wiIEFORCBidWlsZF9uYW1lOlwiY2hlY2stdGVtcGVzdC1kc3ZtLW5ldXRyb24tZHZyXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0MTgzMzAyMzAyMDgsIm1vZGUiOiIiLCJhbmFseXplX2ZpZWxkIjoiIn0=

This is related to bug for #1376325

I can see three patches:

https://review.openstack.org/#/c/138654/
https://review.openstack.org/#/c/136947/
https://review.openstack.org/#/c/134676/

Do we need them all? Can we check which one(s) will help address this issue?

I cherry picked all three of these patches and still saw an issue where RAs were not being recieved by guests, and IPv6 networking was not set up.

Updated the patch https://review.openstack.org/#/c/134676/ to resolve a merge conflict but test failure still seen.

From the logs, the radvd appears to be getting spawned, so it points to the VM to radvd communication path or the configuration provided to radvd

Stderr: '' execute /opt/stack/new/neutron/neutron/agent/linux/utils.py:79
2014-12-12 00:04:06.539 25957 DEBUG neutron.agent.linux.ra [-] Enable IPv6 RA for router 54e84edc-8cd9-4605-bb49-4bb0baf4a84b enable_ipv6_ra /opt/stack/new/neutron/neutron/agent/linux/ra.py:109
2014-12-12 00:04:06.541 25957 DEBUG neutron.agent.linux.utils [-] Unable to access /opt/stack/data/neutron/external/pids/54e84edc-8cd9-4605-bb49-4bb0baf4a84b.pid.radvd get_value_from_conf_file /opt/stack/new/neutron/neutron/agent/linux/utils.py:164
2014-12-12 00:04:06.542 25957 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-54e84edc-8cd9-4605-bb49-4bb0baf4a84b', 'radvd', '-C', '/opt/stack/data/neutron/ra/54e84edc-8cd9-4605-bb49-4bb0baf4a84b.radvd.conf', '-p', '/opt/stack/data/neutron/external/pids/54e84edc-8cd9-4605-bb49-4bb0baf4a84b.pid.radvd'] create_process /opt/stack/new/neutron/neutron/agent/linux/utils.py:46
2014-12-12 00:04:06.888 25957 DEBUG neutron.agent.linux.utils [-]
Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-54e84edc-8cd9-4605-bb49-4bb0baf4a84b', 'radvd', '-C', '/opt/stack/data/neutron/ra/54e84edc-8cd9-4605-bb49-4bb0baf4a84b.radvd.conf', '-p', '/opt/stack/data/neutron/external/pids/54e84edc-8cd9-4605-bb49-4bb0baf4a84b.pid.radvd']
Exit code: 0
Stdout: ''
Stderr: '' execute /opt/stack/new/neutron/neutron/agent/linux/utils.py:79
2014-12-12 00:04:06.889 25957 DEBUG neutron.agent.linux.ra [-] radvd enabled for router 54e84edc-8cd9-4605-bb49-4bb0baf4a84b _spawn_radvd /opt/stack/new/neutron/neutron/agent/linux/ra.py:96

I would have thought just the last two patches (rule index and rule version) would have fixed the problem, at least my devstack runs with with that in dvr/v4/v6 mode. I'll take a look at the third and update my patch to trunk today.

This one might be necessary as well:

https://review.openstack.org/#/c/141297/

We're getting DVR/IPv6 working one bug at a time :(

It appears the problem is in communication from radvd to the VM. At the moment this patch https://review.openstack.org/#/c/141297/ looks promising for the fix ( Brian has also referred to the same). It is passing the new IPV6 tests that were added recenlty and causing the failure reported here.

The patch has a Jenkins failure currently, I have contacted the co-author.

Yep, the security group fix is probably the patch I needed to cherry pick to make things work. Good detective work all

Fixed by:

https://review.openstack.org/#/c/141297/