[DPDK] Instances do not have network connectivity if bond is configured for private network

According to instance cmdline from http://paste.openstack.org/show/493008/ instance was launched with non-huge-backed memory, therefore vhostuser doesn't work and OSTF was failed.

QA team should either implement OSTF for DPDK, or adjust flavor before OSTF with `nova flavor-key ${FLAVOR} set hw:mem_page_size=2048`.

Ok, I'm going to check how Nova schedules instances with common flavors if cloud has both types of computes with enabled DPDK and default. If such instances (without huge pages) are booted on computes with DPDK (won't be accessible via network), then scheduling mechanism must be improved, not tests. Also since DPDK works with lots of modern NICs, maybe should we force users to enable or disable DPDK on *all* computes? Then we will be able to modify default flavors, so they use huge pages at deployment stage.

> scheduling mechanism must be improved

Scheduling mechanism could be improved by:

1. Creating host aggregates and using attributes
2. Forbid using non-huge instances some way in nova scheduler filters

> since DPDK works with lots of modern NICs, maybe should we force users to enable or disable DPDK on *all* computes
Even with this, DPDK now does not support MTU > 1500 and security groups, so we couldn't do it.

According to HugePage spec http://specs.openstack.org/openstack/nova-specs/specs/juno/approved/virt-driver-large-pages.html,

"The scheduler will be enhanced to take account of the page size setting on the flavor and pick hosts which have sufficient large pages available when scheduling the instance. Conversely if large pages are not requested, then the scheduler needs to avoid placing the instance on a host which has pre-reserved large pages. The enhancements for the scheduler will be done as part of the new filter that is implemented as part of the NUMA topology blueprint. This involves altering the logic done in that blueprint, so that instead of just looking at free memory in each NUMA node, it instead looks at the free page count for the desired page size."

So mechanism of #2 should be implemented with new filters (AggregateInstanceExtraSpecsFilter,NUMATopologyFilter). This filters are added to nova.conf when we enable HP

I tested case when there both types of computes (w/ and w/o DPDK) in cluster and instance connectivity test still fails, because it's spawned on compute node with enabled DPDK but default flavor 'm1.micro':

AssertionError: Failed 1 OSTF tests; should fail 0 tests. Names of failed tests:
  - Check network connectivity from instance via floating IP (failure) Instance is not reachable by IP. Please refer to OpenStack logs for more details.

>So mechanism of #2 should be implemented with new filters (AggregateInstanceExtraSpecsFilter,NUMATopologyFilter). This filters are added to nova.conf when we enable HP

Using Nova availability zones or some filtering mechanism to prevent incorrect instance scheduling (with default flavors) sounds like a solution for me.

Created separate bugs for OSTF: bug #1567439 and bug #1567447

You could create instance with aggregates as it's done in CPU pinning tests:

 https://github.com/openstack/fuel-qa/blob/master/fuelweb_test/tests/test_cpu_pinning.py#L79-L108

There is no way we could make a aggregates and extra specs for them and flavors in order to use AggregateInstanceExtraSpecsFilter. It is not a fuel job right now.

In original scenario you've posted there was only one compute, so you need only flavor modification for hugepages.
In case if you want to make mixed setup (w/DPDK and without) in system test, now you could set up aggregates and extra specs by system test code using nova API.

In 10.0 we're try to address this UX.

Vladimir,

this issue is not about our tests (in the tests we already create falvors use availability zones to schedule instances), it's only about bad UX - after deployment user gets instances w/o network connectivity due to broken scheduling. If we don't plan to fix it in 9.0, please mark in as "Won't fix" for mitaka.

Also please don't assign it to QA team if you agree that VMs scheduling could be improved in the product, we have separate bugs related to tests: bug #1567439 and bug #1567447

We should probably document this for 9.0.

Please provide a detailed description of what needs to be documented.

It is failed on swarm https://product-ci.infra.mirantis.net/job/9.0.system_test.ubuntu.support_dpdk_bond/32/console