virtual machines should not have nat on servers

Ian, thanks for the bug report.

I happen to be subscribed to the ubuntu-virt package for mostly historical reasons, though I'm minimally involved in it today. I have, however, subscribed Scott Moser and Serge Hallyn, who are now more or less the maintainers of this functionality. I'll leave it to them to decide how to address this.

Thanks!
Dustin

@Ian, If i understand the bug report correctly, this would seem to be a generic bug regarding the default networking option setup as part of libvirt. If this is the case, I agree it's not a great ideal for servers - but also not an unreasonable default for the platform as a whole use case.

The default network provided for kvm, for example is:
$ cat /etc/libvirt/qemu/networks/default.xml
<network>
  <name>default</name>
  <bridge name="virbr%d" />
  <forward/>
  <ip address="192.168.122.1" netmask="255.255.255.0">
    <dhcp>
      <range start="192.168.122.2" end="192.168.122.254" />
    </dhcp>
  </ip>
</network>

I haven't checked for other virtulisation methods, but I imagine it is similar. It is easy enough to provide a chosen networking model, so I'm not fully convinced this is a valid bug.

Please can you confirm I have understood the bug report correctly.

Thanks.

Dave Walker writes ("[Bug 865515] Re: virtual machines should not have nat on servers"):
> @Ian, If i understand the bug report correctly, this would seem to be a
> generic bug regarding the default networking option setup as part of
> libvirt. If this is the case, I agree it's not a great ideal for
> servers - but also not an unreasonable default for the platform as a
> whole use case.

I agree with this. But the default should be different for servers
and desktops/laptops.

> I haven't checked for other virtulisation methods, but I imagine it is
> similar. It is easy enough to provide a chosen networking model, so I'm
> not fully convinced this is a valid bug.

The existence of virbr0 convinces the Xen startup script, in the
default configuration, not to do its own networking arrangements.
Instead, since there is already a bridge, it causes all the VMs to use
it, on the principle that if there is a bridge it should probably be
used.

Ian.

I agree the default of nat is appropriate for desktops and I feel quite strongly it should not change there. The default for servers is probably better as non-nat (though this could be debated), as this bug suggests. However having different behavior depending on the type of system it is would likely cause confusion. Perhaps it would be appropriate to add a preseedable debconf question to allow proper setup of the bridge for servers. This could be of priority 'medium' so desktops don't get prompted, but server environments (which would arguably use preseeding more often) can set debconf as part of installation. Admittedly, this is not perfect, but might help.

My 2 cents

Status changed to 'Confirmed' because the bug affects multiple users.

Agreed the default isn't ideal for servers, but I'm not convinced there is a good wya to set up a works-everywhere ideal solution for servers. I would assume that any site which cares enough about the network performance to object to a NATed bridge would have serveral NICs, some perhaps bonded - and leaving configuration to the admin is best.

I could be wrong though. How would you suggest configuring out of the box on install for a server?

(The above question is seeking advice from anyone willing to comment - thanks)

Serge Hallyn writes ("[Bug 865515] Re: virtual machines should not have nat on servers"):
> Agreed the default isn't ideal for servers, but I'm not convinced there
> is a good wya to set up a works-everywhere ideal solution for servers.
> I would assume that any site which cares enough about the network
> performance to object to a NATed bridge would have serveral NICs, some
> perhaps bonded - and leaving configuration to the admin is best.

The problem with NAT is not performance, it is the unwanted address
translation. In a server setup, you want a simple bridge. The
individual VMs will be allocated their own real addresses (either by
the local dhcp server, or statically) just as you would with a
physical server.

Ian.

How would you suggest configuring out of the box on install for a server?

I don't believe it is reasonable to assume we can bridge eth0 and have that be the right thing to do. Perhaps we could find the nic for the default route, but that could be fragile especially in esoteric setups. Also, adding a NATed bridge is safe to do at any time, but taking the primary nic and turning it into a bridge during upgrade is not safe.

Is there an alternative?

Serge Hallyn writes ("[Bug 865515] Re: virtual machines should not have nat on servers"):
> How would you suggest configuring out of the box on install for a
> server?

eth0 should be bridged. Specifically, for a vm host, the primary
network interface (by which I mean the one used for the original
installation, and set up automatically by the installer) should be
enslaved to a new bridge.

> I don't believe it is reasonable to assume we can bridge eth0 and have
> that be the right thing to do.

On a server VM host system with one in-use network interface, it is
exactly the right thing to do.

> Perhaps we could find the nic for the default route, but that could
> be fragile especially in esoteric setups.

If there are multiple in-service network interfaces then obviously
we don't know what to do. In that case the software shouldn't do
anything automatically.

But that's OK because if there are multiple in-service network
interfaces on a server, the admin has _already_ done some manual
setup. So they will know how they want to organise the VM networking
and it is fine to expect them to tell the VM system what to do.

Specifically, in that case it is fine for the VM setup to ask them
"please tell me which bridge to use".

> Also, adding a NATed bridge is safe to do at any time, but taking the
> primary nic and turning it into a bridge during upgrade is not safe.

The question of what to do during upgrade, and what to do during
install, are somewhat different.

> Is there an alternative?
>
> ** Changed in: libvirt (Ubuntu)
> Status: Confirmed => Incomplete

Can you please stop changing the status of this bug ?

Ian.

> eth0 should be bridged

If anything I'd prefer to go the debian route and, for servers only, not set up any network at all. I don't care to support "if desktop, NAT. If server, if there is only eth0; bridge; otherwise, do nothing."

I'm not against shipping a hand-run script which runs netcf to turn eth0 into a bridge.

> Can you please stop changing the status of this bug ?

You hadn't answerd the question. It was incomplete.

From my experience I think there was a time when libvirt would only allow the use of networks defined with libvirt. Nowadays (at least starting with Trusty/12.04) it is picking up and allowing the use of network interfaces outside the control of libvirt. So I am not sure this is still in need.

Not sure whether the creation of the NAT bridge is challenged as something that should not be done on server. It can be "not used". In addition there appear to be choices for macvtap on host network devices or when the host defines bridges (/etc/network/interfaces style) those can be used. Also not really sure about a script. Maybe an example of how to define a bridge manually. But mostly it seems like admins are rather expected to want to do it on their own.