Comment 23 for bug 1163147

1. Note that my instructions in comment #12 are partly superseded by my comments in #16. I think you understood that. To make it clear for other readers I will repeat the instructions below.

2. In comment #16 I wrote

> Also you will have to do
> echo "nameserver 127.0.3.1" | resolvconf -a lo.dnsmasq-libvirt
> and
> resolvconf -d lo.dnsmasq-libvirt

without saying *when* these commands need to be run. You understood what I meant, but to make it clear for others: one has to do

    echo "nameserver 127.0.3.1" | resolvconf -a lo.dnsmasq-libvirt

after dnsmasq-libvirt has started and

    resolvconf -d lo.dnsmasq-libvirt

before stopping it.

3. Your etc-init-lxc-net.conf starts dnsmasq with incorrect options. You need to be careful with how "bind-interfaces", "interface" and "listen-address" options are used. You do the following

    --bind-interfaces --listen-address=10.0.3.1 --interface=lxcbr0 --interface=127.0.3.1

3.1. You presumably meant the following.

    --bind-interfaces --listen-address=10.0.3.1 --interface=lxcbr0 --listen-address=127.0.3.1

The argument of "--interface=" must be the name of an interface, not an IP address; the "--interface=127.0.3.1" will cause dnsmasq to fail to start. On my machine, dnsmasq 2.65-1ubuntu1 with "--interface=127.0.3.1" prints "unknown interface 127.0.3.1".

3.2. In bind-interfaces mode dnsmasq binds all and only the addresses on interfaces that it is configured to listen on. By default it listens on all interfaces that are present when it starts, but when you give an "interface" option, dnsmasq listens only that interface plus interface lo. You can add listen addresses by giving additional "interface" and "listen-address" options. So if you give "--listen-address=10.0.3.1 --interface=lxcbr0 --listen-address=127.0.3.1" then dnsmasq will listen on 10.0.3.1, 127.0.3.1, 127.0.0.1 plus the address of lxcbr0.

What you want is for it to listen on the virtual-machine-facing address(es) and on 127.0.3.1 but not on 127.0.0.1. Assuming that the virtual-machine-facing addresses are 10.0.3.1 and the address of lxcbr0, the options should be:

     --bind-interfaces --listen-address=10.0.3.1 --interface=lxcbr0 --except-interface=lo --listen-address=127.0.3.1

3.3. (We would be better off using "bind-dynamic" than "bind-interfaces" so that dnsmasq-libvirt continues to listen on the correct addresses when interfaces go up and down and have their addresses changed. However, my testing seems to indicate that you can't use "bind-dynamic" along with "listen-address". That's a bug, I think, which I will report, but until it's fixed we can't use "bind-interfaces" along with "listen-address", alas.)

4. Now create the script /etc/resolvconf/update.d/dnsmasq-libvirt (not .../dnsmasq which will conflict with the script installed by the dnsmasq package). The script you posted (etc-resolvconf-update.d-resolvconf) is good for that except that one line has to be changed. The line

    RSLVCNFFILES="$(/lib/resolvconf/list-records | sed -e '/^lo.dnsmasq$/d')"

must be

    RSLVCNFFILES="$(/lib/resolvconf/list-records | sed -e '/^lo.dnsmasq-libvirt$/d')"

to reflect the fact that we are going to use the record name "dnsmasq-libvirt". To be ready for resolvconf 1.74 it should be

    RSLVCNFFILES="$(/lib/resolvconf/list-records --after lo.dnsmasq-libvirt | sed -e '/^lo.dnsmasq-libvirt$/d')"

so please use this.

5. Currently the Upstart job file starts dnsmasq in the pre-start script. You should actually use an "exec" clause to start dnsmasq so that Upstart can monitor it. This means that you need to create a separate dnsmasq-libvirt job file with triggers that look something like the following.

    start on starting lxc-net
    stop on stopping lxc-net

The Upstart Cookbook

    http://upstart.ubuntu.com/cookbook

recommends that the daemon be "exec"ed in foreground mode. So instead of running dnsmasq in a pre-start script, add an "exec" stanza.

    exec dnsmasq --keep-in-foreground ...other...options...

Then put the following

    echo "nameserver 127.0.3.1" | resolvconf -a lo.dnsmasq-libvirt

in a post-start script stanza and

    resolvconf -d lo.dnsmasq-libvirt

in a pre-stop script stanza.

Warning: I haven't tested this.

6. After all this is done, when lxc is started it should start dnsmasq-libvirt which registers one of its listen addresses 127.0.3.1 with resolvconf which calls its update scripts which writes the addresses of external nameservers to /var/run/dnsmasq-libvirt/resolv.conf and writes 127.0.3.1 to /etc/resolv.conf so that resolving on the host goes through dnsmasq-libvirt and dnsmasq-libvirt forwards queries that it can't answer itself to the external nameservers.

7. To simplify things we could omit 127.0.3.1 and just register one of the virtual-machine-facing addresses with resolvconf. Edit the Upstart job file to start dnsmasq with

     --bind-interfaces --listen-address=10.0.3.1 --interface=lxcbr0 --except-interface=lo

and in the post-start do

    echo "nameserver 10.0.3.1" | resolvconf -a lo.dnsmasq-libvirt

Let me know what works.