Testing a multi-node HA setup (3 controllers 1 compute) I'm hitting this, tryting to re-run the deploy with no changes:
" (at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in `deprecation')\u001b[0m",
"+ '[' -z '' ']'",
"+ archivedirs=(\"/etc\" \"/root\" \"/var/lib/ironic/tftpboot\" \"/var/lib/ironic/httpboot\" \"/var/www\")",
"+ rsync_srcs=",
"+ for d in '\"${archivedirs[@]}\"'",
"+ '[' -d /etc ']'",
"+ rsync_srcs+=' /etc'",
"+ for d in '\"${archivedirs[@]}\"'",
"+ '[' -d /root ']'",
"+ rsync_srcs+=' /root'",
"+ for d in '\"${archivedirs[@]}\"'",
"+ '[' -d /var/lib/ironic/tftpboot ']'",
"+ for d in '\"${archivedirs[@]}\"'",
"+ '[' -d /var/lib/ironic/httpboot ']'",
"+ for d in '\"${archivedirs[@]}\"'",
"+ '[' -d /var/www ']'",
"+ rsync -a -R --delay-updates --delete-after /etc /root /var/lib/config-data/haproxy",
"rsync: rename failed for \"/var/lib/config-data/haproxy/etc/hostname\" (from etc/.~tmp~/hostname): Device or resource busy (16)",
"rsync: rename failed for \"/var/lib/config-data/haproxy/etc/resolv.conf\" (from etc/.~tmp~/resolv.conf): Device or resource busy (16)",
"rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]",
"",
It seems we've got some files held open on the bind to haproxy, so we either need to solve that or rework how we do the rsync (e.g create a unique directory per docker-puppet run and switch over to the latest when restarting the container.
One potential difference between haproxy service and other HA services is the fact that we bind mount directly /var/lib/
I'm going to propose a change to see if it fixes the issue