unbound-client.yml failure depending on TERMCAP envvar content

Bug #1803382 reported by Francois Deppierraz
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
openstack-ansible
Confirmed
High
Francois Deppierraz

Bug Description

It looks like ansible is trying to evaluate the content of environment variables (such as TERMCAP) as jinja template. This leads to failure when an environment variable contains special characters such as "|".

This issue happens in practice when openstack-ansible is run from inside a GNU Screen session.

TASK [Apply resolver sorting] *************************************************************************************************************************************************************************************
fatal: [infra1_keystone_container-44f47275]: FAILED! => {"msg": "The conditional check 'physical_host in hostvars['localhost']['unbound_physical_hosts']' failed. The error was: template error while templating string: unexpected '|'. String: SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual terminal:\\\n\t:DO=\\E[%dB:LE=\\E[%dD:RI=\\E[%dC:UP=\\E[%dA:bs:bt=\\E[Z:\\\n\t:cd=\\E[J:ce=\\E[K:cl=\\E[H\\E[J:cm=\\E[%i%d;%dH:ct=\\E[3g:\\\n\t:do=^J:nd=\\E[C:pt:rc=\\E8:rs=\\Ec:sc=\\E7:st=\\EH:up=\\EM:\\\n\t:le=^H:bl=^G:cr=^M:it#8:ho=\\E[H:nw=\\EE:ta=^I:is=\\E)0:\\\n\t:li#56:co#211:am:xn:xv:LP:sr=\\EM:al=\\E[L:AL=\\E[%dL:\\\n\t:cs=\\E[%i%d;%dr:dl=\\E[M:DL=\\E[%dM:dc=\\E[P:DC=\\E[%dP:\\\n\t:im=\\E[4h:ei=\\E[4l:mi:IC=\\E[%d@:ks=\\E[?1h\\E=:\\\n\t:ke=\\E[?1l\\E>:vi=\\E[?25l:ve=\\E[34h\\E[?25h:vs=\\E[34l:\\\n\t:ti=\\E[?1049h:te=\\E[?1049l:us=\\E[4m:ue=\\E[24m:so=\\E[3m:\\\n\t:se=\\E[23m:mb=\\E[5m:md=\\E[1m:mh=\\E[2m:mr=\\E[7m:\\\n\t:me=\\E[m:ms:\\\n\t:Co#8:pa#64:AF=\\E[3%dm:AB=\\E[4%dm:op=\\E[39;49m:AX:\\\n\t:vb=\\Eg:G0:as=\\E(0:ae=\\E(B:\\\n\t:ac=\\140\\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:\\\n\t:po=\\E[5i:pf=\\E[4i:Km=\\E[M:k0=\\E[10~:k1=\\EOP:k2=\\EOQ:\\\n\t:k3=\\EOR:k4=\\EOS:k5=\\E[15~:k6=\\E[17~:k7=\\E[18~:\\\n\t:k8=\\E[19~:k9=\\E[20~:k;=\\E[21~:F1=\\E[23~:F2=\\E[24~:\\\n\t:F3=\\E[1;2P:F4=\\E[1;2Q:F5=\\E[1;2R:F6=\\E[1;2S:\\\n\t:F7=\\E[15;2~:F8=\\E[17;2~:F9=\\E[18;2~:FA=\\E[19;2~:kb=:\\\n\t:K2=\\EOE:kB=\\E[Z:kF=\\E[1;2B:kR=\\E[1;2A:*4=\\E[3;2~:\\\n\t:*7=\\E[1;2F:#2=\\E[1;2H:#3=\\E[2;2~:#4=\\E[1;2D:%c=\\E[6;2~:\\\n\t:%e=\\E[5;2~:%i=\\E[1;2C:kh=\\E[1~:@1=\\E[1~:kH=\\E[4~:\\\n\t:@7=\\E[4~:kN=\\E[6~:kP=\\E[5~:kI=\\E[2~:kD=\\E[3~:ku=\\EOA:\\\n\t:kd=\\EOB:kr=\\EOC:kl=\\EOD:km:\n\nThe error appears to have been in '/opt/openstack-ansible/playbooks/common-tasks/unbound-clients.yml': line 18, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n#on the same physical_host\n- name: Apply resolver sorting\n ^ here\n"}

PLAY RECAP ********************************************************************************************************************************************************************************************************
infra1_keystone_container-44f47275 : ok=6 changed=0 unreachable=0 failed=1
localhost : ok=3 changed=0 unreachable=0 failed=0

Revision history for this message
Mohammed Naser (mnaser) wrote :

Suggested workaround by Francois:

"an easy workaround is simply deleting this variable from /etc/openstack_deploy/ansible_facts/localhost"

Changed in openstack-ansible:
status: New → Confirmed
importance: Undecided → High
Revision history for this message
Francois Deppierraz (francois-ctrlaltdel) wrote :

In https://github.com/openstack/openstack-ansible/blob/1ab4e908f2cfd537171deff5dc3e237c3cf37dd7/inventory/host_vars/localhost/unbound.yml#L17, the unbound_physical_hosts is defined this way:

unbound_physical_hosts: >-
  {{ groups[resolvconf_resolver_group] |
    map('extract', hostvars, 'physical_host') |
    select('defined') |
    list
  }}

Because ansible seems to do lazy evaluation of jinja2 in variables, this one actually contains all the data from groups[resolvconf_resolver_group], including environment variables.

Not sure yet if there's an easy way to set unbound_physical_hosts without running into this issue.

Changed in openstack-ansible:
assignee: nobody → Francois Deppierraz (francois-ctrlaltdel)
Revision history for this message
Francois Deppierraz (francois-ctrlaltdel) wrote :
Download full text (4.3 KiB)

Oops, please disregard the previous comment.

Here's how this issue can be reproduced on an AIO deployment running on Ubuntu 16.04.

root@francois-osa-aio:/opt/openstack-ansible# git branch
* (HEAD detached at 18.1.0)
  master
  stable/queens
root@francois-osa-aio:/opt/openstack-ansible# openstack-ansible --version
Variable files: "-e @/etc/openstack_deploy/user_secrets.yml -e @/etc/openstack_deploy/user_variables.yml "
ansible-playbook 2.5.10
  config file = None
  configured module search path = [u'/etc/ansible/roles/plugins/library', u'/etc/ansible/roles/ceph-ansible/library']
  ansible python module location = /opt/ansible-runtime/local/lib/python2.7/site-packages/ansible
  executable location = /opt/ansible-runtime/bin/ansible-playbook
  python version = 2.7.12 (default, Nov 12 2018, 14:36:49) [GCC 5.4.0 20160609]

EXIT NOTICE [Playbook execution success] **************************************
===============================================================================
root@francois-osa-aio:/opt/openstack-ansible# cat /tmp/test.yaml
---
- hosts: localhost
  tasks:
    - debug:
        msg: "{{ hostvars['localhost'] }}"
root@francois-osa-aio:/opt/openstack-ansible# openstack-ansible /tmp/test.yaml
Variable files: "-e @/etc/openstack_deploy/user_secrets.yml -e @/etc/openstack_deploy/user_variables.yml "
 [WARNING]: Unable to parse /etc/openstack_deploy/inventory.ini as an inventory source

PLAY [localhost] ****************************************************************************************

TASK [debug] ********************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "template error while templating string: unexpected '|'. String: SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual terminal:\\\n\t:DO=\\E[%dB:LE=\\E[%dD:RI=\\E[%dC:UP=\\E[%dA:bs:bt=\\E[Z:\\\n\t:cd=\\E[J:ce=\\E[K:cl=\\E[H\\E[J:cm=\\E[%i%d;%dH:ct=\\E[3g:\\\n\t:do=^J:nd=\\E[C:pt:rc=\\E8:rs=\\Ec:sc=\\E7:st=\\EH:up=\\EM:\\\n\t:le=^H:bl=^G:cr=^M:it#8:ho=\\E[H:nw=\\EE:ta=^I:is=\\E)0:\\\n\t:li#54:co#211:am:xn:xv:LP:sr=\\EM:al=\\E[L:AL=\\E[%dL:\\\n\t:cs=\\E[%i%d;%dr:dl=\\E[M:DL=\\E[%dM:dc=\\E[P:DC=\\E[%dP:\\\n\t:im=\\E[4h:ei=\\E[4l:mi:IC=\\E[%d@:ks=\\E[?1h\\E=:\\\n\t:ke=\\E[?1l\\E>:vi=\\E[?25l:ve=\\E[34h\\E[?25h:vs=\\E[34l:\\\n\t:ti=\\E[?1049h:te=\\E[?1049l:us=\\E[4m:ue=\\E[24m:so=\\E[3m:\\\n\t:se=\\E[23m:mb=\\E[5m:md=\\E[1m:mh=\\E[2m:mr=\\E[7m:\\\n\t:me=\\E[m:ms:\\\n\t:Co#8:pa#64:AF=\\E[3%dm:AB=\\E[4%dm:op=\\E[39;49m:AX:\\\n\t:vb=\\Eg:G0:as=\\E(0:ae=\\E(B:\\\n\t:ac=\\140\\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:\\\n\t:po=\\E[5i:pf=\\E[4i:Km=\\E[M:k0=\\E[10~:k1=\\EOP:k2=\\EOQ:\\\n\t:k3=\\EOR:k4=\\EOS:k5=\\E[15~:k6=\\E[17~:k7=\\E[18~:\\\n\t:k8=\\E[19~:k9=\\E[20~:k;=\\E[21~:F1=\\E[23~:F2=\\E[24~:\\\n\t:F3=\\E[1;2P:F4=\\E[1;2Q:F5=\\E[1;2R:F6=\\E[1;2S:\\\n\t:F7=\\E[15;2~:F8=\\E[17;2~:F9=\\E[18;2~:FA=\\E[19;2~:kb=:\\\n\t:K2=\\EOE:kB=\\E[Z:kF=\\E[1;2B:kR=\\E[1;2A:*4=\\E[3;2~:\\\n\t:*7=\\E[1;2F:#2=\\E[1;2H:#3=\\E[2;2~:#4=\\E[1;2D:%c=\\E[6;2~:\\\n\t:%e=\\E[5;2~:%i=\\E[1;2C:kh=\\E[1~:@1=\\E[1~:kH=\\E[4~:\\\n\t:@7=\\E[4~:kN=\\E[6~:kP=\\E[5~:kI=\\E[2~:kD=\\E[3~:ku=\\EOA:\\\n\t:kd=\\EOB:kr=\\EOC:k...

Read more...

Revision history for this message
Francois Deppierraz (francois-ctrlaltdel) wrote :

Downgrading to ansible 2.4.6 also fixed this issue while keeping the cache activated, it looks like an upstream ansible bug.

Revision history for this message
Francois Deppierraz (francois-ctrlaltdel) wrote :

Upgrading to ansible version 2.6.0 also fixes this issue.

Revision history for this message
Francois Deppierraz (francois-ctrlaltdel) wrote :

Here's the upstream ansible bug: https://github.com/ansible/ansible/issues/33259.

Manually applying the patch from https://github.com/ansible/ansible/commit/dae737c8b714f2c6d28663ef0afeab10e6d3a667 worked for fixing this issue and I'm pretty sure that it could speed deployment up because of less unused jinja2 interpretation.

We discussed during this weekly meeting and the conclusion is that the safest way would be to get it patched in the next 2.5 point release and then have OSA use it.

Revision history for this message
Francois Deppierraz (francois-ctrlaltdel) wrote :

Here's how to patch ansible in an easy way:

root@os-deploy:~# (cd /opt/ansible-runtime/lib/python2.7/site-packages; curl -s https://github.com/ansible/ansible/commit/dae737c8b714f2c6d28663ef0afeab10e6d3a667.patch | patch -p2)
patching file ansible/parsing/yaml/dumper.py
patching file ansible/plugins/filter/core.py
patching file ansible/template/__init__.py
Hunk #2 succeeded at 358 (offset 2 lines).
Hunk #3 succeeded at 511 (offset 2 lines).
patching file ansible/vars/hostvars.py
root@os-deploy:~#

Unfortunately, getting this fixed in the next ansible 2.5 point release doesn't seem possible, see https://github.com/ansible/ansible/pull/50101

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.