CI: jobs failing with Error: /Stage[main]/Rabbitmq::Config/Rabbitmq_erlang_cookie[/var/lib/rabbitmq/.erlang.cookie]/content: <snip> `encode': "\xC3" on US-ASCII (Encoding::InvalidByteSequenceError)

Bug #1643059 reported by James Slagle
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
James Slagle

Bug Description

Seeing a lot (if not all) job failing with this error on the overcloud-controller-0:

Nov 18 19:37:25 localhost os-collect-config: #033[1;31mError: /Stage[main]/Rabbitmq::Config/Rabbitmq_erlang_cookie[/var/lib/rabbitmq/.erlang.cookie]/content: change from NFNKSOGWBJTZCZDNLQEN to weDcaWyG7Obatc48K8T6 failed: Execution of '/usr/bin/puppet resource service rabbitmq-server ensure=stopped' returned 1: /usr/share/gems/gems/json-1.7.7/lib/json/common.rb:155:in `encode': "\xC3" on US-ASCII (Encoding::InvalidByteSequenceError)

Example failed job:

http://logs.openstack.org/62/385162/6/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/3a08ed4/

Tags: ci
Revision history for this message
James Slagle (james-slagle) wrote :

Comparing package manifests from a failed job to a recently passed one, the differences are:

[jslagle@localhost tmp.L5KeHNvp42]$ diff -u work2 broke2
--- work2 2016-11-18 15:40:08.286175901 -0500
+++ broke2 2016-11-18 15:39:23.380459895 -0500
@@ -617,7 +617,7 @@
 openvswitch-2.5.0-2.el7.x86_64
 os-apply-config-5.0.1-0.20161116200506.d72c4d9.el7.centos.noarch
 os-collect-config-6.0.0-0.20161117222156.e8e2f3d.el7.centos.noarch
-os-net-config-6.0.0-0.20161117225400.9971eb3.el7.centos.noarch
+os-net-config-6.0.0-0.20161118182225.433a0f3.el7.centos.noarch
 os-prober-1.58-5.el7.x86_64
 os-refresh-config-5.0.1-0.20161116194316.796f4e8.el7.centos.noarch
 p11-kit-0.20.7-3.el7.x86_64
@@ -770,7 +770,7 @@
 puppet-pacemaker-0.4.0-0.20161108164007.1e2560f.el7.centos.noarch
 puppet-rabbitmq-5.6.0-0.20161111010126.12aa7cc.el7.centos.noarch
 puppet-redis-1.2.3-0.20161015002837.9711564.el7.centos.noarch
-puppet-remote-0.0.1-0.20161003203714.35cc557.el7.centos.noarch
+puppet-remote-0.0.1-0.20161118173454.b370d28.el7.centos.noarch
 puppet-rsync-0.4.0-0.20161003183908.8cc3c6f.el7.centos.noarch
 puppet-sahara-9.4.0-0.20161117203415.99639c8.el7.centos.noarch
 puppet-sensu-2.1.0-0.20161107104621.6d10bae.el7.centos.noarch
@@ -783,7 +783,7 @@
 puppet-tempest-9.4.0-0.20161117195904.185f467.el7.centos.noarch
 puppet-timezone-3.3.0-0.20161003185332.cf62f1b.el7.centos.noarch
 puppet-tomcat-1.6.0-0.20161111192530.f53d54b.el7.centos.noarch
-puppet-tripleo-6.0.0-0.20161118172008.e1b7ad8.el7.centos.noarch
+puppet-tripleo-6.0.0-0.20161118162559.92f9c95.el7.centos.noarch
 puppet-trove-9.4.0-0.20161117203846.47e28a5.el7.centos.noarch
 puppet-uchiwa-1.0.1-0.20161003185621.64ce619.el7.centos.noarch
 puppet-vcsrepo-1.4.0-0.20161118170152.f012768.el7.centos.noarch

The new puppet-remote might be the problem. I noticed this new commit:

https://github.com/paramite/puppet-remote/commit/b370d28ca4eff9af64e176e83ae0cbccb7ccb144

The added file metadata.json is not valid json (should not be a comma at end of line 25).

I'm not sure if that would cause this issue or not.

tags: added: alert ci
Changed in tripleo:
status: New → Triaged
importance: Undecided → Critical
milestone: none → ocata-2
Revision history for this message
James Slagle (james-slagle) wrote :
Revision history for this message
Gabriele Cerami (gcerami) wrote :

I think https://github.com/paramite/puppet-remote/issues/2 is the issue here. The á in Mágr encoding is \xC3\xA1 in iso-8859-1.

Revision history for this message
Gabriele Cerami (gcerami) wrote :
Revision history for this message
Gabriele Cerami (gcerami) wrote :

Parsing was blocking at á because it comes first, but fixed that, parsing would have blocked at the incorrect comma after, so both pull requests are needed.

They are both merged.

Changed in tripleo:
status: Triaged → Fix Released
tags: removed: alert
Changed in tripleo:
assignee: nobody → James Slagle (james-slagle)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.