From mcollecitve log on node-1:
2015-03-11T20:30:42.657620+00:00 debug: 20:30:42.737132 #8391] DEBUG -- : rabbitmq.rb:225:in `receive' Waiting for a message from RabbitMQ 2015-03-11T20:30:42.657878+00:00 debug: 20:30:42.737740 #8391] DEBUG -- : runnerstats.rb:49:in `received' Incrementing total stat 2015-03-11T20:30:42.657993+00:00 debug: 20:30:42.737853 #8391] DEBUG -- : pluginmanager.rb:83:in `[]' Returning cached plugin security_plugin with class MCollective::Security::Psk 2015-03-11T20:30:42.658165+00:00 warning: 20:30:42.738090 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: Failed to handle message: TypeError: incompatible marshal file format (can't be read) 2015-03-11T20:30:42.658258+00:00 info: format version 4.8 required; 89.111 given 2015-03-11T20:30:42.658417+00:00 warning: 20:30:42.738166 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/libexec/mcollective/mcollective/security/psk.rb:22:in `load' 2015-03-11T20:30:42.658472+00:00 warning: 20:30:42.738228 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/libexec/mcollective/mcollective/security/psk.rb:22:in `decodemsg' 2015-03-11T20:30:42.658671+00:00 warning: 20:30:42.738287 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ruby/site_ruby/1.8/mcollective/message.rb:178:in `decode!' 2015-03-11T20:30:42.658759+00:00 warning: 20:30:42.738345 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ruby/site_ruby/1.8/mcollective/runner.rb:121:in `receive' 2015-03-11T20:30:42.658861+00:00 warning: 20:30:42.738402 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ruby/site_ruby/1.8/mcollective/runner.rb:55:in `run' 2015-03-11T20:30:42.659087+00:00 warning: 20:30:42.738461 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ruby/site_ruby/1.8/mcollective/runner.rb:53:in `loop' 2015-03-11T20:30:42.659205+00:00 warning: 20:30:42.738518 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ruby/site_ruby/1.8/mcollective/runner.rb:53:in `run' 2015-03-11T20:30:42.659321+00:00 warning: 20:30:42.738586 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ruby/site_ruby/1.8/mcollective/unix_daemon.rb:30:in `daemonize_runner' 2015-03-11T20:30:42.659421+00:00 warning: 20:30:42.738650 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ruby/site_ruby/1.8/mcollective/unix_daemon.rb:13:in `daemonize' 2015-03-11T20:30:42.659586+00:00 warning: 20:30:42.738709 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ruby/site_ruby/1.8/mcollective/unix_daemon.rb:5:in `fork' 2015-03-11T20:30:42.659692+00:00 warning: 20:30:42.738767 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ruby/site_ruby/1.8/mcollective/unix_daemon.rb:5:in `daemonize' 2015-03-11T20:30:42.659785+00:00 warning: 20:30:42.738825 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ruby/site_ruby/1.8/mcollective/unix_daemon.rb:20:in `daemonize_runner' 2015-03-11T20:30:42.659969+00:00 warning: 20:30:42.738889 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/sbin/mcollectived:47
From mcollecitve log on node-1:
2015-03- 11T20:30: 42.657620+ 00:00 debug: 20:30:42.737132 #8391] DEBUG -- : rabbitmq.rb:225:in `receive' Waiting for a message from RabbitMQ 11T20:30: 42.657878+ 00:00 debug: 20:30:42.737740 #8391] DEBUG -- : runnerstats. rb:49:in `received' Incrementing total stat 11T20:30: 42.657993+ 00:00 debug: 20:30:42.737853 #8391] DEBUG -- : pluginmanager. rb:83:in `[]' Returning cached plugin security_plugin with class MCollective: :Security: :Psk 11T20:30: 42.658165+ 00:00 warning: 20:30:42.738090 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: Failed to handle message: TypeError: incompatible marshal file format (can't be read) 11T20:30: 42.658258+ 00:00 info: format version 4.8 required; 89.111 given 11T20:30: 42.658417+ 00:00 warning: 20:30:42.738166 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/libexec/ mcollective/ mcollective/ security/ psk.rb: 22:in `load' 11T20:30: 42.658472+ 00:00 warning: 20:30:42.738228 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/libexec/ mcollective/ mcollective/ security/ psk.rb: 22:in `decodemsg' 11T20:30: 42.658671+ 00:00 warning: 20:30:42.738287 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ ruby/site_ ruby/1. 8/mcollective/ message. rb:178: in `decode!' 11T20:30: 42.658759+ 00:00 warning: 20:30:42.738345 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ ruby/site_ ruby/1. 8/mcollective/ runner. rb:121: in `receive' 11T20:30: 42.658861+ 00:00 warning: 20:30:42.738402 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ ruby/site_ ruby/1. 8/mcollective/ runner. rb:55:in `run' 11T20:30: 42.659087+ 00:00 warning: 20:30:42.738461 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ ruby/site_ ruby/1. 8/mcollective/ runner. rb:53:in `loop' 11T20:30: 42.659205+ 00:00 warning: 20:30:42.738518 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ ruby/site_ ruby/1. 8/mcollective/ runner. rb:53:in `run' 11T20:30: 42.659321+ 00:00 warning: 20:30:42.738586 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ ruby/site_ ruby/1. 8/mcollective/ unix_daemon. rb:30:in `daemonize_runner' 11T20:30: 42.659421+ 00:00 warning: 20:30:42.738650 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ ruby/site_ ruby/1. 8/mcollective/ unix_daemon. rb:13:in `daemonize' 11T20:30: 42.659586+ 00:00 warning: 20:30:42.738709 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ ruby/site_ ruby/1. 8/mcollective/ unix_daemon. rb:5:in `fork' 11T20:30: 42.659692+ 00:00 warning: 20:30:42.738767 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ ruby/site_ ruby/1. 8/mcollective/ unix_daemon. rb:5:in `daemonize' 11T20:30: 42.659785+ 00:00 warning: 20:30:42.738825 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/lib/ ruby/site_ ruby/1. 8/mcollective/ unix_daemon. rb:20:in `daemonize_runner' 11T20:30: 42.659969+ 00:00 warning: 20:30:42.738889 #8391] WARN -- : psk.rb:22:in `decodemsg' PLMC10: /usr/sbin/ mcollectived: 47
2015-03-
2015-03-
2015-03-
2015-03-
2015-03-
2015-03-
2015-03-
2015-03-
2015-03-
2015-03-
2015-03-
2015-03-
2015-03-
2015-03-
2015-03-
2015-03-
2015-03-