ocf.log (node-14)
-----------------------
2015-03-25T10:47:50.469809+00:00 err: ERROR: /usr/lib/ocf/lib/heartbeat/ocf-shellfuncs: line 379: kill: (16245) - No such process
2015-03-25T10:47:50.478260+00:00 err: ERROR: /usr/lib/ocf/lib/heartbeat/ocf-shellfuncs: line 379: kill: (5423) - No such process
2015-03-25T10:47:50.483580+00:00 info: INFO: OpenStack DHCP Agent (neutron-dhcp-agent) stopped
pacemaker.log (node-14)
---------------------------------
<28>Mar 25 03:48:47 node-14 lrmd[6974]: warning: child_timeout_callback: p_neutron-dhcp-agent_stop_0 process (PID 19318) timed out
<28>Mar 25 03:48:47 node-14 lrmd[6974]: warning: operation_finished: p_neutron-dhcp-agent_stop_0:19318 - timed out after 60000ms
<27>Mar 25 03:48:47 node-14 crmd[6977]: error: process_lrm_event: LRM operation p_neutron-dhcp-agent_stop_0 (407) Timed Out (timeout=60000ms)
<29>Mar 25 03:48:47 node-14 crmd[6977]: notice: process_lrm_event: node-14.domain.tld-p_neutron-dhcp-agent_stop_0:407 [ 2015-03-25 03:47:50,827 - INFO - Started: /usr/bin/q-agent-cleanup.py --agent=dhcp
--cleanup-ports\n ]
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_cs_dispatch: Update relayed from node-7.domain.tld
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-p_neutron-dhcp-agent (INFINITY)
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_update: Sent update 2845: fail-count-p_neutron-dhcp-agent=INFINITY
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_cs_dispatch: Update relayed from node-7.domain.tld
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_trigger_update: Sending flush op to all hosts for: last-failure-p_neutron-dhcp-agent (1427280528)
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_update: Sent update 2848: last-failure-p_neutron-dhcp-agent=1427280528
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_cs_dispatch: Update relayed from node-7.domain.tld
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-p_neutron-dhcp-agent (INFINITY)
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_update: Sent update 2851: fail-count-p_neutron-dhcp-agent=INFINITY
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_cs_dispatch: Update relayed from node-7.domain.tld
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_trigger_update: Sending flush op to all hosts for: last-failure-p_neutron-dhcp-agent (1427280528)
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_update: Sent update 2854: last-failure-p_neutron-dhcp-agent=1427280528
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_update: Sent update 2857: fail-count-p_neutron-dhcp-agent=INFINITY
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_update: Sent update 2860: last-failure-p_neutron-dhcp-agent=1427280528
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_update: Sent update 2863: fail-count-p_neutron-dhcp-agent=INFINITY
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_update: Sent update 2866: last-failure-p_neutron-dhcp-agent=1427280528
Logs: (note that fuel master in GMT, nodes in PDT)
neutron-dhcp-agent (node-14) ------- ------- ------- ------- ------- openstack. common. service [req-1b24b7b7- de3c-4244- ae0e-1e75ed8243 99 None] Caught SIGTERM, exiting
-------
2015-03-25 03:47:47.461 16245 INFO neutron.
neutron-dhcp-agent (node-9) ------- ------- ------- ------- ------ 25T10:47: 47.467007+ 00:00 info: 2015-03-25 03:47:47.466 16290 INFO neutron. openstack. common. service [req-62050a3f- f4bf-4cee- 99ca-aae8f7e99b 6c None] Caught SIGTERM, exiting
-------
2015-03-
ocf.log (node-14) ------- ------- -- 25T10:47: 50.469809+ 00:00 err: ERROR: /usr/lib/ ocf/lib/ heartbeat/ ocf-shellfuncs: line 379: kill: (16245) - No such process 25T10:47: 50.478260+ 00:00 err: ERROR: /usr/lib/ ocf/lib/ heartbeat/ ocf-shellfuncs: line 379: kill: (5423) - No such process 25T10:47: 50.483580+ 00:00 info: INFO: OpenStack DHCP Agent (neutron- dhcp-agent) stopped
-------
2015-03-
2015-03-
2015-03-
pacemaker.log (node-14) ------- ------- ------- ----- callback: p_neutron- dhcp-agent_ stop_0 process (PID 19318) timed out dhcp-agent_ stop_0: 19318 - timed out after 60000ms dhcp-agent_ stop_0 (407) Timed Out (timeout=60000ms) domain. tld-p_neutron- dhcp-agent_ stop_0: 407 [ 2015-03-25 03:47:50,827 - INFO - Started: /usr/bin/ q-agent- cleanup. py --agent=dhcp update: Sending flush op to all hosts for: fail-count- p_neutron- dhcp-agent (INFINITY) update: Sent update 2845: fail-count- p_neutron- dhcp-agent= INFINITY update: Sending flush op to all hosts for: last-failure- p_neutron- dhcp-agent (1427280528) update: Sent update 2848: last-failure- p_neutron- dhcp-agent= 1427280528 update: Sending flush op to all hosts for: fail-count- p_neutron- dhcp-agent (INFINITY) update: Sent update 2851: fail-count- p_neutron- dhcp-agent= INFINITY update: Sending flush op to all hosts for: last-failure- p_neutron- dhcp-agent (1427280528) update: Sent update 2854: last-failure- p_neutron- dhcp-agent= 1427280528 update: Sent update 2857: fail-count- p_neutron- dhcp-agent= INFINITY update: Sent update 2860: last-failure- p_neutron- dhcp-agent= 1427280528 update: Sent update 2863: fail-count- p_neutron- dhcp-agent= INFINITY update: Sent update 2866: last-failure- p_neutron- dhcp-agent= 1427280528
-------
<28>Mar 25 03:48:47 node-14 lrmd[6974]: warning: child_timeout_
<28>Mar 25 03:48:47 node-14 lrmd[6974]: warning: operation_finished: p_neutron-
<27>Mar 25 03:48:47 node-14 crmd[6977]: error: process_lrm_event: LRM operation p_neutron-
<29>Mar 25 03:48:47 node-14 crmd[6977]: notice: process_lrm_event: node-14.
--cleanup-ports\n ]
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_cs_dispatch: Update relayed from node-7.domain.tld
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_trigger_
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_cs_dispatch: Update relayed from node-7.domain.tld
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_trigger_
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_cs_dispatch: Update relayed from node-7.domain.tld
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_trigger_
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_cs_dispatch: Update relayed from node-7.domain.tld
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_trigger_
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_
<29>Mar 25 03:48:47 node-14 attrd[6975]: notice: attrd_perform_