Dec 7 14:55:46 radius1 lrmd: [16595]: WARN: raddb_ip:monitor process (PID 24109) timed out (try 1). Killing with signal SIGTERM (15).
Dec 7 14:55:46 radius1 lrmd: [16595]: WARN: operation monitor[4] on raddb_ip for client 16598: pid 24109 timed out
Dec 7 14:55:47 radius1 crmd: [16598]: ERROR: process_lrm_event: LRM operation raddb_ip_monitor_15000 (4) Timed Out (timeout=20000ms)
Dec 7 14:55:47 radius1 crmd: [16598]: info: process_graph_event: Action raddb_ip_monitor_15000/8 (2:-2;8:4:0:eda89d53-139f-4d09-8754-803f1947c411) initiated by a different transitioner
Dec 7 14:55:47 radius1 crmd: [16598]: info: abort_transition_graph: process_graph_event:467 - Triggered transition abort (complete=1, tag=lrm_rsc_op, id=raddb_ip_monitor_15000, magic=2:-2;8:4:0:eda89d53-139f-4d09-8754-803f1947c411, cib=0.52.131) : Foreign event
Dec 7 14:55:47 radius1 crmd: [16598]: WARN: update_failcount: Updating failcount for raddb_ip on radius1 after failed monitor: rc=-2 (update=value++, time=1323248147)
Dec 7 14:55:47 radius1 crmd: [16598]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Dec 7 14:55:47 radius1 crmd: [16598]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Dec 7 14:55:47 radius1 crmd: [16598]: info: do_pe_invoke: Query 63: Requesting the current CIB: S_POLICY_ENGINE
Dec 7 14:55:48 radius1 crmd: [16598]: info: do_pe_invoke_callback: Invoking the PE: query=63, ref=pe_calc-dc-1323248148-29, seq=304, quorate=1
Dec 7 14:55:48 radius1 attrd: [16596]: info: attrd_local_callback: Expanded fail-count-raddb_ip=value++ to 1
Dec 7 14:55:48 radius1 attrd: [16596]: info: attrd_trigger_update: Sending flush op to all hosts for: fail-count-raddb_ip (1)
Dec 7 14:55:48 radius1 attrd: [16596]: info: attrd_perform_update: Sent update 50: fail-count-raddb_ip=1
Dec 7 14:55:48 radius1 crmd: [16598]: info: abort_transition_graph: te_update_diff:149 - Triggered transition abort (complete=1, tag=nvpair, id=status-radius1-fail-count-raddb_ip, magic=NA, cib=0.52.132) : Transient attribute: update
Dec 7 14:55:48 radius1 crmd: [16598]: info: do_pe_invoke: Query 64: Requesting the current CIB: S_POLICY_ENGINE
Dec 7 14:55:48 radius1 crmd: [16598]: info: do_pe_invoke_callback: Invoking the PE: query=64, ref=pe_calc-dc-1323248148-30, seq=304, quorate=1
Dec 7 14:55:48 radius1 attrd: [16596]: info: attrd_trigger_update: Sending flush op to all hosts for: last-failure-raddb_ip (1323248147)
Dec 7 14:55:48 radius1 attrd: [16596]: info: attrd_perform_update: Sent update 53: last-failure-raddb_ip=1323248147
Dec 7 14:55:48 radius1 crmd: [16598]: info: abort_transition_graph: te_update_diff:149 - Triggered transition abort (complete=1, tag=nvpair, id=status-radius1-last-failure-raddb_ip, magic=NA, cib=0.52.133) : Transient attribute: update
Dec 7 14:55:48 radius1 crmd: [16598]: info: do_pe_invoke: Query 65: Requesting the current CIB: S_POLICY_ENGINE
Dec 7 14:55:48 radius1 crmd: [16598]: info: do_pe_invoke_callback: Invoking the PE: query=65, ref=pe_calc-dc-1323248148-31, seq=304, quorate=1
Dec 7 14:55:49 radius1 pengine: [16597]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 7 14:55:49 radius1 pengine: [16597]: WARN: unpack_rsc_op: Processing failed op raddb_ip_monitor_15000 on radius1: unknown exec error (-2)
Dec 7 14:55:49 radius1 pengine: [16597]: notice: group_print: Resource Group: raddb
Dec 7 14:55:49 radius1 pengine: [16597]: notice: native_print: raddb_ip#011(ocf::heartbeat:IPaddr2):#011Started radius1 FAILED
Dec 7 14:55:49 radius1 pengine: [16597]: notice: RecurringOp: Start recurring monitor (15s) for raddb_ip on radius1
Dec 7 14:55:49 radius1 pengine: [16597]: notice: LogActions: Recover raddb_ip#011(Started radius1)
Dec 7 14:55:50 radius1 crmd: [16598]: info: handle_response: pe_calc calculation pe_calc-dc-1323248148-29 is obsolete
Dec 7 14:55:55 radius1 pengine: [16597]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 7 14:55:55 radius1 pengine: [16597]: WARN: unpack_rsc_op: Processing failed op raddb_ip_monitor_15000 on radius1: unknown exec error (-2)
Dec 7 14:55:55 radius1 pengine: [16597]: notice: group_print: Resource Group: raddb
Dec 7 14:55:55 radius1 pengine: [16597]: notice: native_print: raddb_ip#011(ocf::heartbeat:IPaddr2):#011Started radius1 FAILED
Dec 7 14:55:55 radius1 pengine: [16597]: notice: common_apply_stickiness: raddb_ip can fail 999999 more times on radius1 before being forced off
Dec 7 14:55:55 radius1 pengine: [16597]: notice: RecurringOp: Start recurring monitor (15s) for raddb_ip on radius1
Dec 7 14:55:55 radius1 pengine: [16597]: notice: LogActions: Recover raddb_ip#011(Started radius1)
Dec 7 14:55:55 radius1 crmd: [16598]: info: handle_response: pe_calc calculation pe_calc-dc-1323248148-30 is obsolete
Dec 7 14:55:59 radius1 pengine: [16597]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 7 14:55:59 radius1 pengine: [16597]: WARN: unpack_rsc_op: Processing failed op raddb_ip_monitor_15000 on radius1: unknown exec error (-2)
Dec 7 14:55:59 radius1 pengine: [16597]: notice: group_print: Resource Group: raddb
Dec 7 14:55:59 radius1 pengine: [16597]: notice: native_print: raddb_ip#011(ocf::heartbeat:IPaddr2):#011Started radius1 FAILED
Dec 7 14:55:59 radius1 pengine: [16597]: notice: common_apply_stickiness: raddb_ip can fail 999999 more times on radius1 before being forced off
Dec 7 14:55:59 radius1 pengine: [16597]: notice: RecurringOp: Start recurring monitor (15s) for raddb_ip on radius1
Dec 7 14:55:59 radius1 pengine: [16597]: notice: LogActions: Recover raddb_ip#011(Started radius1)
Dec 7 14:55:59 radius1 crmd: [16598]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Dec 7 14:55:59 radius1 crmd: [16598]: info: unpack_graph: Unpacked transition 5: 8 actions in 8 synapses
Dec 7 14:55:59 radius1 crmd: [16598]: info: do_te_invoke: Processing graph 5 (ref=pe_calc-dc-1323248148-31) derived from /var/lib/pengine/pe-input-811.bz2
Dec 7 14:55:59 radius1 crmd: [16598]: info: te_pseudo_action: Pseudo action 10 fired and confirmed
Dec 7 14:55:59 radius1 crmd: [16598]: info: te_rsc_command: Initiating action 2: stop raddb_ip_stop_0 on radius1 (local)
Dec 7 14:55:59 radius1 lrmd: [16595]: info: cancel_op: operation monitor[4] on raddb_ip for client 16598, its parameters: CRM_meta_interval=[15000] ip=[10.99.2.57] cidr_netmask=[32] CRM_meta_timeout=[20000] crm_feature_set=[3.0.5] CRM_meta_name=[monitor] cancelled
Dec 7 14:55:59 radius1 crmd: [16598]: info: do_lrm_rsc_op: Performing key=2:5:0:089051e1-b418-4a37-8261-2296bc5c8d29 op=raddb_ip_stop_0 )
Dec 7 14:55:59 radius1 lrmd: [16595]: info: rsc:raddb_ip stop[5] (pid 24163)
Dec 7 14:55:59 radius1 crmd: [16598]: info: process_lrm_event: LRM operation raddb_ip_monitor_15000 (call=4, status=1, cib-update=0, confirmed=true) Cancelled
Dec 7 14:56:19 radius1 lrmd: [16595]: WARN: raddb_ip:stop process (PID 24163) timed out (try 1). Killing with signal SIGTERM (15).
Dec 7 14:56:20 radius1 lrmd: [16595]: WARN: operation stop[5] on raddb_ip for client 16598: pid 24163 timed out
Dec 7 14:56:20 radius1 crmd: [16598]: ERROR: process_lrm_event: LRM operation raddb_ip_stop_0 (5) Timed Out (timeout=20000ms)
Dec 7 14:56:20 radius1 crmd: [16598]: WARN: status_from_rc: Action 2 (raddb_ip_stop_0) on radius1 failed (target: 0 vs. rc: -2): Error
Dec 7 14:56:20 radius1 crmd: [16598]: WARN: update_failcount: Updating failcount for raddb_ip on radius1 after failed stop: rc=-2 (update=INFINITY, time=1323248180)
Dec 7 14:56:20 radius1 crmd: [16598]: info: abort_transition_graph: match_graph_event:265 - Triggered transition abort (complete=0, tag=lrm_rsc_op, id=raddb_ip_stop_0, magic=2:-2;2:5:0:089051e1-b418-4a37-8261-2296bc5c8d29, cib=0.52.134) : Event failed
Dec 7 14:56:20 radius1 crmd: [16598]: info: update_abort_priority: Abort priority upgraded from 0 to 1
Dec 7 14:56:20 radius1 crmd: [16598]: info: update_abort_priority: Abort action done superceeded by restart
Dec 7 14:56:20 radius1 crmd: [16598]: info: match_graph_event: Action raddb_ip_stop_0 (2) confirmed on radius1 (rc=4)
Dec 7 14:56:20 radius1 attrd: [16596]: info: attrd_trigger_update: Sending flush op to all hosts for: fail-count-raddb_ip (INFINITY)
Dec 7 14:56:20 radius1 crmd: [16598]: info: run_graph: ====================================================
Dec 7 14:56:20 radius1 crmd: [16598]: notice: run_graph: Transition 5 (Complete=2, Pending=0, Fired=0, Skipped=6, Incomplete=0, Source=/var/lib/pengine/pe-input-811.bz2): Stopped
Dec 7 14:56:20 radius1 crmd: [16598]: info: te_graph_trigger: Transition 5 is now complete
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd ]
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_pe_invoke: Query 67: Requesting the current CIB: S_POLICY_ENGINE
Dec 7 14:56:20 radius1 attrd: [16596]: info: attrd_perform_update: Sent update 55: fail-count-raddb_ip=INFINITY
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_pe_invoke_callback: Invoking the PE: query=67, ref=pe_calc-dc-1323248180-33, seq=304, quorate=1
Dec 7 14:56:20 radius1 pengine: [16597]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 7 14:56:20 radius1 pengine: [16597]: WARN: unpack_rsc_op: Processing failed op raddb_ip_monitor_15000 on radius1: unknown exec error (-2)
Dec 7 14:56:20 radius1 pengine: [16597]: WARN: unpack_rsc_op: Processing failed op raddb_ip_stop_0 on radius1: unknown exec error (-2)
Dec 7 14:56:20 radius1 pengine: [16597]: notice: group_print: Resource Group: raddb
Dec 7 14:56:20 radius1 pengine: [16597]: notice: native_print: raddb_ip#011(ocf::heartbeat:IPaddr2):#011Started radius1 (unmanaged) FAILED
Dec 7 14:56:20 radius1 pengine: [16597]: notice: common_apply_stickiness: raddb_ip can fail 999999 more times on radius1 before being forced off
Dec 7 14:56:20 radius1 pengine: [16597]: notice: LogActions: Leave raddb_ip#011(Started unmanaged)
Dec 7 14:56:20 radius1 attrd: [16596]: info: attrd_trigger_update: Sending flush op to all hosts for: last-failure-raddb_ip (1323248180)
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Dec 7 14:56:20 radius1 crmd: [16598]: info: unpack_graph: Unpacked transition 6: 3 actions in 3 synapses
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_te_invoke: Processing graph 6 (ref=pe_calc-dc-1323248180-33) derived from /var/lib/pengine/pe-input-812.bz2
Dec 7 14:56:20 radius1 crmd: [16598]: info: te_pseudo_action: Pseudo action 9 fired and confirmed
Dec 7 14:56:20 radius1 crmd: [16598]: notice: run_graph: ====================================================
Dec 7 14:56:20 radius1 crmd: [16598]: WARN: run_graph: Transition 6 (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=2, Source=/var/lib/pengine/pe-input-812.bz2): Terminated
Dec 7 14:56:20 radius1 crmd: [16598]: ERROR: te_graph_trigger: Transition failed: terminated
Dec 7 14:56:20 radius1 crmd: [16598]: WARN: print_graph: Graph 6 (3 actions in 3 synapses): batch-limit=30 jobs, network-delay=60000ms
Dec 7 14:56:20 radius1 crmd: [16598]: WARN: print_graph: Synapse 0 is pending (priority: 0)
Dec 7 14:56:20 radius1 crmd: [16598]: WARN: print_elem: [Action 10]: Pending (id: raddb_stopped_0, type: pseduo, priority: 0)
Dec 7 14:56:20 radius1 crmd: [16598]: WARN: print_elem: * [Input 5]: Pending (id: raddb_ip_stop_0, loc: radius1, priority: 0)
Dec 7 14:56:20 radius1 crmd: [16598]: WARN: print_elem: * [Input 9]: Completed (id: raddb_stop_0, type: pseduo, priority: 0)
Dec 7 14:56:20 radius1 crmd: [16598]: WARN: print_graph: Synapse 1 was confirmed (priority: 0)
Dec 7 14:56:20 radius1 crmd: [16598]: WARN: print_graph: Synapse 2 is pending (priority: 0)
Dec 7 14:56:20 radius1 crmd: [16598]: WARN: print_elem: [Action 7]: Pending (id: raddb_start_0, type: pseduo, priority: 0)
Dec 7 14:56:20 radius1 crmd: [16598]: WARN: print_elem: * [Input 10]: Pending (id: raddb_stopped_0, type: pseduo, priority: 0)
Dec 7 14:56:20 radius1 crmd: [16598]: info: te_graph_trigger: Transition 6 is now complete
Dec 7 14:56:20 radius1 crmd: [16598]: info: notify_crmd: Transition 6 status: done - <null>
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_state_transition: Starting PEngine Recheck Timer
Dec 7 14:56:20 radius1 crmd: [16598]: info: abort_transition_graph: te_update_diff:149 - Triggered transition abort (complete=1, tag=nvpair, id=status-radius1-fail-count-raddb_ip, magic=NA, cib=0.52.135) : Transient attribute: update
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_pe_invoke: Query 68: Requesting the current CIB: S_POLICY_ENGINE
Dec 7 14:56:20 radius1 attrd: [16596]: info: attrd_perform_update: Sent update 57: last-failure-raddb_ip=1323248180
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_pe_invoke_callback: Invoking the PE: query=68, ref=pe_calc-dc-1323248180-34, seq=304, quorate=1
Dec 7 14:56:20 radius1 crmd: [16598]: info: abort_transition_graph: te_update_diff:149 - Triggered transition abort (complete=1, tag=nvpair, id=status-radius1-last-failure-raddb_ip, magic=NA, cib=0.52.136) : Transient attribute: update
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_pe_invoke: Query 69: Requesting the current CIB: S_POLICY_ENGINE
Dec 7 14:56:20 radius1 crmd: [16598]: info: do_pe_invoke_callback: Invoking the PE: query=69, ref=pe_calc-dc-1323248180-35, seq=304, quorate=1
Dec 7 14:56:28 radius1 pengine: [16597]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 7 14:56:28 radius1 pengine: [16597]: WARN: unpack_rsc_op: Processing failed op raddb_ip_monitor_15000 on radius1: unknown exec error (-2)
Dec 7 14:56:28 radius1 pengine: [16597]: WARN: unpack_rsc_op: Processing failed op raddb_ip_stop_0 on radius1: unknown exec error (-2)
Dec 7 14:56:28 radius1 pengine: [16597]: notice: group_print: Resource Group: raddb
Dec 7 14:56:28 radius1 pengine: [16597]: notice: native_print: raddb_ip#011(ocf::heartbeat:IPaddr2):#011Started radius1 (unmanaged) FAILED
Dec 7 14:56:28 radius1 pengine: [16597]: WARN: common_apply_stickiness: Forcing raddb_ip away from radius1 after 1000000 failures (max=1000000)
Dec 7 14:56:28 radius1 pengine: [16597]: notice: LogActions: Leave raddb_ip#011(Started unmanaged)
Dec 7 14:56:28 radius1 crmd: [16598]: info: handle_response: pe_calc calculation pe_calc-dc-1323248180-34 is obsolete
Dec 7 14:56:31 radius1 pengine: [16597]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 7 14:56:31 radius1 pengine: [16597]: WARN: unpack_rsc_op: Processing failed op raddb_ip_monitor_15000 on radius1: unknown exec error (-2)
Dec 7 14:56:31 radius1 pengine: [16597]: WARN: unpack_rsc_op: Processing failed op raddb_ip_stop_0 on radius1: unknown exec error (-2)
Dec 7 14:56:31 radius1 pengine: [16597]: notice: group_print: Resource Group: raddb
Dec 7 14:56:31 radius1 pengine: [16597]: notice: native_print: raddb_ip#011(ocf::heartbeat:IPaddr2):#011Started radius1 (unmanaged) FAILED
Dec 7 14:56:31 radius1 pengine: [16597]: WARN: common_apply_stickiness: Forcing raddb_ip away from radius1 after 1000000 failures (max=1000000)
Dec 7 14:56:31 radius1 pengine: [16597]: notice: LogActions: Leave raddb_ip#011(Started unmanaged)
Dec 7 14:56:31 radius1 crmd: [16598]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Dec 7 14:56:31 radius1 crmd: [16598]: info: unpack_graph: Unpacked transition 8: 3 actions in 3 synapses
Dec 7 14:56:31 radius1 crmd: [16598]: info: do_te_invoke: Processing graph 8 (ref=pe_calc-dc-1323248180-35) derived from /var/lib/pengine/pe-input-814.bz2
Dec 7 14:56:31 radius1 crmd: [16598]: info: te_pseudo_action: Pseudo action 9 fired and confirmed
Dec 7 14:56:31 radius1 crmd: [16598]: notice: run_graph: ====================================================
Dec 7 14:56:31 radius1 crmd: [16598]: WARN: run_graph: Transition 8 (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=2, Source=/var/lib/pengine/pe-input-814.bz2): Terminated
Dec 7 14:56:31 radius1 crmd: [16598]: ERROR: te_graph_trigger: Transition failed: terminated
Dec 7 14:56:31 radius1 crmd: [16598]: WARN: print_graph: Graph 8 (3 actions in 3 synapses): batch-limit=30 jobs, network-delay=60000ms
Dec 7 14:56:31 radius1 crmd: [16598]: WARN: print_graph: Synapse 0 is pending (priority: 0)
Dec 7 14:56:31 radius1 crmd: [16598]: WARN: print_elem: [Action 10]: Pending (id: raddb_stopped_0, type: pseduo, priority: 0)
Dec 7 14:56:31 radius1 crmd: [16598]: WARN: print_elem: * [Input 5]: Pending (id: raddb_ip_stop_0, loc: radius1, priority: 0)
Dec 7 14:56:31 radius1 crmd: [16598]: WARN: print_elem: * [Input 9]: Completed (id: raddb_stop_0, type: pseduo, priority: 0)
Dec 7 14:56:31 radius1 crmd: [16598]: WARN: print_graph: Synapse 1 was confirmed (priority: 0)
Dec 7 14:56:31 radius1 crmd: [16598]: WARN: print_graph: Synapse 2 is pending (priority: 0)
Dec 7 14:56:31 radius1 crmd: [16598]: WARN: print_elem: [Action 7]: Pending (id: raddb_start_0, type: pseduo, priority: 0)
Dec 7 14:56:31 radius1 crmd: [16598]: WARN: print_elem: * [Input 10]: Pending (id: raddb_stopped_0, type: pseduo, priority: 0)
Dec 7 14:56:31 radius1 crmd: [16598]: info: te_graph_trigger: Transition 8 is now complete
Dec 7 14:56:31 radius1 crmd: [16598]: info: notify_crmd: Transition 8 status: done - <null>
Dec 7 14:56:31 radius1 crmd: [16598]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Dec 7 14:56:31 radius1 crmd: [16598]: info: do_state_transition: Starting PEngine Recheck TimerAdd a code snippet to your website: www.paste.org