rssLink RSS for all categories
 
icon_red
icon_green
icon_red
icon_red
icon_blue
icon_green
icon_green
icon_red
icon_red
icon_red
icon_orange
icon_green
icon_green
icon_green
icon_green
icon_blue
icon_red
icon_orange
icon_red
icon_red
icon_red
icon_red
icon_green
icon_red
icon_red
icon_red
icon_red
icon_orange
icon_green
 

FS#2896 — FS#6857 — Paris : th2-10-n5

Attached to Project— ADSL / SDSL
Incident
Backend / Core
CLOSED
100%
Le switch has crashed brutally :
Jun 8 15:33:32 th2-10-n5 : 2012 Jun 8 13:33:09 UTC: %SYSMGR-3-HEARTBEAT_FAILURE: Service "fwm" sent SIGABRT for not setting heartbeat for last 10 periods. Last heartbeat 51.93 secs ago.
Jun 8 15:33:33 th2-10-n5 : 2012 Jun 8 13:33:10 UTC: %SYSMGR-2-SERVICE_CRASHED: Service "fwm" (PID 3129) hasn't caught signal 6 (core will be saved).
Kernel uptime is 0 day(s), 0 hour(s), 12 minute(s), 15 second(s)
Last reset at 519567 usecs after Fri Jun 8 13:33:21 2012
Reason: Reset triggered due to HA policy of Reset
System version: 5.1(3)N1(1)
Service: fwm hap reset
We seek the origin of the switch software crash.

Date:  Monday, 11 June 2012, 10:52AM
Reason for closing:  Done
Comment by OVH - Friday, 08 June 2012, 16:31PM

8 15:30:24 th2-10-n5 : 2012 Jun 8 13:30:02 UTC: %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet2/5 is down (Link failure)
Jun 8 15:30:24 th2-10-n5 : 2012 Jun 8 13:30:02 UTC: %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel127: Ethernet2/6 is down
Jun 8 15:30:24 th2-10-n5 : 2012 Jun 8 13:30:02 UTC: %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet2/4 is down (Link failure)
Jun 8 15:30:24 th2-10-n5 : 2012 Jun 8 13:30:02 UTC: %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel127: Ethernet2/5 is down
Jun 8 15:30:24 th2-10-n5 : 2012 Jun 8 13:30:02 UTC: %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel127: first operational port changed from Ethernet2/16 to none
Jun 8 15:30:24 th2-10-n5 : 2012 Jun 8 13:30:02 UTC: %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel127: first operational port changed from Ethernet2/15 to Ethernet2/16
Jun 8 15:30:24 th2-10-n5 : 2012 Jun 8 13:30:02 UTC: %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet2/3 is down (Link failure)
Jun 8 15:30:24 th2-10-n5 : 2012 Jun 8 13:30:02 UTC: %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel127: Ethernet2/4 is down
Jun 8 15:30:24 th2-10-n5 : 2012 Jun 8 13:30:02 UTC: %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel127: Ethernet2/3 is down
Jun 8 15:30:24 th2-10-n5 : 2012 Jun 8 13:30:01 UTC: %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet2/2 is down (Link failure)
Jun 8 15:30:24 th2-10-n5 : 2012 Jun 8 13:30:01 UTC: %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel127: Ethernet2/2 is down
Jun 8 15:30:24 th2-10-n5 : 2012 Jun 8 13:30:01 UTC: last message repeated 2 times
Jun 8 15:30:24 th2-10-n5 : 2012 Jun 8 13:30:01 UTC: Jun 8 13:30:01 %KERN-3-SYSTEM_MSG: pfm_gem_hp_removal_thread: gem hotlplug unregister successful - kernel
Jun 8 15:30:23 th2-10-n5 : 2012 Jun 8 13:30:00 UTC: %BTCM-2-BTCM_LOG_ROUTER_PORT_STATUS_CHANGE: Router port 0x1a080000 is down.
Jun 8 15:30:23 th2-10-n5 : 2012 Jun 8 13:30:00 UTC: %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet2/1 is down (Link failure)
Jun 8 15:30:23 th2-10-n5 : 2012 Jun 8 13:30:00 UTC: %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel127: Ethernet2/1 is down
Jun 8 15:30:23 th2-10-n5 : 2012 Jun 8 13:30:00 UTC: Jun 8 13:30:00 %KERN-3-SYSTEM_MSG: pfm_gem_hp_intr_handler - Got slot-1 GEM Interrupt! - kernel


Comment by OVH - Friday, 08 June 2012, 16:32PM

A problem between the switch and the routing module are at the origin of the software crash. We will forward the problem to cisco.

[Fri Jun 8 13:40:14 2012 870037 usecs]
fwm_l2mp_es.c:746(fwm_l2mp_es_apply_df_ftag_update) intf: 1600000e does
not exist in l2mp intf mgr caller ra: 0x80c99cc 0x81c5208 0x81ea7e8
0x8146d76 0xb36bada3 0xb36d06e8 0x814b459 0x414735c5
[Fri Jun 8 13:40:14 2012 867753 usecs]
fwm_fwim.c:7340(fwm_fwim_copy_info_to_pc) fwm_fwim_copy_info_to_pc: fabric
pinning info unkwn for if 0x1a00e000!
caller ra: 0x8245c8f 0x826a730 0x81e9cbc 0x8146d76 0xb36bada3 0xb36d06e8
0x814b459 0x414735c5

[Fri Jun 8 13:40:14 2012 662180 usecs]
fwm_msg_handler:5242(fwm_ppf_callback) fwm ppf callback: ppf info didn't
have a callback or an mts reference
caller ra: 0x844ffe5 0xb3d33254 0xb3085422 0xb30856d4 0xb306c3d2
0xb33f17a4 0xb3d344c6 0x8460631