OVHcloud Network Status

Current status
Legend
  • Operational
  • Degraded performance
  • Partial Outage
  • Major Outage
  • Under maintenance
FS#6677 — ACE rbx-s1
Incident Report for Network & Infrastructure
Resolved
Yesterday at night, the ACE 20 card in rbx-s1 has rebooted.
This has happened in transparency for customers since all contexts have switched the card to rbx-s2.

Reboot reason:
last boot reason: NP 1 Failed : NP ME Hung

Since 9am today, all connections were reset.
We are going to switch all contexts on the router rbx-s2 where the phenomenon won't appear and then restart the ACE 20 card in rbx-s1.

Details:

An example : http://178.33.8.65
When our test context is active on the router rbx-s1, the web responds: Connection failed.
After many attempts: It works!


vrack2009# sh conn
conn-id np dir proto vlan source destination state
----------+--+---+-----+----+---------------------+---------------------+------+
2793953 1 in TCP 209 213.186.50.98:35106 178.33.8.65:80 ESTAB
3491280 1 out TCP 2009 172.16.0.1:80 172.31.255.248:1042 ESTAB

A second later, the connection TCP has disappeared:

rbx-s1-ace1/vrack2009# sh conn

total current connections : 2

conn-id np dir proto vlan source destination state
----------+--+---+-----+----+---------------------+---------------------+------+
2457963 2 in ICMP 209 213.186.50.98:43084 178.33.8.65:2048 --
2106058 2 out ICMP 209 178.33.8.65:43084 213.186.50.98:0 --

A new request responds then: Connection failed.

We are starting to switch the context.





Rebooting the card ACE 20 on rbx-s1 is in progress.

1y0w: SP: The PC in slot 2 is shutting down. Please wait ...


02:30:04 GMT: %OIR-SP-6-INSCARD: Card inserted in slot 2, interfaces are now online

The card restarted. The interfaces ft are back one by one.
After the synchronization,we will check if the connections cut-off persists.


It seems that we recovered normal running:

rbx-s1-ace1/vrack2009# sh conn

total current connections : 4

conn-id np dir proto vlan source destination state
----------+--+---+-----+----+---------------------+---------------------+------+
629305 1 in TCP 209 91.121.65.141:54690 178.33.8.65:80 ESTAB
629306 1 out TCP 2009 172.16.0.1:80 172.31.255.248:1039 ESTAB
292046 2 in ICMP 209 213.186.50.98:43084 178.33.8.65:2048 --
292047 2 out ICMP 209 178.33.8.65:43084 213.186.50.98:0 --


The connections remain on ESTAB.

The contexts are back on rbx-s1.
Posted May 05, 2012 - 00:30 UTC