Get webhook notifications whenever Network & Infrastructure creates an incident, updates an incident, resolves an incident or changes a component status.
Date: 2017-09-14 08:54:37 UTC At 06:07 UTC:
Everything seems stable since the downgrade an hour ago.
Date: 2017-09-14 08:53:34 UTC At 05:10 UTC:
We downgraded the version.
The n7 seems stable now.
The two cards (Modules 3 and 7) booted well.
We isolate the peers while monitoring the nexus.
Date: 2017-09-14 08:52:22 UTC 04:46 UTC:
We isolate all the peer present on fra-1-n7.
Date: 2017-09-14 08:51:28 UTC At 04:39 UTC:
We decide to downgrade the version.
Date: 2017-09-14 08:50:36 UTC At 04:23 UTC:
The card just crashed again.
We cut the module 3 at this time.
We analyze.
Date: 2017-09-14 08:49:28 UTC At 04:14 UTC (14/09/2017):
The new card in Module 3 has been reload twice in a row between 4:00 am and 4:10 am UTC.
We analyze.
Date: 2017-09-14 08:43:31 UTC At 13:20 UTC:
All peers and links between DCs have been restored.
We will plan a maintenance to restore the 4 links to fra-1-a9.
We are still in analysis with our supplier to understand everything that has happened.
Date: 2017-09-14 08:40:44 UTC At 13:04 UTC:
After investigation with our supplier, we will leave the card on the other module for now.
By reconfiguring the ports of the new module, we had a problem in configuring the port-channel to fra-1-a9.
We will leave these ports isolated for the moment (no saturation problem, there are 11 other 100G links that take charge).
We are putting back into production the optical links between DC and peer.
Date: 2017-09-14 08:38:41 UTC At 10:30 UTC:
The card boots well on another slot.
We are currently analyzing with our supplier.
For the moment, we have not reinserted the optics.
Date: 2017-09-14 08:37:14 UTC At 10:20 UTC:
The removal / insertion in the same module again crashed the card.
We move it to another module of the n7.
Date: 2017-09-14 08:35:11 UTC The card crashed rather quickly after being inserted (10:20 UTC).
We sent technicians on site to move the crad.
Date: 2017-09-14 08:34:53 UTC Done at 07:16 UTC (13/09/2017).
Date: 2017-09-13 07:06:01 UTC We are changing the linecard