17 Aug 2017
19:04 UTC
Tom Hill
Resolving this again: the fan tray fans weren't running at high RPM, it was just being reported incorrectly. The process restart (which monitors the environment) seems to have fixed the false RPM reports.
We'll continue to monitor this along with Cisco TAC, and will re-open again if there are any further updates.
Tom
17 Aug 2017
17:38 UTC
Tom Hill
Hello,
We have now been advised to remove & re-insert the new fan tray, and to restart an operating system process thereafter.
As before, these actions can be carried out whilst the device is online, with no expected interruption to service.
Due to the outside chance of some unexpected behaviour impacting service, please deem this as an at-risk period.
Tom
17 Aug 2017
16:09 UTC
Tom Hill
The new fan tray was installed ca. 13:35 - my apologies for not including resolution until now.
The new part has solved half of the problem - all 14 fans are now operational, but they are still running too high - so we're still liaising with Cisco to determine what steps to take next.
As it stands, we're happy that services are not in any immediate danger of interruption. Should there be any further action to take, we'll re-open this issue.
Tom
17 Aug 2017
12:01 UTC
Tom Hill
Hello,
The fan tray in one of our core routers has developed a fault, and requires replacement.
The fan tray can be replaced whilst the router is online, providing that a the new one is installed within 2 minutes. (An official, Cisco-documented process).
We're confident that this is ample time to swap the tray, and thus this work should not pose any risk to customer traffic. We are, however, denoting this as an at risk period, should something unexpected happen.
We expect to complete the work between now and 14:00 (UTC+1), and will update when completed.
Tom