Incident Report: 20200304: Difference between revisions
No edit summary |
|||
| Line 50: | Line 50: | ||
= Timeline = | = Timeline = | ||
[https://docs.google.com/spreadsheets/d/1P61vGuJai7zTar07iOwK8H1sNvb5J2WGZ78o6hJRBiA/edit#gid=0 All logs] | [https://docs.google.com/spreadsheets/d/1P61vGuJai7zTar07iOwK8H1sNvb5J2WGZ78o6hJRBiA/edit#gid=0 All logs] | ||
19:00-21:00 MS and MP investigating CPU issues on Dolby. | 19:00-21:00 MS and MP investigating CPU issues on Dolby. | ||
| Line 69: | Line 69: | ||
21:29-21:32 The AM goes to silence with occasional bursts of techno. At the same time, Dante Controller reports '''various clock switches and devices muting''' (which is Dante-speak for “oman i am no good with audio pls to halp”). The clock master cycles between Phil, Office, and Studio Red several times. MP says in Slack at 21:31 “[Dante] is not in a good place right now.” | 21:29-21:32 The AM goes to silence with occasional bursts of techno. At the same time, Dante Controller reports '''various clock switches and devices muting''' (which is Dante-speak for “oman i am no good with audio pls to halp”). The clock master cycles between Phil, Office, and Studio Red several times. MP says in Slack at 21:31 “[Dante] is not in a good place right now.” | ||
: Right as the clocking storm starts, the switch starts reporting <code>IFNET Error LINK_UPDOWN GigabitEthernet1/0/11 link status is DOWN.</code> - port 11 is uryStores | |||
: Relevant log entry: 2020-03-04 21:30:53 GMT 1583357453638 Information "uryStores" "Timed out 3 times sending message 'UpdateRxChannels' to uryStores, giving up." | : Relevant log entry: 2020-03-04 21:30:53 GMT 1583357453638 Information "uryStores" "Timed out 3 times sending message 'UpdateRxChannels' to uryStores, giving up." | ||