Incident Report: 20170225 - Revision history

10418 at 18:27, 14 November 2018

2018-11-14T18:27:59Z

← Older revision		Revision as of 19:27, 14 November 2018
Line 31:		Line 31:

	[[Category:Incident Reports]]		[[Category:Incident Reports]]


	~~[[Incident Report:_20181114]]~~

10418: /* Work Required */

2018-11-14T18:22:56Z

Work Required

← Older revision		Revision as of 19:22, 14 November 2018
Line 31:		Line 31:

	[[Category:Incident Reports]]		[[Category:Incident Reports]]


			[[Incident Report:_20181114]]

8743 at 11:05, 27 February 2017

2017-02-27T11:05:03Z

← Older revision		Revision as of 12:05, 27 February 2017
Line 24:		Line 24:

	== Causes ==		== Causes ==
	Output failing to switch to OB lead to some bad decisions. Original cause ~~as yet unclear~~ and ~~investigations ongoing~~.		Output failing to switch to OB lead to some bad decisions. Original cause was due to the "OB listener" being "moved" to the wrong stream earlier and the audio path getting unconfigured.

	== Work Required ==		== Work Required ==

8743 at 01:14, 27 February 2017

2017-02-27T01:14:00Z

← Older revision		Revision as of 02:14, 27 February 2017
Line 19:		Line 19:
	At 19:01, GodEmperor Anthony was phoned.		At 19:01, GodEmperor Anthony was phoned.

	At 19:08, the stream was brought back up (by running `startAudio.sh`), and all was well again.		At 19:08, the stream was brought back up (by running `startAudio.sh`), and all was well again. (Total dead air: 18:53:39 - 19:09:00)

	At 19:22, output was successfully switched to OB (they were running late anyway), which then proceeded without incident.		At 19:22, output was successfully switched to OB (they were running late anyway), which then proceeded without incident.

8743: Created page with "{{Incident |brief=Streaming services were restarted, then failed to come back up |severity=Critical |impact=High (Dead air for around 15 minutes) |start=25/02/2017 18:..."

2017-02-26T13:44:35Z

Created page with "{{Incident |brief=Streaming services were restarted, then failed to come back up |severity=Critical |impact=High (Dead air for around 15 minutes) |start=25/02/2017 18:..."

New page

{{Incident
|brief=Streaming services were restarted, then failed to come back up
|severity=Critical
|impact=High (Dead air for around 15 minutes)
|start=25/02/2017 18:45
|end=25/02/2017 19:10
|mitigation=Don't restart streams in termtime.
|leader=[[Charles Pigott]] <charles.pigott@ury.org.uk>
|others=[[Chris Taylor]] <christhebaron@ury.org.uk>
[[Brooke Hatton]] <brooke@ury.org.uk>

[[Anthony Williams]] <anthony@ury.org.uk>
}}

During setup for the OB for ERN2017, it was found that the OB line was not actually getting selected when switched to, only continuing to broadcast jukebox. This was suspected to be because the jack audio inputs were not being mapped to the correct output, but qjackctl on dolby was unable to be accessed to confirm this (as it requires horrible X-forwarding to an office PC and the process for which is unclear). A possible cause for this is when listeners to the `/OB-Line` stream were "moved" within icecast to `/jukebox` at an earlier point in the week while testing.

At 18:54, having run out of further options and running out of time (the OB was scheduled to start at 19:00), both liquidsoap and icecast services were restarted (until that point had been running solidly since September) which then failed to come back properly. At this point station output went offline.

At 19:01, GodEmperor Anthony was phoned.

At 19:08, the stream was brought back up (by running `startAudio.sh`), and all was well again.

At 19:22, output was successfully switched to OB (they were running late anyway), which then proceeded without incident.

== Causes ==
Output failing to switch to OB lead to some bad decisions. Original cause as yet unclear and investigations ongoing.

== Work Required ==
Investigate a better method of accessing jack config (jack_lsb?)
Clear up the process for restarting the streaming services, and the possible (likely) results of doing so.

[[Category:Incident Reports]]