URY Wiki - User contributions [en-gb]

Christmas Endeavours

2016-10-18T22:03:17Z

7746:

Every year, towards the end of the Autumn Term, URY has this awful habit of doing two "festive" "things".

== URY Christmas Single ==
URY's Committee, Presenters and general happeners all get together on a Saturday (or other convenient time), to record a cute original music video for a popular song. Of course, this is then followed up with a studio recording of our wonderful voices singing - I can assure you it's an experience.

Usually, these songs are covers of traditional Christmas tunes. Although sometimes they will be parodies relevant to the University of York, or even completely original compositions. As of 2013, all of these are available in URY's Central Music Library.

{| class="wikitable mw-collapsible"
! Year
! Song
! Notes
|-
| 2015
| All I Want for Christmas is You
| Cover of Mariah Carey's Christmas hit
|-
| 2014
| Christmas is All Around
| Recreating the opening song to the movie ''Love Actually'' Which itself is a cover of the song "''Love is All Around''" but this has more christmas.[https://www.youtube.com/watch?v=kWiOggaWkZU]

Cast (in approximate order of appearance): [[Andy Durant]], [[Alex Light]], [[Harry Whittaker]], ??, [[Jay Glover]], [[Adam Brain]], [[Joe Willis]], [[Naomi Gildert]], [[Mat Carrington-Mackenzie]], ??
|-
| 2013
| Fairytale of New York
| A nice traditional outright cover. Entirely produced by URY. [http://youtu.be/o22F27cOpGg]
|-
| 2013
| This is Hes East
| Part of the 2013 Christmas Panto. Yes, we know the panto version doesn't normally do "A Whole New World". [http://youtu.be/qEO9mYaIuTg]
|-
| 2012
| Merry Christmas Everyone
| There are rumours an alternate version consisting entirely of the URY Computing Team exists, but was never allowed to see daylight. Entirely produced by URY. [http://www.youtube.com/watch?v=b6m1Y0Uu6DE]

Cast (In approximate order of appearance): [[Rob Stoneduck]], [[Rachael Venables]], [[Ben Bason]], [[Will Chalk]], [[Kirsty Rogers]], [[Andy Durant]], [[Tom Haines]], [[Robyn Haycock]], [[Wil Bennett]], [[Stephen Clarke]], [[Tom Edwards]], [[Anthony Williams]], [[Matt Windsor]], [[Ellie Wright]], [[John Wakefield]], [[Rebecca Elson]], [[Charles Lewis]], [[Will Edwards]], [[Steffan Woodruff]], [[Lloyd Wallis]], [[Karl Bos]], [[George Lane]], [[Hannah Smith]]
|-
| 2011
| Black Christmas
| A parody of Rebecca Black's Friday. Due to lack of forward planning, was performed live on air by whoever was around at the time. Failed miserably.
|-
| 2010
| I Wish It Could Be Christmas Everyday
| Outright cover. The music video was produced by YSTV. We don't talk about it. [https://ystv.co.uk/watch/URYXmas/URYXmas2010/]
|-
| 2010?
| I Go To York Uni, It's True
| Little is known of this parody of All I Want For Christmas Is You - not even the source of the vocals. However, we can learn a valuable lesson - the only thing that has changed ever since is that you can't get olives in YUSU Bars anymore.
|-
| 2009
| Do They Know It's Christmas?
| The older tech we had back then is quite obvious. [https://ystv.co.uk/watch/URYXmas/URY-Christmas-single-2009/]
|-
| 2008
| Christmas On My Face (A Terrible Case of Xmas)
| This is original. Original! And also The Best Thing! [http://www.youtube.com/watch?v=OVICvXorh-s]
|}

[[Category: URY History]]

== Christmas Panto ==
Of course, what radio station would be completely without a Christmas Panto? Featuring a "Celebrity" guest, each year we put our own spin on popular pantomimes.

{| class="wikitable mw-collapsible"
! Year
! Production
! Guest Star
! Notes
|-
| 2015
| Oliver Twist
| David Benson Phillips
|
|-
| 2014
| Dick Whittington and His Cat [https://www.mixcloud.com/harrywhittaker2/the-ury-panto-2014-dick-whittington-his-cat-feat-cbbcs-raven/]
| CBBC's Raven
|
|-
| 2013
| Aladdin [https://ury.org.uk/uryplayer/podcasts/851/]
| Jon Lee (off of S Club 7)
| Someone talk about this.
|-
| 2012
| Rumplestiltskin [https://ury.org.uk/uryplayer/podcasts/723/]
| Alan Fletcher (Dr. Karl Raymond Marx Kennedy)
|
|}

First Generation Computing Loggers

2016-07-19T12:19:29Z

7746:

The First Generation Computing Loggers, more formally known back in their prime as logger1.ury.york.ac.uk and logger2.ury.york.ac.uk, are two servers known as achieving amazing feats during their time in production, the most notable of which being that they were first put into use in the 1998-1999 (?) academic year by a team including [[Jon Dye]], [[Simon Hildrew]], [[Mat Fortune]], [[Alan Raison]] and [[Gavin Atkinson]].

== Pre-Computing Loggers ==
Before we implemented a digital log recording system using hard disks, all our output was logged using VHS tapes that were changed every 12 hours by a dedicated committee position. The name of this position is currently difficult to retrieve due to it being before the days of digital committee management, but it is well known that [[James Brookes]] held the position for some time.

Multiple boxes of these tapes still exist in URY's [[stores]], although playback of any of these has not been attempted in living history. With the increasing proliferation of "Computers" in everyday life, it was decided that maybe there was a way of automating this historic task.

== Introducing logger[12] ==
When specifying the hardware specification of the loggers, with advice from [[Tom Whitehouse]] (serious, try and find a wiki page *without* his name on it), two systems were build with similar overall power, but with hardware sources as different as possible to reduce the possibility of a design flaw in one causing a failure in another. Word has it that these were built was cheaply as possible, hence the use of AT-style motherboards and cases (complete with DIN keyboard port) in the late 90's.

=== Specifications of logger1 ===
* Operating System: FreeBSD 5.2-RC (Installed 2003)
* Processor: AMD-K6 3D (350.80MHz)
* Cores: 1
* Architecture: i386
* Memory: 27MB available (32MB installed)
* PSU: xxW AT power supply
* HDD: 40GB (8379 Power-On Hours)

''Last known updating advice for this server: "Don't update this server. It works just fine."''

=== Specifications of logger2 ===
* Operating System: Debian 2.4.18-1-k6 (Installed 2004)
* Processor: AMD-K6 3D (350.80MHz)
* Cores: 1
* Architecture: i386
* Memory: 29.8MB available (32MB installed)
* PSU: xxW AT power supply
* HDD: 40GB

''Last known updating advice for this server: "Don't update this server. It works just fine."''

== The Second Coming of Digital Logging ==
In Summer 2012, a plan was devised to start decommissioning these then 12-year-old devices. Due to the need for something that was provably highly stable (at least 42 days continuous service runtime), it took the next two academic years to reach a point where the replacement devices, uryred and uryblue (see [[Second Generation Computing Loggers]]), were considered stable enough to prepare turning off loggers 1 and 2.

A date was set for Friday, Week 10 of the Summer Term 2013/4, as it was the end of an academic year and also team veteran [[Lloyd Wallis]]'s last day in York (and birthday). It would be a ceremonial event where the loggers would be considered no longer mission-critical to our operations. However, as this was a short time before many components reached 10 years uptime, it was decided that they would likely be kept running (albeit not necessarily doing much) until that milestone was reached.

On Wednesday, Week 10 (25/06/2014), it was discovered that logger1 was not as healthy as was hoped - his PSU fan had failed and he was running hot. As a result, the decommissioning was brought forward to that evening.

Amusingly, logger2 then outlived one of the replacements when uryblue's motherboard packed up without warning in Spring 2015. After some arguing with ebuyer about the length of warranty, some partial refund was acquired, and in true logger style, the cheapest possible motherboard was acquired to repair uryblue.

Finally, on Friday 15 July 2016, logger2 was halted for the last time by [[Anthony Williams]] and [[Andy Durant]], in full academic gownery for it was their graduation. It had been suffering from a failing network card for some weeks prior to this and it was finally decided to lay it to rest rather than repair. logger2 was then delivered to [[Gavin Atkinson]] in his office.

== Trivia ==
* In Summer 2011/12, an eighth birthday cake was made for logger2's disk installation date. It was enjoyed at a station meeting, and a poem was written in a birthday card for him. Someone should transcribe this.

[[Category:Computing]]

Incident Report: 20131201

2013-12-03T22:15:40Z

7746:

'''''The below email was a post-mortem of a Campus North power outage at 01:42am, Sunday, 1st December 2013. Assistant Head of Computing Anthony Williams and Live-Next-To-The-Station-Man Lloyd Wallis were on site to respond. This is how it went.'''''

Hey everyone,

'''tl;dr: There was a power cut. We turned everything off and back on again. It’s fine now.'''

As promised, this is a full breakdown of how the power outage on Sunday affected our systems and how we responded. You know, so we can learn from it and do better next time.

This outage came as a surprise, but a well overdue one. Usually, we expect to experience at a rate of about once a term, and we went the entirety of the last academic year without any environment issues. Naturally, this started shortly after a conversation about how we hadn’t had a power cut for some time.

== Event Start: 01:42 ==
At first, we didn’t think there was a substantial issue - the power in James college was knocked out for about a second, but their UPS coped so we assumed ours would too. Some amusement was poked at YSTV for neglecting to put their web/db server on said UPS. Once we confirmed our website was inaccessible, we made our way over to investigate, but assumed bits of Vanbrugh network were just rebooting.

We reached URY in a state of running on UPS power, so started a graceful shutdown of all servers using ACPI (i.e. the power button). We left logger2 running at this stage because of the uptime, and logger2 is a hardy-ish device that’s seen a lot anyway. Everything but that was powered down by 01:52.

== UPS Failure: 02:13 ==
At this point, logger2 had exhausted the UPS and was powered off. At this point, the transmitter was also powered off as we would not be logging when power is restored. Security have confirmed all emergency staff have been called in, but the interesting arrangement of areas without power meant it would likely be a while.

The patch bay UPS running jukebox and the switches lasted a bit longer but turned off a little while after. Of note is that for the majority of the time, both UPS units were on ‘1 flashing bar’ of power, suggesting they needed calibrating again.

We spend the time wandering around campus in the dark and drinking hot chocolate in YSTV, living off the few drips of Internet Lloyd’s 3G tethering could provide, and the rare view of stars due to the reduced light pollution.

== Power Restored: 05:42 ==
We notice a flicker from YSTV’s lights. We have a wander over and see power is back. We also learn the method of ‘looking out of YSTV’s window to see if the physics bridge is lit’ doesn’t work well. We rejoice.

== Problem #1: Stores Distribution Board ==
The B16 breaker that estates told me they would upgrade to C16 last year tripped from the sudden desire of a large quantity of batteries to charge up. Turning the power back on caused the breaker to trip again.

Luckily, switching off the compressors and distribution amplifier for the audio path was sufficient, and the UPS then began to charge at 05:44, and enabled server power at 05:49.

== Problem #2: logger2 ==
As is traditional, logger2 decided not to play ball when we powered it up. It made strange noises, so we left it off overnight. Confirmation that logger1 and uryblue were operating left us happy to power the transmitter on by 06:30. It did start up later in the day, so was probably just being fussy because it was a bit cold.

== Problem #3: IT Services Systems ==
Other servers experienced delays coming up due to reliance on the campus DNS and NTP servers, but once running worked as expected. Core IT Services Systems came up automatically shortly after power was restored.

== Problem #4: Music Store Mounting ==
Partly Lloyd’s fault from playing with drbd way back when, /etc/fstab refused to agree with the fact that /dev/sdb1 is an ext4 filesystem and would not mount. Mounting it manually as ext4 worked. This is a manual step that needs removing, however.

At this stage, we verified that essential services - the digital audio path, member web services, loggers and BAPS - were all working sufficiently, and declared our state of emergency at an end.

The only remaining immediate issue at hand was our Outside Broadcast scheduled to start at 10am. A contingency plan was devised and communicated in the event that IT Services had not restored NAS by this time. Unfortunately, the on-site engineers for the OB assumed that the wired network was still unavailable because the OB machines did not detect a network connection, when, in fact, they had used the wrong network socket.

'''''Event End: 07:11'''''

== Other Points of Note ==
After the initial restoration of services, the following days raised several issues that required manual intervention to fully restore service:
* When the power failed in the IT Services Berrick Saul Datacenter, our Virtual Machine, uryrrod, was automatically powered on in the TFTA Datacenter. However, due to it having no network access and an NFS mount in its fstab, the server was found to be sitting in Single-User Mode waiting for administrator intervention when it was next checked on Sunday evening.
* The Stores UPS (the main backup supply for our servers), has now successfully acknowledged that it has 6 minutes of runtime, not 3-4 (of course, we know from the experience of this event it is easily 15).
* When the scheduled backup processes ran in the early hours of Monday morning, the backup system failed to detect that the network mountpoints were not set up on uryfs1. The system started to back up to its own root filesystem until all space was consumed, at which point services started to fail overnight. Luckily, Icecast was not an affected service, and successfully fell back to a static jukebox feed when it detected that Liquidsoap had stopped responding. This issue was not noticed until mid-afternoon on Monday when a technical member entered the station to see the Studio Clock reporting that the server was out of space (it turns out the studio clock does that now).
* The ZFS pools on urybackup0 were not automatically mounted on boot. This was also not noticed until attempts to resolve the uryfs1 backup issue were made. Our primary archive filestore was therefore unavailable to members for most of the day.

== Changes Made ==
* The monitor in the server cupboard has been moved from mains power to UPS power - this is needed to power off the loggers gracefully as they have AT power supplies. The next cable tidyup in that rack should remove the excess mains cable.
* uryfs1 will now detect missing backup mounts and attempt to restore them.
* Remove the need to enter console passwords on boot for ury and uryfs1
** Decrypt the SSL certificates and review permissions on their storage directory

== Changes to be Made ==
* Investigate Stores Distribution Board upgrade
** Re-raise the issue with Estates and YUSU
* Prevent uryfs1 and other servers from backing up if the mount fails to be brought up
** Anyone want to practice their bash-fu?
* Ensure urybackup0 mounts /pool0 and /pool1 on boot
** Currently need to run zfs mount pool0 && zfs mount pool1
* Ensure uryfs1 mounts /music on boot
** Server seems to get its filesystem types confused
* Remember to power uryfw0 up first
** Much spinning waiting on NTP etc otherwise (partly not our fault)
* Add checking backup mounts, uryrrod to our standard boot procedure, plus generally update the document
** https://docs.google.com/document/d/12gdrkNWPqC0hc0sJ1ETqM9TXCwmQy3ZBma1anc8ZO7M/edit
* Review policy for communication on social media during a failure
** Should named Technical persons have access for these scenarios?
* Review policy for calling named persons at 2am (this kind of thing normally happens during the day)
** Would Al have rather been woken up than not know?
* Improve documentation on Outside Broadcast systems and standard contingency plans if 802.1x is unavailable on the campus network
** Known non-NAS/opfa ports, IP addresses available to us
** Give engineering the learnings of ‘how to tell what a port is’?
** We should generally have a how-to guide for the freshers
* Review whether the stores distribution board should be touched (I mean, look at it)
** Talk to Estates/Health and Safety? Last time I did I got a “What the f***” from the electrician
* Investigate automatic shutdown options?
** We now know the switch/jukebox UPS lasts a good length of time so the limiting factor of very short comms runtime (~90s) last time this was considered is no longer a problem.

Hopefully this’ll be of some use in the future.

Lloyd & Anthony

Were-there-when-it-happened Officers

[[Category:Incident Reports]]

Using URY's Filestores

2013-10-13T10:18:26Z

7746: /* Temporary Filestores */

URY Computing maintains several places in which files can be stored, including:

* General Filestore
* Temporary Filestores
** production
** studio2
* Central Music Library
* Managed Resource/Personal Resource Libraries
* Podcast Archive

In October 2013, we changed around how some of this works, to ensure that data is stored in a more logical way and is properly backed up.

== General Filestore ==
On all URY computers, this is available as the S:\ drive. Physically located at \\urybackup0\pool1, this is where any files that you want to keep should be placed, such as finished jingles, images or packages. You should not use this filestore for works in progress, such as Audacity projects.

Information in this filestore is sorted into 4 key areas:
* Shows
* Teams
* People
* Photos

When saving a file, please make sure you put it in the relevant place. Within each of these categories, there is also an Archive folder.

=== Maintenance of this Filestore ===
Computing Team currently maintains a guarantee on this filestore. The server it is stored on can tolerate disk failures and takes nightly snapshots of the data on it.

Deduplication technology is enabled on this filestore to reduce disk usage. The Archive folder for each area is also compressed.

At the end of each Membership Year, Computing Team will move Shows and People that have left into their respective Archive folders. Each Team will have all of the current contents of their folder moved into a subfolder labelled by academic year (e.g. 12-13 for the 2012-2013 academic year), which will then be compressed. Management Team is excepted from this.

== Temporary Filestores ==
production and studio2 (and to a lesser extent, all other computers in the office and studios) have hard drives which should be used when actively editing content locally. Once the work is finished, the final output should be stored on the General Filestore (S Drive).

=== Maintenance of this Filestore ===
Computing Team make '''no promises''' about data stored here. This information is not backed up or otherwise fault tolerant.

At the end of each academic term, Computing Team will delete all data on these disks to promote correct usage of the General Filestore.

== Central Music Library ==
This filestore is where all of URY's PPL licensed audio must be kept. No such audio may be stored '''anywhere else''' on URY's Filestores, as this is illegal. It is secured and access restricted as per the terms of our various licences. To upload resources to this filestore, please use [[Show Planner]].

=== Maintenance of this Filestore ===
Computing Team have rigorous checks in place on this filestore, which is backed up daily. The filestore itself is not tolerant to a failure, however.

The physical space on which this filestore lives is shared with the Managed Resource/Personal Resource Libraries and the Podcast Archive.