Our transmitter is located out in the Apalachicola Forest. The Talquin Electric people have been moving some power lines out there, so the transmitter has apparently been running on generator. The first generator failed. The university loaned us one. That one failed as well. A third generator was rented. THAT one caught fire on Saturday morning.
(Pictures courtesy of various engineers)Of course all this happened on the weekend. Our signal kept going off the air as generators died.
Boy, are we prepared for anything!!!
You probably don't know exactly how you get whatever is on your television channels. In our case, we have several methods of delivery. Programming can come from satellite, from tape, or from server (kind of like a giant, industrial strength DVR). Then there's the stuff that runs in between programs. Promos, special event spots, underwriters and IDs. We house those on a separate, smaller server.
Today, the smaller server crashed. Which meant that we had programming, but nothing to run in between. Luckily, PBS actually has interstitial material between their program feeds, so if we HAD to, we could just go to the network for those three minutes between programs. We did that for a couple of breaks while those of us who push paper and schedule that in-between stuff ran around looking for backup tapes and wondering how it is that such things don't exist for much of our material.
While some of us were running around like headless chickens, Ray (the Chief Engineer) was trying to figure out how to fix the problem. He tried several things and the server just wouldn't reboot. Finally, Leo, another engineer mentioned that this had happened before. Ray asked what he did to fix the problem. Leo showed him. They went back in to Master and Leo whacked the side of the server.
It rebooted. (Thank goodness for those engineering-types. You gotta know just where to hit it!)
Lesson learned. Get some bloody backups made!!!!
At least the satellite didn't fall out of the sky... this time.
Seriously, the station couldn't run without Ray and Leo. They have more between their ears than the average bear. Ray is retiring soon and I don't know how the place will run without him.