Highly Available Systems

Prior to working in software, I built hardware — audio plumbing at scale.

To effectively present music brands on international, stadium-sized stages, I designed highly-available, fault-tolerant, road-worthy music systems. See gearhead images below.

If you're still reading, you know "the show must go on". Live performance demands high reliability.

Most concerts were at outdoor venues. Weather varied from equatorial humidity to Moscovian snow. For gear that required climate control, our department packed a 3 ton AC unit mounted on wheels. We did not rely on power from the local grid. Think "camping with a mainframe" except with a lot more people.

My team's determined engineering efforts were mostly bounded by the length of time allocated for pre-production. Tuning continued in the field. For what could not be automated, we developed and rehearsed system-failure response procedures. Of course, we did not foresee everything we encountered.

Learning Apache Kafka reminds me of Synapse MSB16/20 MIDI switcher in the background photo of this page.

  • My "fifth Beatle" perch in Warsaw with Stevie Wonder
    My "fifth Beatle" perch in Warsaw with Stevie Wonder
  • Luddites were openly hostile to anything that required booting.
    Luddites were openly hostile to anything that required booting.
  • Rain likely.
    Rain likely.
  • Discussing impending launch with Yuko.
    Discussing impending launch with Yuko.