The Lights Go Out in Ohio and Your Homelab Shrugs: Designing for the Graceful Blackout
Photo: Ehrobod01, CC BY-SA 4.0, via Wikimedia Commons
There's a particular sound a UPS makes when the power dies mid-winter: a soft beep, then another, then the low electronic hum of your entire digital life running on borrowed time. If you've heard it, you know. If you haven't, winter is coming, and your grid — yes, yours, the one held together by 1950s substations and optimism — has plans.
In Finland, where I'm told the sun clocks out around 3:40 PM in December, power outages aren't an emergency. They're weather. You light a candle, put the coffee pot on the wood stove, and resume living. Meanwhile, in the US, a blink of the grid sends half of Twitter into a spiral. Your homelab should take after the Finn, not the feed.
Accept That Downtime Is a Design Material
The first mistake is treating uptime as a moral virtue. It isn't. It's a budget, and like any budget, it should be spent where it matters. Your Plex server does not need five nines. Your router might. Your Home Assistant instance that controls the thermostat — okay, that one earns a battery.
Sit down and rank every box in your rack into three tiers:
- Must survive silently: Router, switch, DNS. These should ride through a blip without you ever knowing it happened.
- Must shut down politely: NAS, hypervisors, anything with a filesystem that holds grudges.
- May die dramatically: The experimental Kubernetes node you named after a depressed moose. Let it crash. It builds character.
This ranking is your architecture. Everything else follows from it.
Size Your UPS Like You Mean It
Most people buy a UPS by vibes and price tag. Do it with math instead. Add up the actual wattage of your tier-one and tier-two gear — a kill-a-watt meter costs twenty bucks and will tell you the truth your spec sheets won't. A typical homelab core (router, switch, NAS, one small server) draws 150–250W, not the 800W the marketing photos imply.
Then decide how long you need to ride. You need exactly enough runtime for two things: surviving transient blips (usually under a minute) and running your shutdown scripts (usually under five). Anything beyond that is paying to delay the inevitable. A 1000VA unit with a replaced battery covers most labs for under $150, especially if you buy refurbished and swap in fresh cells — a project with a satisfying thunk-to-screwdriver ratio.
Scripts That Do the Boring Thing Correctly
Graceful shutdown is where the hobbyists separate from the hoarders. Every hypervisor and NAS on tier two should be wired into your UPS via NUT (Network UPS Tools), which is free, ancient, and dependable in the way only old open-source software can be.
The shape of the script is simple: when the UPS reports on battery, wait a grace period. If power hasn't returned, shut down in dependency order — VMs first, then hosts, NAS last so it can catch any final writes. Log everything, because future-you will absolutely want to know what happened at 2:14 AM.
And test it. Pull the plug. On purpose. The first unscripted test will reveal that one VM that ignores shutdown signals, and you'd rather meet it on a Tuesday than during an ice storm.
The Cold Start Problem Nobody Warns You About
Here's the part the blackout guides skip: power coming back is its own hazard. When the grid returns after a storm, it often flickers, dips, and dies again before settling. Your gear booting into that is a machine doing a trust fall with a drunk electrician.
Two fixes. First, configure your BIOS and PDU settings so machines stay off until power is stable — most servers have a "restore previous state" or "last state" option worth setting to "stay off" with a delayed auto-start. Second, stage your boot sequence. Router and switch first, NAS second, compute last. Your filesystems will mount in order instead of in chaos.
A Word About the Emotional Layer
There's a reason the Finnish approach works: it removes the panic. When you know your lab will shut itself down cleanly and boot back up without hand-holding, an outage stops being an emergency and becomes an evening. You light the candle. You read a book. The machines handle themselves.
That's the actual project here. Not the UPS, not the scripts — the calm. Resilience isn't a rack full of redundant hardware; it's a system you trust enough to ignore.
So this winter, before the first ice storm clips a transformer outside Cincinnati or a squirrel in Tulsa does what squirrels do, pull the plug on your own lab. Watch it ride, watch it fold, watch it come back. Then go make coffee on the stove, just to practice.
The grid will fail. Your lab, if you've done this right, will simply yawn, save its work, and wait for the lights — the way any sensible Nordic design would.