Pomf is getting stupidly popular these days. Storage usage is growing exponentially (don't worry I'm not deleting any files), bandwidth is through the roof (250TB a month and counting), and more and more sites are using it as a resilient hosting service for whatever it is they need hosting for. At this rate, we might eclipse Catbox. 

On November 29th, 2021 at 12:51pm EST, The lain.la datacenter (ha.) suffered a total power outage after a brief moment of voltage fluctuations on the main grid. This had been the first time since I started lain.la that a real, extended power outage had occurred, and I got to do a generator cutover and learn many lessons along the way. The incident ended at 1:53pm EST when all systems were restored.

So if you've ever visited my actual homepage (https://lain.la) there was always a big fat UptimeRobot icon at the bottom where you could call me a shitty sysadmin by staring at my uptime stats. UptimeRobot is a pretty cool service and I had been freeloading for the past year on their free version which, admittedly, is VERY good.

So I got pro recently. They had a sale. It was like $5 a month. No brainer. Here's what I got for going pro:

What a cute bird. I hope nothing ever happens to it.

I've grappled with the thought recently that my peers may look down on me due to some proprietary software running in Lain.la's stack. The purists among you may completely discredit my infrastructure because of non-free software. Usually I pay these people no mind, because their fanaticism isolates precisely those that they wish to convert, but I needed to justify to myself and others exactly why I didn't use entirely FOSS for this project and why I disappointed ol' RMS.

Hello again! It has been a while since I wrote a new article, only because most things have been stable. We've been through another maintenance cycle (which is really just patching and certificate rotation these days) and I've documented my procedures and setup processes more so than ever. Here's some highlights of the updates:

This article is partially to remind myself what the heck I have made myself responsible for but also list out all the nice things I do. Please see the homepage for links to these services where applicable.

Class A Services:

(Note: Class A Services are ones I take extra good care of. These are expected to stay afloat with minimal or no downtime, and high standards of performance.)

~

A malicious actor, a nuisance masquerading as a public service, and a braindead web hosting company walk into a strange bar.

Stop me if you've heard this one before.

I've gotten this question a couple times, about how much it costs to run all this. The cost may seem high, but remember, it won't go higher than this since just about every item on my infrastructure wish list is complete, including most of the absurd redundancy stuff. I don't need more nodes, more servers, anything really. So this should be the cap for a long time. EDIT: It always gets more expensive because I'm a maniac. I do keep this list up to date if anything changes.

This article will be an absolute monster, so here are jump links to each section in case you want specific sections: