Hello! It is time for my triumphant return to blog writing and Lain.la development. My usual cadence of writing at least one article per month was severely disrupted this year, but for good reasons. Much has happened behind the scenes that I would consider to be just as valid "work" as anything else, most of it was just generally intangible. However, we can try to bring it to reality. I have many pictures, and many more things to share than a usual blog post. Let's get it all out on (virtual) paper! This post's going to be a big one.
-
Section A: The Move
-
Section B: The Datacenter
-
Section C: The Future
The Move
Moving to another state is a project in its own right. For the record, I moved from New Jersey, my old home, to a state that the Blues Brothers may or may not have had a hand in making just a little bit more famous. It is gorgeous out here and my hasty decision hasn't let me down one bit, but the trip was the hardest thing I've ever had to accomplish, leaving my home behind after over 25 years.
Uprooting my entire life, the house that I worked so hard to build for myself (and you, my users), and leave behind friends and family was, surprisingly, not that hard of a decision, and I haven't quite felt the sting of homesickness. Maybe sitting in the same place, stagnating for that long, makes anything new drown out any nostalgia for the old. I have no regrets, and cherish my new lease on life as an opportunity few ever get the chance to seize. I have Jade to thank for that.
The logistics of moving were shockingly difficult. We had basically worked it out that from the day of signing the lease at the start of March (a challenge in its own right), that we had to move everything critical in one fell swoop. Two trips would be much too difficult to coordinate, and I needed to work on selling my house immediately after moving, as paying rent AND a mortgage would be a serious financial drain. I had given myself about three weeks to prepare by setting the downtime date of March 23rd, and the challenge of packing my entire life into cardboard boxes would proceed through those three weeks. Many things I had to choose whether or not it was worth lugging it across the United States. I had to rent a dumpster with how much crap I needed to get rid of that didn't make the cut. Seriously. That's the dumpster below.
Once the arduous process of packing approximately one room a day was complete, and all my junk thrown away into the shed for dumping into the aforementioned dumpster at a later date, I called in Jade and some backup for the actual move day, and rented the biggest U-Haul I could get my non-CDL-license-holding hands on, a 26 foot Ford F-650 or something. Fate would smile upon me that day, as my truck was brand spanking new. Like 100 miles on the odometer new. See below (this is the same image I used for the 5xx pages during the downtime):
I parked the U-Haul here to load everything EXCEPT Lain.la's critical systems the day before. I had everything stripped down to the bare minimum needed to run all of my services by the night of the 22nd. This is what it looked like just twelve hours before we rolled out.
I drove that thing something like 800 miles, with the first few hundred in pouring rain. Yes, some of my personal stuff broke, but nothing of consequence. All the servers and systems made it (obviously as everything is online). I think I got in at about midnight. After offloading the important things like the freezer and remaining server gear, I spent quite a few hours racking things with my friend. He eventually ran out of energy, and I started bringing up systems myself at about 4am. This is what the rack looked like after I had breathed life into Lain.la once again. I was just getting routing and switching going at the time of this photo.
This next picture is how Lain.la stayed for a few weeks as I focused on unpacking and cooling off after a monster 26 hour day. I didn't go to bed until 6am.
The Datacenter
I am perfectly content calling this Lain.la v3.0. The upgrades and improvements I've made are really awesome. Look at it! We've got the battery system in place (with some upgraded wiring), cameras going, tons of (custom) cooling and dehumidification, temperature monitoring, THREE 20A circuits (I put two in myself, teehee), the works!
Bonus: My workstation lives on!
The Future
So, you've seen everything that I've accomplished over the past two months (minus the fact that I killed Invidious in favor of Piped. Now you know. Check out https://piped.lain.la) Where do we go from here? Dear reader, I have many plans in the pipeline. With the sale of the old forest datacenter, I will have sufficient cash flow to sustain Lain.la's growth for quite some time. Also, the power bills are cheaper out here, so the power cost calculation should drop once I get enough data to figure out by how much and update the budget article with the cost breakdowns. Here are some plans on the docket:
- VMWare was recently acquired by Broadcom. This spells bad news for my cluster. I have purchased two R630 servers with 28 cores and 384GB of RAM each to test out Proxmox. If it goes well, I will migrate to it and then also kill off Veeam because it doesn't support Proxmox. I'll roll my own backup solution, perhaps with Proxmox Backup Server. Then the old hosts will be added to the cluster, giving me 96 cores and about 1.25 TB of RAM. This will be a massive, multi-month undertaking, but will kill the last shred of proprietary software in my stack while increasing my available capacity by nearly 3x. Fuck Broadcom, by the way.
- Yes, I need to clean up the wiring in the rear of the rack.
- The next Lain.la service will be XMPP.
- My free hosting offer is back on the table again!
- Eventually I will run out of backup or primary storage. That may change this year...
2024 has already been one of the most insane and productive years of my life and the course of Lain.la's history. Just the fact that I moved all my possessions to another state AND all of Lain.la AND managed to keep downtime to approximately one day was a herculean feat. I dare you find someone else out there that will literally pack all of their services into a truck and drive them halfway across the country and power it all back on free of charge. My dedication to my craft has always been the defining feature of Lain.la. I may run relatively cookie-cutter and uninteresting services, but it's the way I run them, at the scale they currently are, at no cost, that I feel is unique.
With love,
-7666