Merry Christmas. Lots of big pictures in this article FYI.
Since today is a calm day, I figure I'd go write some stuff down. I am very proud of this place I now lovingly call the "Log Cabin Datacenter". So let's talk about it!
For starters, this is a 3 bed, 2 bath home on about 2.5 acres of land. Nothing crazy. It does have a garage and full basement and a layout that is conducive to "future upgrades". We're not here to talk about the house though, more about the systems that enable Lain.la to scale vertically, infinitely. I'll give you a bonus picture though (and you can always see the pictures on my previous article, A Return to Form).
First, I needed to do some relatively extensive renovations on the place to get it to where I needed to be. While epoxying the garage (In the colors of Nulled, by the way) or gutting a bathroom sound fun, the real meat and potatoes was in the electrical, racks, networking, and fiber.
Electrical:
So, the house started with a largely unlabeled and totally full breaker panel from the 90s, using Challenger breakers that were likely recalled for starting fires. Great. And there was very little available power in the basement where the servers would eventually go. So if I didn't want to burn the house down drawing 20 amps over a single circuit, I needed to upgrade. I called out a local electrician who didn't need to pull permits and happily took wads of cash, and we set to work relabeling the panel, swapping the bad breakers, and adding a subpanel where all the servers would be hung off of. Now we have 4x20A circuits available for all power needs.
"But 7666, why didn't you go 240 volt!!!!" - Would have to replace a lot of intermediate equipment like UPSes, PDUs, etc. at a large expense. Not worth it right now.
But wait! We're not done with this section yet! Look! It's a GENERAC. Remember how I said that would be too expensive back in the old forest datacenter and I put in that battery system? Funny how life works, huh. Called a company out to give this thing a tune-up, verified it worked, ran it a bunch, load tested it at 40 amps, and now I trust it to do the job (minus one UPS that gets up in a tizzy about the frequency being a smidge off). So now I have unlimited backup power!
Racks:
When I say I nearly died for Lain.la, I mean it. I was on the ass end of a 250+ lb rack going down a flight of stairs to get into the basement. Yes, I un-racked all the servers (but left the switches and rails and cables in), and it was STILL the heaviest thing I've ever had staring me down with gravity acting as a cocked pistol. By some miracle and a lot of extra muscle at the top of the stairs, the main rack for Lain.la made it. And then we moved three more down, because why not? So now we've gone from "Under the bed" to "A full rack" to "Four full racks". Each rack is 42U. The original Lain.la one is Rack 4. The remaining ones are new. Quite the journey! Here's the picture from the previous article again, and two bonus ones.
Networking:
So yeah, this was a challenge. There were two layers here - physical networking and logical networking. Let me explain.
From a physical networking standpoint, we ran about 16 CAT6 drops all over the house. 12 inside (for APs, computers, TVs, etc.), 4 outside (for cameras, outdoor WiFi, etc.) I wanted to just go ahead and punch the holes in the walls now and get this place properly networked, especially because I prefer to have my work devices on separate VLANs so they can't talk to any devices on the same LAN (requiring two drops per office). I did all the terminations rack-side into a patch panel while the electrician I hired did the drops. Here's me and a beer.
Now, the logical side. The problem here was that we had multiple racks. No longer could the primary Lain.la router actually also be the router for the whole house, because I didn't want the whole house to have access into that network. So here's a quick and dirty chart of what we decided instead (below). In essence, Lain.la's network is entirely encapsulated in Rack 4, with an uplink to the core router in Rack 1. Some CAT6 drops are patched over to Rack 4's core switch so I can access Lain.la directly from devices like my computer, but the house stuff and majority of CAT6 runs stay in Rack 1. For anything in Rack 4 needing to talk to Rack 1, a static route is placed and works because we used non overlapping RFC 1918 addresses. Lain.la is 192.168.1.0/24 (with subnets below it for VMs like 10.0.0.0/24 and 10.0.1.0/24 as before) and the home network is 10.1.1.0/24. So we have bidirectional communication between everything with firewalling in place at every layer to control access where needed in between.
Fiber:
Yeah there's no fiber out here. At all. "So Mr. 7666, how did you get fiber?"
Yeah I paid out the ass to have this happen.
Yep. I got an enterprise line trenched to my house. The same type a business or datacenter gets. Completely dedicated line. Even the fiber on the pole is just for me. They had to string it on the power lines. It costs an arm and a leg every month, but it's there, and it works. I have 500 mbit symmetrical, the fastest internet in the entire town. Just so I can shitpost and let people upload total garbage to Pomf. I sure hope people appreciate it. I have a handful of static IPs, and with this level of service I can do BGP or expand it later for an additional sum of arms and legs.
Do you see why I was gone for so long? This was a project of ridiculous proportions, far beyond anything I have ever done before. Now I have lots more plans to go through, like the solar project, doing BGP out of here with public /24s, racking Nulled customer equipment (at half price!) down here, so on and so forth. This place isn't done. No way. I'm just getting started.