It's been a while since I've done one of these! Hello again, dear metrics nerds. There's been a lot since November's last metrics update that I'll run through, and then all the near-term future projects I'll go over:
Updates:
Historical updates are as follows:
- In December, Pomf got a new LVM caching hierarchy and new slice based caching system, vastly improving speed across the board.
- Also in December, lain.la got a business fiber connection, a big upgrade from the residential line. Speed is still symmetrical gigabit. I now have an account manager though!
- I built that electrical deathtrap of a UPS that sits in my crawlspace in January, providing power to all of lain.la.
- I built a Pleroma instance in January, bringing lain.la onto the Fediverse. It's open registration, so anyone can join.
- The Redundant LAN project was completed in February, giving every server two paths per virtual switch to all resources.
- This month, I replaced the RAID card battery and swapped the CPUs for newer, lower power CPUs in the primary storage server.
- HSTS Preloading has now been enabled across all of lain.la.
- Emailed temperature alarms have been implemented via the intake temperature sensors on the iDRAC management card in my servers.
- At the end of the 1 year agreement on my GreenCloud Hentai VMs, they will be deleted. 95% of my contribution to the H@H project will close. This will occur around November.
And now for the stuff in the pipeline:
We have a new server coming to the lain.la family! I'd like to introduce this PowerEdge R340, the newest server in my fleet (Seriously. It was manufactured in 2021!)
One of the least reliable portions of lain.la right now is my Seedbox, which is used to seed almost 2,000 torrents. These torrents range from files my friends upload to important research documents. With the news of the Internet Archive losing its recent copyright lawsuit (article), I have wanted to pitch in, just in case the long arm of capitalism knocks out one of the world's most valuable internet resources.
However, the seedbox as it stands is pretty much at capacity in that the thing falls over if you so much as breathe on it. It currently hammers my storage server which is not equipped to handle many in-order random reads that seeding 11TB of data simultaneously can cause, and it appears that single threaded performance is paramount for rtorrent to function properly. This server, being very new, has some very high frequency CPUs available, and the one I have will hit 4.5GHz, a full 900MHz over what ESXi3 can do, while being three generations newer.
The R340 will be equipped with 4x16TB HDDs in RAID5, 64GB of DDR4 RAM, and a Xeon E-2124G. Redundant PSUs, 4x1Gb ethernet, etc. I will move the seedbox data over to it, and begin splitting rtorrent into four separate instances, each with 1 core and 16GB of RAM. This is the poor man's way to multi-thread a single threaded program, but it's effective. My hope is to be able to handle over 40TB of active content across 5,000 torrents or more, thus helping preserve the internet's data. This process will take a few weeks at least to accomplish, as I have to determine some scheme to splitting the data across these new virtual instances. The total cost for this server is around $1,625, which is a steal for this kind of firepower in my opinion. Oh, and it will get its own gateway too. Slice6.
Other news - I have added these services to my "implementation" roadmap:
- IPFS
- CryptPad
- XMPP
- PeerTube
- Modded Halo
As I'm just one person, don't expect these for some time.
Metrics:
- Cluster stats:
- VM Count: 31 on-prem, 12 hentai nodes, 7 endpoints, for a total of 50 VMs.
- Edge Bandwidth: Good lord. New record, again, by far. One of the four endpoints graphs is below. After totaling up all the data across everything, the bandwidth total for inbound and outbound in the last 30 days is: 907.32TB. We're almost at a fucking petabyte. Francisco's gonna kill me.
Internal bandwidth (between me and my ISP) rose a bit as well, mostly from seedbox related activity. Even if it is quite broken, it still works 95% of the time, pushing quite a lot of data (~18TB by my count last month). With my business connection, however, this is no longer a ToS violation, hehe.
- Costs: In the last article, lain.la's costs were $422.50/mo. This is now $519.00/mo. Most of this is in upgraded edge nodes to handle all this bandwidth.
- Uptime: We hit 4 or 5 9's across almost the entire board! Pomf even got a 100% uptime rating for the past 90 days! It's so green here now. See for yourself.
So, that's all for now! As always, feel free to email me at 7666@lain.la to say hello or if you have questions or suggestions.