Submitted by 7666 on

Hello. I'm 7666. I single-handedly run this massive collection of services known as Lain.la. I started all this a little over three years ago (August 31st, 2020) as a productive outlet for myself, to make systems and services I'd be proud of that were also useful to me and my friends, while keeping privacy and security paramount. As time has gone on, I have purchased large amounts of hardware to install in my house, rented huge collections of virtual servers hosted in multiple clouds, spun up new domains and subdomains, engineered unique security solutions, and expanded my services list beyond my wildest dreams. I like to think I have a reputation now for fast and reliable hosting, whether it be for files, servers, services, etc. Lain.la is used by thousands of people daily across over a dozen countries, all free of charge with no ads or strings attached.

As it stands, Lain.la pushes over 600TB a month of bandwidth from approximately 50 different virtual machines, some at my house, some across the world.  I work diligently to maintain reliability, building capacity where I can. This capacity is not cheap, but I have fronted 100% of the cost for the past two years, whether it be for giant server purchases, or recurring operational expenses. These costs have exceeded $15,000 in total at this point.

I make a habit of not being reliant on anyone. I am perfectly capable of paying for all of this, and I have a personal pride in doing so. However, certain endeavors of mine do get extremely pricey as we scale into uncharted homelab territory. I usually hold off for months at a time on the big purchases because there simply isn't enough cash in the budget to drop, say, $4,000 on hard drives or something. I recently saw this post by my "competitor", Catbox, and was inspired by the turnout they achieved.

So, my proposition to you is this - if you like what I'm doing and want me to keep going down this insane path in self-hosting, then funding some of my crazier ideas is the fastest way to go. The operational costs will continue to be paid by yours truly (unless they grow exponentially for some reason).

If you are interested in donating, I now have a Ko-Fi and a Liberapay. Ko-Fi can be used for one time donations, and Liberapay if you want to provide recurring payments. For more privacy, my crypto addresses are below:

BTC: 32NBSVoQiVBnDGPqwAQLBFnHaKUaPUMTMk
ETH: 0x47Cf194Ea49770668cC365feF2a11Ac7dAbaf080
DOGE: DRpU6mWjWiUwPAW93HLw83XrxYatUPyuHL

You can also contact me via 7666@lain.la to say hello, suggest something, or get a status update.

I also have a BuyVM affiliate link, if you need a VPS on the same platform that Lain.la runs. This funds the edge nodes a little bit for every VM you purchase. https://my.frantech.ca/aff.php?aff=5815

You can send as little or as much as you want. I'll keep it all running either way - but the projects above will move faster with your help! 100% will go right into Lain.la. No avocado toast or Rolexes. Promise.

What are some of those ideas, you may ask? Here's a small sample:

  • Big Bertha (also known as stor1.lain.local) (COMPLETE!). I just bought an R730xd server, formerly decommissioned from AWS, to act as my new storage server for Pomf and other services. By the time it is complete, it will carry over 200TB of raw storage over 10GBe iSCSI via TrueNAS and ZFS. This, however, requires 12x16TB Seagate Exos X16 drives. Each drive is valued at approximately $300. So, finishing Big Bertha will take $3,600. See the picture below for what it looks like!
    • UPDATE October 5th, 2022: I bought 6 of the drives with my own money at a very good price ($1,300)! The other 6 will still need your help.
    • UPDATE October 23rd, 2022: I bit the bullet and bought the rest of the drives, plus a hot spare. $1,250 for 6x16TB.

  • Automatic Backup Generator. One of the biggest risks to Lain.la right now is the lack of redundant power. My home "datacenter" has one power line to it, as do most residential areas. I live in the middle of the woods, where Lain.la is one tree away from a power outage. I have a backup generator that requires me to manually start and run wiring to my UPSes, a 20 minute process. If I am not home, Lain.la is going down. I estimate this project to take approximately $6,000 to install a 20kW propane generator with an automatic transfer switch. I have a 500 gallon propane tank already on site which would provide multiple days of runtime in the event of a serious emergency.
    • UPDATE October 5th, 2022: This is planned to be installed at an approximate cost of $12,500 before the end of year. I will finance this myself. Ouch. You can still help dig me out of crippling debt by donating though!
    • UPDATE: November 6th, 2022: The Generac project is dead! Too expensive. I have a new project that replaces this now.

  • A Unified Backup Server (COMPLETE!). Right now my warm backups for all data are stored on USB 3 hard drives attached to one of the hypervisor hosts. This is cheap, but not fast or ideal. I want to make a companion to Stor1.lain.local specifically as a backup target on the network so that I don't have to keep buying HDDs and HDD docks. It gets very confusing when you are drowning in HDDs that all have crappy power supplies and garbage bin SATA controllers. I estimate this to cost approximately $4,000 for 80TB of backup storage on a T620 or similar enterprise tower. We don't need quite as much as Stor1.lain.local thanks to deduplication and compression.
    • UPDATE October 5th, 2022: I have actually ordered an R720xd with 12 bays to act as the new backup server! This will be ready to go with 6x8TB drives in RAIDZ2. I put in about $800 to make this happen. The other 6 drive bays will still need funding.
    • UPDATE October 23rd, 2022: I bit the bullet and bought the rest of the drives, plus a hot spare. $1,250 for 6x16TB.

  • Solar. Want all your porn video downloads to be carbon neutral? Want to permanently fund Lain.la via selling Solar power? Buy me a solar array! A 10-15kW solar array with a 30kW lithium battery array would probably run me between $50,000 and $60,000. However, this one time investment would ensure that money would never be a problem again for Lain.la, and we'd completely offset all the power we're using to run everything, plus we'd gain another redundant power source. I already have the land. I just need the money to build it. And yes. I'm serious.

There are also the following ideas in the pipe:

  • Redundant networking (COMPLETE!), to prevent a single point of failure. This is Lain.la v2.3. Probably about $500-$1,000, but will also take a lot of time and effort to do. Networking is my weakest point. (I also hear multi-WAN may be back on the table with Comcast's upload speed upgrades in the northeast. Maybe for $60/mo we can have that second uplink after all...) Here's the shopping list as of now:
    • 2x QSFP-8LC-AOC-1001 40G -> 4x10G SFP+ Breakout Cables, 10 meters - $200 (Done!)
    • 2x 40G-QSFP-C-00501 QSFP+ Passive DAC cables - $50 (Done!)
    • 2x Broadcom 57810 Cards - $70 (Done!)
  • New and interesting security systems. You have probably seen my transparency log. Wouldn't it be great to prevent abusive material from finding its way to Pomf entirely? I have some ideas that require funding, either for hiring people to do some digging or taking up my time to engineer. While I can't really put a price tag on this yet without further scoping, we can call it $1,000. One such idea involves PhotoDNA.
  • Full Public IP Routing. One day, I really want an ASN and a block of routable IPs, so I can learn BGP and all the other things that go along with having real IP addresses in place. I could then provide people with virtual machines with real IP addresses so they don't have to fight through NAT. This could be an expensive journey. I don't know what any of this would cost in terms of time or money.
  • Incorporating. As a measure of personal protection and also as a mark of legitimacy for Lain.la, incorporation of either an LLC or, preferably, a charity, would be beneficial in legal and operational areas. (No, becoming a corporation doesn't suddenly make me evil or make me want profit.)
  • Scaling past Lain.la V2 (COMPLETE!). We currently top out at 4Gbps of outbound capacity. Pomf hits this fairly often. Exploring new and interesting ways to expand this capacity on the cheap costs money to explore and would help everyone who uses the service. Update: We can totally do 8Gbps now.
  • Another Hypervisor. Not that we REALLY need it, but for the sake of redundancy and extra free hosting capacity, another 256GB T5810 (or T5820) with a E5-2697A V4 CPU (Or Xeon W) would be nice. $1,000.
  • Redundant WAN. Currently we have a single point of failure in that there's only one business fiber line backhauling all services. We really should get this upgraded, plus add the hardware required to make sure we have true redundancy through and through. This will increase operational costs by something around $80/mo and maybe be a $500 capital charge.
  • Jettison Proprietary Software. Getting rid of my VMWare and Veeam stack will be tough, but would be the last exorcism of proprietary software I'd need to do. Moving to Proxmox with Backup Server would be the target, but we can't take lain.la down to test this lengthy process, so we'd need a second cluster and a migration plan. This is months of effort and probably a few thousand dollars in new toys.
  • The Change List for 2023! Let's not forget about that. There's some overlap, but items on there do need to get done. Most only have a time cost, at least.

I have so many more ideas past this that I simply don't have the cash for immediately. Building a home server room by renovating a bedroom, Getting a full 48U rack, Installing 30A service, installing redundant enterprise WAN links. It just doesn't end. I could go on forever. I have the drive to go on forever. I do need to cut this list off at some point though!

-7666