Submitted by 7666 on

We're out of space folks! Come back when the Proxmox project is completed! Another 768GB of RAM is standing by!

 

As you may or may not be aware, I offer free (yes, as in beer) hosting and support for anyone in need of it. I have a stupid amount of hardware, and hoarding it all to myself doesn't make much sense with the amount of time and money I've dumped into it all. This article is meant to be an overview of what that offer entails, and what you can and cannot do with your free box.

Specifications:

The image above shows the machine where I stuff all of the free VPSes. It's a Dell Precision T5810, with a 12 core E5-2678 v3 CPU. It runs at a base clock of 2.5GHz, but should turbo into the 2.9GHz to 3.0GHz range with the current load. There's 128GB of DDR4 memory, and redundant 10Gb networking for the LAN. You will be sharing this machine with 14 other VMs, as of this article's writing. Also, in the unlikely event of that server kicking the bucket, your VM will automatically migrate to another server and reboot itself. Here are the current resource usages:

Plenty of room there for your needs. You may notice that it would be physically impossible to stuff that much storage in that chassis - that's true. The storage is provided via iSCSI over that 10Gb network from a set of dedicated storage servers. All data is encrypted by default. I also automatically back up your data, twice a week, to my secondary storage server. Every three months, that data is copied to USB hard drives and taken to a secure, off-site location. The backups are double encrypted.

Please note: There are two non-free components in my stack: The hypervisor, and the backup solution. The hypervisor is VMWare ESXi, and the backup system is Veeam. Both are legitimately licensed, but still not FOSS. Everything else is free software (outside of things like Intel processors, or Dell iDRAC modules, or Brocade switch firmware).

Networking:

I am not a cloud service provider. I do not have a big block of IPs sitting around I can hand out. What I do have are the aforementioned "edge nodes", which act as proxies for inbound traffic. Traffic comes into these proxies and is routed back to pfSense, and then to your VPS via NAT from there. Here's a very quick graphic:

This allows me to publish anything and everything I want (and you want) without having to buy an IP block. The problem is, it also means that I have to do the routing of things on your behalf, if you're publishing a service on a port. Therefore - if you want to host a service, you must tell me in advance. I will set up a port for you that will route to your VPS. Outbound traffic is completely unrestricted, however, and you'll always get an SSH port pointed at your VPS in the 20000-29999 range (unless you ask me not to open SSH).

Note: Advanced users can get around this limitation by tunneling their free VPS, via a VPN, to a public VPS of their choice, emulating the same "edge node" concept. You can control your firewall and routing yourself by doing so, without ever needing my intervention. This is great if you can't afford the firepower of a large VPS and want to offload that to me while keeping a small one at a public cloud provider as the point of ingress.

As for what all this proxying goes over (the WAN), I have a gigabit symmetrical fiber connection, however, you'll likely not hit those speeds, as your VPS will be tunneled out via Slice 5 (OPT5), the "edge node" dedicated for outbound traffic. This usually gets about 250 mbps download and 150 mbps upload. The IP address for this is 198.251.81.40, and that's what you'll see if you hit a "What's my IP" website from the command line.

For the full network diagram, please see here. The only part of this diagram that hasn't been implemented is the redundant WAN links. Comcast sucks still.

Websites:

Note: If you want to host a static website, you don't need a VPS from me. Please use Gitlab Pages, at https://gitlab.lain.la. You can host whatever you want there and it's entirely self service, so long as you're ok using a domain on <yourname>.pages.lain.la (I do not support custom domains).

This part is tricky. On every single edge node, I am already running Nginx on the default HTTPS port. This means you can't use Nginx on port 443 yourself. You can use a non-standard HTTPS port, but then you'll need to supply your users with this port and hope their firewall doesn't axe the connection. This leaves a bit of a conundrum. However, if you wish, you can use my edge nodes as Nginx reverse proxies by pointing your DNS A record at one (I'll tell you which one). I use Server Name Indication to differentiate between HTTPS requests for different websites made to the same IP and port, which means you can have your website hosted via my edge nodes and it will be 100% functional. There are HUGE caveats to this, however!

  • I AM MAN IN THE MIDDLING YOUR TLS. Because you have your DNS pointed at my edge node, TLS terminates there. It also means I am doing your LetsEncrypt certificate renewals for you. If this is a problem, use a non-standard HTTPS port OR tunnel out to your own box somewhere. I promise that I'm not looking at your traffic, but still.
  • I highly recommend you have internal HTTPS configured, as I don't want unencrypted traffic traveling through my network tunnels, even if they are encrypted with AES-128. Please make sure you let me know if you wish to opt for this. However, by default, I'll use port 80 for the reverse proxy for simplicity's sake if you don't tell me otherwise. (Note: You can use a self signed certificate for HTTPS internally quite easily: Here's an OpenSSL one-liner to generate one.)
  • You won't be protected by my failover system, because I cannot control your DNS. This means if I do maintenance or Francisco (owner of BuyVM, the provider I use for the edge nodes) trips over a cable, you're out of luck. If you have an uptime-sensitive application and use Namecheap, however, you can generate me an API key in your account and I'll happily add you to my monitoring and failover system. Note this means giving me access to your DNS records.
Reliability Expectations (New Section):

My infrastructure has had a good record of reliability, rivaling most cloud providers. You can check my Uptime page for proof. However, there are some weak spots in it simply because the cost/benefit of fixing those weak spots hasn't made sense, or I have no better alternative at this time. You should be fully aware of these.

Single Points of Failure:

  • ISP - I only have my dedicated symmetrical business fiber line here. If that goes kaputt, connectivity is gone.
  • Core Router - There's only one of these at this point, so a hardware failure will have the same issue. I have a full spare in reserve though.
  • Publishing your service out one edge node - If that node goes, so does your service/website/whatever. You can opt for quad redundancy however!

Moderately Redundant Components:

  • Storage - Excluding RAID, dual PSUs, and dual network uplinks, the storage server data is also copied to a second, nearly identical storage server. It's not an immediate failover if it completely dies (as it requires my intervention to swap in the spare server), but I count it as moderately redundant.
  • PfSense - I only run one instance of these, so if there's a software fault or other problem, it's going to impact availability, but if there's a hardware fault of some kind it won't be an issue as it's a fully virtualized instance - I can just restore it from backup to any other server.

Strong Redundant Components:

  • Compute - The host your machine runs on is fully redundant. If it dies, it automatically moves to another one and reboots itself without my intervention.
  • LAN - This is fully redundant, with two separate NICs, two separate uplinks, and two separate switches for every single device.
  • Power - This is fully redundant. Each server and switch has one power lead that goes directly into wall power, and another one that goes through my UPS system. The UPS system has about 6-7 hours of runtime, which, in almost all cases, is enough time for power to return or for me to get home and fire up the generator.
  • Publishing your service out all of my edge nodes - If you opt for this, your service will be published on 4 edge nodes with 4 IP addresses simultaneously. Failures at the edge will be detected and the offending node removed in 5 minutes or less. It just requires Namecheap API access (sorry, I only wrote it for Namecheap's API).
  • Data Resiliency - Your data is extremely safe, with redundancy at every layer. The running copy has RAID and encryption, the twice-weekly automatic backups have RAID and double encryption, and the offsite backups every three months have double encryption and are stored on the second floor of a remote brick fortress.
Requirements:

So, if you've made it this far, I assume you're still interested. Great! I will require the following information from you to provision a VPS. You are to email me this information (at 7666@lain.la).

  • What are you planning on running?
  • What resources (CPU, RAM, Disk, etc) do you need?
  • Do you need any inbound ports, or need a website published via my proxies?
  • Are you using your own domain, or would you like to use a *.lain.la subdomain (e.g. yourname.lain.la)?
  • What operating system do you want? (Note: if you want a GUI or want to use Windows, I have an Apache Guacamole server available so you can VNC/RDP in.)
  • What are your uptime/reliability requirements? If it's a critical service we may want to take some extra time planning it out.
  • Please provide a public key (RSA or ED25519) for SSH access (for non-GUI deployments).
  • Any special requirements or requests that you may need.

After I receive your email, I will provision your VM manually and email you back with an IP and port, as well as any other additional details. Root is always used as the login user, however most operating system installers require a regular user account to be made, so I'll likely pick your online alias for this account. You can either transfer your key to it and use that account (and lock down sshd_config), or delete the account. In either scenario, once you are logged in, change the password. That's it!

As part of my deployment procedure, I will add your public key to authorized_users and configure networking. I will also configure SSH for public key only access. Please note that I use open-vm-tools (auto-installed in Debian, manually installed elsewhere) as my IP address management system as well as for VMWare specific drivers where required. If you remove this, it will become impossible for me to organize VPSes by IP and I'll probably break your VPS the next time I deploy one because I won't see the IP as in use. Open-vm-tools is FOSS and non-intrusive, so there's no reason to really mess with it. Lastly, I don't clear bash history, so you can peek at what I did for configuration on your VPS for transparency's sake. Just don't laugh at me if I make a typo.

Limits:

While I'm very generous with the amount of hardware I provide, please be aware that I'm not made of money, and you are sharing the infrastructure with other people. The following limitations are in effect for your offer. If you don't like it, tough. Remember, it's literally free.

  • CPU/RAM requests above 2 cores / 8GB of RAM will generally be rejected. Also, slamming the CPU 24/7 is not a good use of resources (no bitcoin mining lol).
  • Flash storage is generally limited to 100GB max. I typically provide 40GB or 60GB. HDD storage, however, can be granted up to 1TB (or even more with justification).
  • If I intend to, or already am, hosting a public service (e.g. Invidious), I will typically run that myself instead. Exceptions can be made for things like Fediverse instances if they're materially different to my own.
  • Seedboxes, VPNs, Tor nodes, File upload services, or anything else extremely resource intensive or abuse-prone are not welcome.
  • Please don't beat the shit out of the network. My ISP already probably hates me. Pushing more than 2TB of outbound bandwidth a month, or using well over 100mbps up or down constantly, would not be good.
  • Do NOT try to hack the other 14 people's boxes, please. You are in an internal /24 subnet with them. Be a good neighbor. Also, network scanning or discovery tools are prohibited. You are still eligible, however, for the Lain.la bug bounty if you find something insecure with my own systems and infrastructure, but only if you weren't actively looking for things to exploit. You're basically cheating as I've personally given you network access.
  • I reserve the right to terminate your free hosting at any time, for any reason, however your data is your data and I will provide you a virtual disk export upon termination if you request it. I have never had to terminate anyone's hosting before though.

 

I think this covers everything related to the hosting offer. There is one other point I do wish to make, and it's that if you do end up utilizing my free hosting services, consider donating sometime down the road, or pointing the users of whatever you're hosting to my donation page. Again, I'm doing this for free, so this means I'm paying your costs for you. You are absolutely not obligated to do so, but it would help offset the astronomical $520/mo Lain.la budget and allow me to build more neat things.

Love,

-7666