Here's another updates and metrics article for your enjoyment.

Updates:

We've had quite a few improvements lately:

  • All Pomf traffic carrying edge nodes now have 2.5x the cache size and double the CPU/RAM resources. This prevents tons of callbacks to the storage over the OpenVPN links (60G of active content just wasn't enough anymore to keep everything loaded) and keeps things fast at peak load - the previous single core CPU was giving up at around 700mbit.
  • The NCMEC database has been 100% implemented into Pomf to ensure no evil files make their way to the datastore. If evil files DO make it in, I make reports to the NCMEC to get their database updated - meaning that organizations all around the world (Facebook, Google, Twitter, etc) will all benefit from my reports. Take that, jackasses.
  • We made it through another maintenance window. I was yelling about downtime, doom, and gloom but it ended up being all of 5 minutes of interruption. I also completely changed my backup scheme to something more reliable with less proprietary tooling and better encryption.
  • Still didn't get the bulk of the improved networking done. Soon!
  • I tried to get Francisco to give me 2 gbit/s per Pomf edge node for a sizeable monthly fee but he doesn't have the capability to do so. Yet.
  • I'd like to announce 7666.io! It's literally going to be Pomf.lain.la, but on a dedicated domain to work around recent "availability" issues with certain websites. Same files, same network, same systems. Just different domain. Coming soon!
Metrics:

Not much has changed, although bandwidth has been all over the place lately with various interesting events going on such as the Indonesian Porn Incident.

  • CPU: 6.27 GHz / 111.04 GHz used (CPU GHz is an arbitrary metric derived from multiplying core count * base clock of each server).
  • RAM: 177.67GB / 512GB used.
  • Disk: 7.45TB / 19.86TB used.
  • VM Count: 29 On-prem, 12 unique Hentai nodes, 7 endpoints, total of 48 VMs.
  • NAS LUN:

  • Edge Bandwidth (Multiply this chart by 4x, roughly, for a total outbound of 215.48TB):

  • Internal Bandwidth (Between edge and datacenter - calculated cache ratio of 26.4x):

  • Costs: We'll be killing off Greencloud's hentai nodes to save $32/mo (Edit: reverted this change on 7/18/2022. Also the redundant WAN project is on hold due to performance concerns.) and adding in the beefier edge nodes to add $60/mo. I also now account for 100W of extra power as well as my domain registration costs, for a total spend now of $331.50/mo $320.50. Getting a little pricier...
  • Pomf: 147,567 files, and disk space is below.

  • Uptime - 4 9's or better across the board. Not bad! Don't forget you can peek at my uptime numbers here and yell at me when things are down (unless I announced maintenance) at my Uptime Site.

So, that's all for now! Keep an eye on the Infrablog as well as the Global To-do List if you're interested in seeing how things shape up as we move through 2022.