Minecraft Server Census

Summary

Minecraft Server Census is a personal project that pings thousands of public Minecraft Java Edition servers every 10 minutes and turns the answers into a public dashboard: players online now and over time, the busiest servers, per-country views, and a page for every server with its player history, accepted client versions, software and hosting details.

Motivation

Minecraft is a big part of my life. I spent years building Rede Sky, which became the largest Minecraft network in Brazil, and since I stepped away from that world I've been curious about what the multiplayer scene looks like today. Which servers are growing, how big the Brazilian community is compared to the rest of the world, and when people actually play.

At Rede Sky we had internal tools to keep an eye on competitors, so the idea wasn't new to me. This time I wanted a broader, public version of it, partly out of curiosity and partly in the hope that one day I'll build something around Minecraft servers again.

How it works

• Sweep: Every 10 minutes, each tracked server gets the same status request the game client sends when you open the multiplayer menu. The full reply is stored: player count, version, message of the day, icon, mods.
• De-duplicate: Hostnames that are really the same server are grouped together, so their players are counted once.
• Aggregate: Totals are written per sweep, overall and per country, and each server's history is rolled up by hour so long time ranges stay cheap to query.
• Serve: A read-only JSON API and the React dashboard are served by the same process.

Challenge: counting each server once

Pinging a server is the easy part. The hard part is deciding what counts as one server. Big networks are reachable through many hostnames (play.example.com, mc.example.com, example.net…), and simply adding them up inflates the total a lot: at the time of writing, the naive sum of every hostname is about twice the real number.

Neither DNS nor the status reply can settle it alone. Large networks spread a single player pool across several IPs and ports, while DDoS protection services and hosting providers put dozens of unrelated servers behind the same IP. So hostnames are only merged when there is a link between them and the status replies corroborate it:

• Same player: the same real player shows up in both player samples.
• Same address: they share an IP and port, report the same player count, and have the same icon, message or domain.
• Same DNS target: they point to the same DNS record, with matching player counts and the same icon or message.
• Same fingerprint: identical icon, message and player limit, with matching player counts, for networks spread across several IPs.

Small servers have to look identical to be merged, since that's where hosting providers place many customers behind one address and tiny player counts agree by accident. Every merge is shown on the server's page along with its evidence, so a wrong one is easy to spot and fix by adjusting the rules.

Challenge: running it for almost nothing

This is a side project with no revenue, so I wanted the monthly bill to be as close to zero as possible. Everything runs as a single Node.js process on one 1 GB, shared-CPU virtual machine, with no separate database server, queue or cache to pay for. Storage is SQLite through Node's built-in node:sqlite, pings use raw TCP sockets, and there are no native dependencies.

Fitting the workload on such a small machine shaped most of the design decisions:

• Lean writes: Pings are the bulk of the data, at around 700,000 rows a day. That table has a single index, and a server's full status reply is only stored again when it changes.
• Hourly rollups: Anything older than three days is read from hourly aggregates instead of raw pings, so charts covering weeks stay fast.
• Reads off the main thread: SQLite in Node is synchronous, so API queries run on a separate thread. A slow query never stalls a sweep in progress.
• Cache per sweep: The data only changes every 10 minutes, so each API response is computed at most once per sweep, no matter how many people are looking at the dashboard.
• Backoff for dead servers: Servers that stop answering are pinged less often, so they don't eat into the sweep's time budget.

The result is a full sweep of almost 5,000 hostnames in about a minute, on a machine that costs just a few dollars a month.

Main takeaways

Minecraft Server Census started as a way to satisfy my own curiosity and turned into an interesting engineering exercise: the data is noisy and self-reported, and getting an honest number out of it took far more thought than collecting it. It also showed me how far a single process and SQLite can go when the workload is designed around them. Above all, it brought me closer to the Minecraft community again.