When people talk about infrastructure of large internet services, the term distributed systems shows up pretty fast. It's basically the umbrella term for "backend stuff that has to keep working even when parts of it are broken." People sometimes make it sound like black magic - servers that somehow recover when nodes crash, databases that don't corrupt everything when a network switch glitches, clusters that autoscale and scaledown.
The funny part is that the core ideas behind this aren't new or mysterious. All of this stuff were forming in the 1970s, mostly in academic and research environments. Cool tech, but not something most people ever had to think about. Things like logical clocks, consensus, and fault tolerance were being worked out long before anyone had "web scale" problems. Baby Internet was already a distributed system, just tiny. So yeah, distributed computing has basically existed as long as computing has been a thing.
What web brought though is that around 2000s the consumer internet services came up and suddenly needed to handle millions of users. At that point, scaling out across many machines stopped being a research topic and became the way to build real-world systems. Companies that hit scale early had to turn decades of ideas into running code and infrastructure. Foundational in ways that shaped the "modern stack".
So if you ship something today, the core machinery for building fast, scalable systems is already there, most of the time you don't need to reinvent anything. But there's still a difference between using the machinery and having a feel for it. You don't have to be a mechanic to drive fast, but a bit of mechanical sympathy helps 🏎️.
Getrafty is a community and a blog with a single goal - make those big, intimidating distributed systems feel small and approachable again. Step by step, no black boxes, no hand-waving, just code you can run, break, and understand.
Catch: Why do we need distributed systems anyway? 🤔
What's Inside?
We're going to build a distributed file system layer by layer, starting with the absolute basic such as operating system primitives all the way down to consensus algorithms.
Get ready 🍿
License
The source code is released under the MIT License.
Contributing
If you want to contribute, you are absolutely welcome! Get in touch ❤️.