Troubles with distributed systems

Partial Failures

When a program runs on one machine, you expect it either works or it doesn't. When a program runs on several computers, it can happen that one node fails — and the problem starts when you need to identify and fix an organism that's partly dead, but for the client it has to act like nothing happened.

That's the whole game with distributed systems: you can't prevent partial failures, so you design around them — timeouts, health checks, retries, failover. Everything below is a different flavor of the same fight.

---

Problem with Time