Joan Navarro

Scalability! But at what cost? This paper is an absolute classic because it explores the underappreciated tradeoffs of distributing systems. It asks about the COST of distributed systems--the Configuration that Outscales a Single Thread. The question is, how many cores does a big distributed system need to outperform some moderately-optimized single-threaded code running on your laptop? As it turns out, scalability often comes with an extremely high COST. The authors examine several graph processing systems--including some big names like Spark--and find that they need dozens to hundreds of cores to outperform a single-threaded program. Why is this the case? It's not because these distributed systems are badly designed, but because distributing computation is inherently inefficient for many problems. Fundamentally, a distributed system cannot rely on all processors sharing state, at least not efficiently. This is a big issue! In graph algorithms, it means servers need to expensively exchange data and eliminates a wide swathe of algorithms and optimizations that rely on shared state. In distributed databases, it means expensive coordination is required to distribute transactions to ensure participating servers have consistent views of data. Does this mean we shouldn't build scalable systems? Of course not! Many problems are well beyond the capability of a single server, no matter how optimized. But it does mean we should be mindful of the efficiency costs of scaling. As an aside, I think this kind of thinking is why Postgres is so popular, despite not being distributed. A large Postgres server can handle a vast amount of traffic (especially with read replicas, which can be cheaply maintained). You need a huge company or incredibly heavy workload to outscale that single server, and when you do, the alternatives come with huge tradeoffs!