Nerd Level Tech

Load Balancing in System Design ➤ What is Load Balancing? → Definition: The process of distributing incoming network traffic across multiple servers to prevent any single server from becoming overloaded. → Goal → Improve performance → Increase availability → Distribute workloads → Prevent bottlenecks → Load balancers sit between clients and backend servers. → Example → 1,000 requests arrive → Load balancer distributes them across 5 healthy servers → Each server handles approximately 200 requests, assuming equal distribution. ➤ How Load Balancing Works → Client sends a request → Load balancer receives the request → Checks server health and availability → Selects an appropriate backend server → Forwards the request to that server → Server processes the request and returns a response → If a server fails → Health checks detect the failure → Traffic is redirected to healthy servers. ➤ Why Load Balancing Matters → Prevents individual servers from becoming overloaded → Improves application responsiveness → Increases availability through server redundancy → Supports horizontal scaling → Enables maintenance with reduced service disruption → Helps applications handle traffic spikes → Common use cases → E-commerce → Social media → Streaming platforms → Banking systems → Cloud applications ➤ Types of Load Balancers → Layer 4 Load Balancing → Operates at the transport layer (TCP/UDP) → Routes traffic using network information such as IP addresses and ports → Offers efficient, low-overhead traffic distribution → Layer 7 Load Balancing → Operates at the application layer (HTTP/HTTPS) → Routes requests using URLs, headers, cookies, or hostnames → Supports content-based routing and application-aware decisions → Hardware Load Balancers → Dedicated physical appliances → Common in traditional enterprise infrastructure → Can provide high performance but may increase infrastructure costs → Software Load Balancers → Run as software on servers or virtual machines → Flexible and configurable → Examples → NGINX → HAProxy → Envoy → Cloud Load Balancers → Managed load-balancing services offered by cloud providers → Examples → AWS Elastic Load Balancing → Google Cloud Load Balancing → Azure Load Balancer ➤ Common Load Balancing Algorithms → Round Robin → Sends requests to servers in a repeating sequence → Best for servers with similar capacity and relatively uniform workloads → Weighted Round Robin → Assigns more requests to servers with higher weights → Useful when backend servers have different capacities → Least Connections → Routes traffic to the server with the fewest active connections → Useful when requests take different amounts of time to complete → Weighted Least Connections → Considers active connections and server capacity → Helps distribute workloads across heterogeneous servers → IP Hash → Uses a client's IP address to select a backend server → Can help maintain routing consistency, but does not guarantee session persistence in every configuration → Least Response Time → Selects servers based on response time and, depending on the implementation, active load → Helps direct traffic toward more responsive instances ➤ Health Checks and Failover → Active Health Checks → Periodically send probes to backend servers → Passive Health Checks → Detect failures from actual traffic → Unhealthy Server → Stop sending new requests to that server → Recovery → Return the server to service after it passes health checks → Failover → Redirect traffic to healthy alternatives → Important → Health checks reduce the impact of failures, but they do not eliminate every failure scenario. ➤ Load Balancing Architecture → Client Requests → Load Balancer → Server 1 → Server 2 → Server 3 → Server N → Shared Cache → Database → Read Replicas → Traffic increases → Add backend instances → Server fails → Redirect traffic to healthy instances → Traffic decreases → Scale down excess capacity → For critical systems → Deploy redundant load balancers to avoid creating a single point of failure. ➤ Challenges in Load Balancing → Uneven traffic distribution → Session persistence and state management → Health-check configuration → Backend server failures → Load balancer capacity limits → TLS termination and certificate management → Cross-region routing and network latency → Maintaining observability across multiple servers ➤ Load Balancing vs Auto Scaling → Load Balancing → Distributes traffic across available servers → Auto Scaling → Adds or removes server instances based on demand → Load balancing helps distribute existing capacity → Auto scaling adjusts the amount of available capacity → Together → Support responsive, scalable applications ➤ In System Design Interviews → Estimate incoming traffic and peak requests per second → Identify potential server bottlenecks → Choose Layer 4 or Layer 7 load balancing → Select an appropriate routing algorithm → Explain health checks and failover → Discuss stateless services and session management → Consider multi-zone or multi-region deployment → Evaluate latency, availability, cost, and operational complexity ➤ System Design Handbook → A practical guide to mastering system design fundamentals and distributed architectures → Covers load balancing → Scalability → Consistency → Fault tolerance → Durability → Caching → Databases → CAP Theorem → Designed for developers preparing for system design interviews and building reliable production systems Get the System Design Handbook here: codewithdhanian.gumroad.com/…