Networking basics

You do not need to recertify in packet switching. You need the numbers and failure modes that change boxes on a whiteboard.

Latency is mostly distance and queues

A packet inside one data center is well under a millisecond. Across the US it is on the order of 60–80 ms round trip, before your handler runs. A TLS handshake on a cold connection adds another round trip or two. None of that shows up in a microbenchmark on localhost.

So the design rule is boring: do not chain synchronous cross-region calls on a request a human is waiting for. Cache, replicate reads near the user, or accept the latency explicitly. Timeouts are part of the API. Every outbound call gets a deadline shorter than the caller’s deadline, so a stuck dependency becomes an error instead of a pile of waiting threads.

TCP, UDP, and what you actually choose

TCP gives you a byte stream, retransmission, and congestion control. UDP gives you datagrams. In interviews you almost always want TCP, then an application protocol on top. UDP shows up when you would rather drop than wait: voice, some multiplayer game state, QUIC (which rebuilds reliability on UDP). Saying “I’ll use UDP for the API because it is faster” without a loss story is a yellow flag.

TCP’s cost is the handshake and head-of-line blocking on a single stream. HTTP/2 and gRPC fix the “one request at a time” problem by multiplexing streams on one connection. The new failure mode is the cap on those streams. A connection can be open, pass health checks, and still refuse new RPCs. That looks like a latency climb with healthy averages. The fix is more connections per destination, an explicit concurrent-stream limit, and metrics on queue time, not just request time.

Application protocols, only as far as the design needs

• HTTP/1.1 and HTTP/2 are the default for public APIs. Status codes are part of the contract: 408/429/5xx may be retryable, 400-class errors are not.
• gRPC is a strong default between services you own: typed protobufs, deadlines, streaming. Do not expose it as the public API unless your clients are also yours.
• SSE is one-way server push over ordinary HTTP. It sits behind normal load balancers more happily than a custom socket.
• WebSockets are bidirectional. Use them when the client sends often (chat, cursors), and plan for sticky sessions or a pub/sub fan-out so any server can deliver.
• WebRTC is peer-to-peer media. Mention it for calls, then put signaling on your servers and media off them.

Load balancers

Layer-4 balancers spray TCP connections. Layer-7 balancers understand HTTP and can route on path, header, or cookie. Both will send traffic to a target that just started failing unless health checks are honest and fast. Draining matters: a deploy that kills a process also kills the in-flight streams on it. For long polls and gRPC, “least connections” is a better policy than round-robin, and even that does not see queued streams inside one connection.

Health checks should hit a dependency that means the process can serve, not a handler that always returns 200. A check that passes while the database pool is exhausted will keep the box in rotation.

flowchart LR
  U[Client] --> DNS
  DNS --> LB[L7 load balancer]
  LB --> A[API A]
  LB --> B[API B]
  A --> DB[(Primary)]
  B --> DB