DesignFlix 5: Load Balancers, Gateways and Proxies Can Share Jobs

A request arrives at the backend. Before the application sees it, something may choose a destination, check credentials and decide whether the caller has sent too much traffic.

We often draw those jobs as separate boxes labelled load balancer, API gateway and reverse proxy. The labels are useful, but one component can do several of the jobs. A request doesn't need to visit three different products just to make the diagram look complete.

Let's follow an order lookup and separate the responsibilities before deciding where to put them.

The load balancer chooses an available instance

Suppose the order service runs on two instances. A load balancer distributes incoming traffic among eligible instances according to its configured algorithm and health information.

That can spread work and avoid sending new requests to an instance marked unavailable. It doesn't guarantee every instance has spare capacity, or that a shallow health check proves the order lookup works.

Load balancing can happen at different network layers. Some systems distribute connections; others inspect HTTP requests. Choose the behaviour you need, including connection handling, health checks and shutdown draining.

The original request path through traffic management and API layers

The gateway applies the API contract

An API gateway can provide a common place for authentication, routing, rate limits and supported request validation. For our GET /orders/318 example, it might reject an invalid token or an incorrectly formed order ID before forwarding the request.

These are configured capabilities, not automatic consequences of installing a gateway. Decide which controls run there and which must also run in the service.

In particular, identifying the account doesn't prove it may read order 318. The order service owns that record's access rules and must enforce them. A caller shouldn't gain another customer's details simply because the gateway accepted a valid login.

An IP allowlist may restrict certain integrations, but it isn't a general identity system. Browser-origin rules such as CORS aren't authentication either; non-browser clients don't become trusted by sending a chosen origin header.

John Wick: Chapter 3 scene — John Wick: Chapter 3 · Tenor · Adapted dialogue

The reverse proxy speaks on the server's behalf

A reverse proxy receives a client request and forwards it to a backend. It gives clients a stable front-facing endpoint while backend locations remain an internal routing concern.

It may also terminate TLS, cache eligible responses or distribute traffic, depending on configuration. NGINX's reverse-proxy guide and HTTP load-balancing guide demonstrate that overlap.

So "reverse proxy" describes the intermediary relationship, while "load balancing" describes a traffic-distribution job. An API gateway can also act as a reverse proxy. These categories don't have to be three exclusive kinds of machine.

Put the roles where they are useful

A small deployment might use one front-facing component for proxying, API policy and balancing. A larger deployment might separate the edge entry point, gateway tier and internal service routing.

Here's one simplified combined arrangement:

Three roles can live in one front-facing component
The front chooses one eligible instance for this request. Each order instance still validates access to the requested order.

Separate components when isolation, ownership, scale or a particular capability justifies it. Each extra hop adds configuration and another place a request can fail. Keep the route understandable enough to trace during an incident.

Headers and retries cross trust boundaries

The backend may need the original client address or request scheme after proxying. Only trust forwarded information from known intermediaries that sanitize incoming values. A header supplied by the public client isn't automatically reliable metadata.

Timeouts and retries also need agreement across layers. If the gateway waits longer than the user's deadline, useful work may continue after the user has given up. If multiple components retry a change, one intended operation can produce several attempts.

Which component made the last decision before this request failed?

Use a trace or request ID to answer that question. Check whether the front rejected the call, routing failed, the chosen instance timed out, or the application returned a business error. Those failures require different fixes.

Keep the service's meaning in the service

When the lookup reaches an order instance, it still has to validate access and return the correct representation. The response then travels back through the configured intermediaries.

Shared API policy can remove repeated work, and balancing can help distribute load. Neither replaces the application rules that make the order result safe and correct.

For your next architecture diagram, label the jobs first: admit, authenticate, route, balance and execute. Then choose the components that should own them. You'll get a clearer design than assigning every familiar product name its own mandatory box.

Try it yourself

The gateway accepts a valid login token for GET /orders/318. Can the order service skip checking whether that account owns or may access order 318?

Reveal the answer

No. Gateway authentication establishes the caller under its contract; record-level authorization remains necessary. Enforce the order's access rules in the service or another authoritative policy arrangement, and preserve trusted identity through the proxy boundary. A valid token, allowed IP or client-supplied Origin header does not by itself authorize this order.