Design the single entry point for a company's public and internal APIs: TLS termination, authentication, routing to services, rate limiting, request/response transformation, retries/circuit breaking, and observability — without becoming a bottleneck or single point of failure.

• 500k RPS; a well-tuned proxy node (Envoy/Nginx) does ~50–100k RPS → 10–20 nodes per region + headroom.
• Per-request work: TLS (offloaded/session resumption), JWT verify (~50 µs with cached keys), route match (trie), rate-limit check (local + Redis), forward.
• Logs: 500k RPS × 500 B = 250 MB/s → sample or ship asynchronously; never block the request path.

Data plane: any HTTP/gRPC request. Control plane:

PUT  /admin/routes   { match:{ host, pathPrefix:"/v2/orders" }, upstream:"orders-svc", timeoutMs, retries, auth:["jwt"], rateLimit:{...} }
PUT  /admin/upstreams/{name}   { endpoints via service discovery, healthCheck, lb:"least_request" }
PUT  /admin/plugins/{route}    ordered filters

Config store (etcd/DB): routes, upstreams, policies, API keys (hashed), JWKS cache. Runtime state on each node: compiled route table, connection pools, circuit-breaker state, local rate-limit buckets. Shared state: Redis for global rate limits and key revocation.

1. Edge: DNS/anycast → L4 load balancers → gateway nodes (Envoy-based) in each region; stateless.
2. Filter chain per request: TLS terminate → request id/tracing headers → auth (API key lookup in local cache; JWT verified with cached JWKS) → authz (scopes) → rate limit (local token bucket, then Redis for global) → route match → transform → upstream call with timeout/retry/circuit breaker → response transform → logging/metrics.
3. Control plane: an admin API writes config to a store; a config distributor pushes versioned snapshots to nodes (xDS-style). Nodes validate and hot-swap the route table atomically.
4. Service discovery integration for upstream endpoints; active + passive health checks; outlier ejection.
5. Observability: metrics per route/upstream, access logs sampled to a pipeline, distributed tracing headers propagated.
6. Resilience: per-route timeouts, retry budgets (not blind retries), circuit breakers per upstream, load shedding under overload (priority-based).

Retries are dangerous: use retry budgets (≤ 10% extra load) and only on idempotent methods/errors; add jitter.

Auth caching: API keys validated against a local cache refreshed from the control plane; revocations pushed immediately.

Config safety: validate, canary to 1 node, then roll; keep the last N snapshots for rollback.

Load shedding: when CPU/queue thresholds trip, reject low-priority routes first (return 503 fast) to protect critical ones.

gRPC/WebSocket: pass-through support with proper timeouts (idle vs request).

• Control plane down: data plane keeps serving with the last config (must be fully decoupled).
• Redis down: rate limits fall back to local buckets (fail-open); alert.
• Upstream slow: circuit breaker opens; fast 503; retries stop; health checks bring it back gradually.
• Gateway node crash: L4 stops routing to it; connections re-established elsewhere.
• Bad config push: validation + canary + auto-rollback on error-rate spike.