shedder-shredder

Why this exists

When a service saturates, uniform rate limits treat a health check the same as an analytics ping. Load shedding is the middle ground: keep accepting work until you cannot, then drop the least important requests first and return 503 with Retry-After so clients back off.

shedder-shredder packages that logic as composable Go middleware, and ships a client + server demo to stress-test shedding behaviour under synthetic load — no Envoy, no mesh.

The middleware

Three priority tiers: high (never shed), medium, low. Each route wraps its handler with Middleware(priority).

Shedding decisions use:

• In-flight request count — separate thresholds for shedding low-only vs low+medium
• Rolling average latency — integer EMA ((old×9 + sample) / 10), reset when idle
• Manual modes — bypass, normal (threshold-driven), force-shed-low, force-shed-medium

Shed responses set X-Priority, return 503, and attach a random Retry-After between 3–10 seconds.

// Normal mode (simplified)
if inFlight >= mediumThreshold && priority < high { shed }
if avgLatency >= mediumLatencyThreshold && priority < high { shed }
if inFlight >= lowThreshold && priority == low { shed }
if avgLatency >= lowLatencyThreshold && priority == low { shed }

Thresholds are configured via environment variables (SH_LOW_THRESHOLD, SH_MEDIUM_THRESHOLD, SH_LOW_LATENCY_THRESHOLD, SH_MEDIUM_LATENCY_THRESHOLD).

Demo stack

Two binaries, one Compose file:

Binary	Role
server (:8086)	Slow simulated endpoints /high, /medium, /low (500–3000 ms). HTMX dashboard to switch shedding mode. WebSocket live stats.
client (:8085)	Load generator UI — spawn N virtual users per priority (including random), hammer the server, stream results over WebSocket.

make docker-up    # distroless images, health probes via -probe

The client runs with a no-op shedder; only the server enforces shedding. Request IDs use UUID v7 (X-Request-ID).

Architecture

Package	Role
internal/shedder	Priority middleware, modes, atomic in-flight/latency counters
internal/webserver	HandlePriorityFunc, request-ID middleware, server lifecycle
internal/user	Virtual user pool and HTTP load generator
internal/websocket	Broadcast stats and notifications to HTMX clients
templates/html	Server and client control panels

Static binaries (CGO_ENABLED=0), Go 1.26.

What it is not

Not a production ingress controller. A reference implementation and playground for priority-tier shedding you can lift into your own http.Handler chain.