119
tail latency
The average can look fine while the slowest 1% makes the page feel broken. Watch the tail.
What is tail latency?
Tail latency is the slow end of the distribution: p95, p99, not the mean. A request that waits on several services inherits the slowest one. Averages hide that. Timeouts and retries make the tail worse if you are careless.
Why does tail latency matter when vibe coding?
Models log the average and ship. One in twenty checkouts takes 8 seconds. Name p99 and a timeout budget across the calls.
How do you do tail latency?
Measure p95 and p99 for the user-facing route. Give each downstream call a timeout that fits inside the parent budget. Do not retry a call that is already the tail.
How do you ask a model for tail latency?
Track p95 and p99 for (route), not only the average. Each downstream call gets a timeout inside the parent budget. Do not retry so long that one slow call becomes three.
What goes wrong with tail latency?
Retrying every failure immediately. You turned a blip into a storm and lengthened the tail. Backoff and jitter, and a limit.