Latency-throughput curve

Appears in 1 tutorial

A plot of throughput vs p99 latency across concurrency levels; the most important serving chart.

As used in LLM Infrastructure →

A plot of throughput vs p99 latency across concurrency levels; the most important serving chart.