TPOT (Time Per Output Token) / ITL (Inter-Token Latency)

Appears in 1 tutorial

Average gap between subsequent output tokens.

Average gap between subsequent output tokens. The "typing speed." ≈ 1 / per-stream tokens-per-second.