TTFT (Time To First Token)
Time from sending a request to the first output token.
Time from sending a request to the first output token. Includes queue wait + prefill. The "pause before it starts typing."
Time from sending a request to the first output token.
Time from sending a request to the first output token. Includes queue wait + prefill. The "pause before it starts typing."