Sampling

A client primitive that lets a server ask the host's model to generate text — so servers can use the LLM without holding their own API key.

Keeping full detail for a representative subset of requests (plus all errors/flagged cases) to control storage cost at scale. (Mod 2)

Choosing the actual next token from the model's predicted probabilities. Controlled by temperature, top-p, top-k.