Token

A unit of text, roughly a word or subword. GPT-3 uses a vocabulary of ~50,000 tokens. Longer sequences (more words) = more tokens = higher computational cost. The context window (~2,000 tokens) limits how much text you can input at once.

A unit of text, roughly a word or subword. GPT-2 uses a 50K-token vocabulary; tokens are the building blocks of training data and model inputs.

A chunk of text (~¾ of a word / ~4 characters). LLMs read and write in tokens; you're usually billed per token.

A chunk of text (roughly ¾ of a word) that models read and generate in. Context limits and costs are measured in tokens.

A credential issued after authentication/authorization that a client presents to prove it may perform scoped actions.

A chunk of text (roughly ¾ of a word). Models read and generate in tokens, and cost and limits are measured in them.

A chunk of text (a word or word-piece); models read and generate in tokens, and cost/limits are measured in them.

A chunk of text (~¾ of a word) the model processes. Models have a max token length per input.

The unit of text a model reads/writes, roughly a word-piece (~4 characters / ~0.75 words in English). (M01)

The unit LLMs read and write in; roughly ¾ of a word in English. You pay per token, and models have token limits. The most important unit in AI production. (Mod 0, 5)

A chunk of text (a word, part of a word, or punctuation) — the actual unit LLMs read and produce. Roughly 1 token ≈ 0.75 English words ≈ 4 characters.

The unit of text the model processes; roughly ¾ of a word. Context size is measured in tokens. (Mod 13)