In-context learning

Appears in 4 papers

Performing a task by providing examples in the prompt, without updating model weights.

As used in Paper 10 — Improving Language Understanding by Generative Pre-Training →

Performing a task by providing examples in the prompt, without updating model weights. GPT-3 demonstrated this; it was not a feature of GPT-1.

As used in Paper 12 — Language Models are Few-Shot Learners →

Learning a task from examples provided in the prompt, without updating the model's weights. The model uses attention to recognize patterns in the examples and applies them to new inputs. This is the central mechanism of GPT-3.

As used in Paper 14 — Chain-of-Thought Prompting Elicits Reasoning in Large Language Models →

The ability of a language model to learn from examples in its input context (the prompt) without updating its weights. CoT is a form of in-context learning where the model learns a reasoning pattern from the provided examples and applies it to new problems. This is distinct from fine-tuning, which modifies model weights.

As used in Paper 15 — Training Language Models to Follow Instructions with Human Feedback →

The ability of a language model to learn from examples in its input context (the prompt) without weight updates. Related but distinct from the learning in this paper, which uses explicit training procedures (SFT, RM, RL) over many examples.

Paper 10 — Improving Language Understanding by Generative Pre-Training → Paper 12 — Language Models are Few-Shot Learners → Paper 14 — Chain-of-Thought Prompting Elicits Reasoning in Large Language Models → Paper 15 — Training Language Models to Follow Instructions with Human Feedback →

Appears in papers