← Dictionary / Attention

Attention

Appears in 1 paper · 2 tutorials

A mechanism introduced the same year as seq2seq (Bahdanau et al., 2014)

As used in Paper 06 — Sequence to Sequence Learning with Neural Networks →

A mechanism introduced the same year as seq2seq (Bahdanau et al., 2014)

As used in Fine-Tuning & Model Customization →

The mechanism letting each token look at other tokens and pull in relevant information. Uses Query, Key, Value, and Output projections. (M01)

As used in LLM Infrastructure →

The mechanism letting each token "look at" relevant other tokens. Where the KV cache comes from.

Appears in

Paper 06 — Sequence to Sequence Learning with Neural Networks → Fine-Tuning & Model Customization → LLM Infrastructure →