Head (attention head)
One of h = 8 parallel attention computations in multi-head attention, each operating in a lower-dimensional subspace (dₖ = d_model / h).
One of h = 8 parallel attention computations in multi-head attention, each operating in a lower-dimensional subspace (dₖ = d_model / h). Each head has its own Q, K, V projection matrices and learns to attend to different relationships. Outputs are concatenated and projected by W^O.
"Where you are now" — a pointer (usually to a branch) for the commit your working directory reflects. (Mod 3, 4)
Points to the current branch (or a commit, when "detached"). (Mod 4)