Hidden Layer

A layer of neurons between the input layer and the output layer. "Hidden" because their outputs are not directly observable — they are intermediate representations used internally by the network. The existence of hidden layers is what allows networks to learn complex, non-linear patterns. Credit assignment to hidden layers is what backpropagation solves.

One block in the stack; each has an attention part and a feed-forward part.