Pretraining
The initial, expensive phase where a model learns language by predicting the next token across enormous amounts of text.
The initial, expensive phase where a model learns language by predicting the next token across enormous amounts of text. (M01)