Definition
Transformer (architecture)
The transformer is a neural network architecture introduced in 2017 that uses an attention mechanism to weigh the relationships between all elements of a sequence, enabling modern language models.
Thanks to attention, the model can consider a word's context across the whole text and parallelise processing heavily. This architecture underpins nearly all of today's LLMs; the "GPT" in ChatGPT stands for "Generative Pre-trained Transformer".