Attention
A mechanism that lets a model weigh how relevant each part of its input is when producing each part of its output.
Self-attention computes, for every token, a weighted mix of all other tokens in the context. It is the core operation of transformers.
Related: Transformer, Context window