QuizzesThe Mathematics Behind Transformers & LLMs — Concept Check
intermediate
math-for-ai

The Mathematics Behind Transformers & LLMs — Concept Check

Questions on The Mathematics Behind Transformers & LLMs. A transformer turns tokens into vectors and lets them exchange information through attention — a few matrix multiplies, a softmax, and a dot product — which is the mathematical core of every modern LLM.

8 minutes 8 questions Pass at 70%
Time Limit
8 minutes - timer starts when you begin
Change Answers Freely
Revisit and change any answer before submitting
Full Review After Submit
Correct answers and explanations shown for every question
Auto-submit on timeout
Quiz submits automatically when time runs out

Questions in this quiz

8 questions on The Mathematics Behind Transformers & LLMs — Concept Check. Work through them here, then start the quiz to answer them and see your score.

  1. 1.A transformer model calculates attention scores using dot products between which components?

    • AKeys and values
    • BEmbeddings and values
    • CQueries and keys
    • DQueries and values
  2. 2.In a transformer, why is the attention score scaled by 1/√d?

    • ATo enhance the similarity between tokens
    • BTo reduce computational complexity
    • CTo ensure numerical stability
    • DTo increase the range of scores
  3. 3.A developer notices that the attention weights in a transformer model sum to more than 1. What might be the issue?

    • AThe dot product is not computed
    • BThe query and key matrices are swapped
    • CThe softmax function is not applied correctly
    • DThe embeddings are not initialized
  4. 4.When computing attention in transformers, what is the role of the softmax function?

    • ATo normalize attention scores into a probability distribution
    • BTo compute dot products between vectors
    • CTo project tokens into embeddings
    • DTo scale values for numerical stability
  5. 5.A model outputs unexpected results after stacking multiple transformer blocks. What could be a potential cause?

    • AIncorrect embedding dimensions
    • BImproper normalization between layers
    • CUntrained query and key matrices
    • DMissing softmax application
  6. 6.Which of the following is NOT a component used in the self-attention mechanism of transformers?

    • AQuery
    • BValue
    • CKey
    • DBias
  7. 7.A software engineer claims that transformers do not use gradient descent for training. What is the flaw in this statement?

    • ATransformers use only matrix multiplication
    • BTransformers rely solely on dot products
    • CTransformers use gradient descent to minimize cross-entropy loss
    • DTransformers do not require training
  8. 8.What happens if the dot product between queries and keys is not scaled in a transformer?

    • AThe model may become numerically unstable
    • BThe values will not be weighted
    • CThe embeddings will not be vectors
    • DThe attention weights will be more accurate