Computer Engineering is coming soon

We're still writing and reviewing the notes for every subject, and we'll open it when every subject is ready. Meanwhile you can practise aptitude, ask Aria about any subject, or see the branches that are ready.

Confusion matrix, precision, recall and F1

Confusion matrix and the measures precision, recall (sensitivity) and F1 score: how to compute them, why they matter, worked numerical examples, common exam mistakes and practice questions.

Drafted with Aria, reviewed by the AiCanCode.org team · Updated . How we write and check our notes. Spotted an error? Use Give Feedback at the bottom of the page.

Why it matters

Engineers use the confusion matrix and the derived measures precision, recall and F1 score to evaluate classification models in real systems — for example, medical diagnosis, spam detection and fault detection. These metrics show different trade‑offs (false alarms vs missed detections) that accuracy alone hides, so they guide model selection and threshold choice in deployment.

Key ideas

Start with a binary classification problem: each sample has a true class (positive or negative) and a predicted class (positive or negative). We summarise predictions with a 2×2 table called the confusion matrix.

  • Define confusion matrix: a 2×2 table that counts predicted vs actual labels. (Convention used here — common in exams): rows are Actual class, columns are Predicted class. So draw a square divided into four cells. Label the top row "Actual = Positive", bottom row "Actual = Negative". Label the left column "Predicted = Positive", right column "Predicted = Negative".

    • Top-left cell: True Positive (TP) — number of samples that are actually positive and predicted positive.
    • Top-right cell: False Negative (FN) — actually positive but predicted negative (a miss).
    • Bottom-left cell: False Positive (FP) — actually negative but predicted positive (a false alarm).
    • Bottom-right cell: True Negative (TN) — actually negative and predicted negative.
  • Define precision (also called positive predictive value): the fraction of predicted positives that are actually positive. Precision answers: "When the model says positive, how often is it correct?"

  • Define recall (also called sensitivity or true positive rate): the fraction of actual positives that the model correctly finds. Recall answers: "Of all real positives, how many did the model detect?"

  • Define F1 score: the harmonic mean of precision and recall. It balances precision and recall into a single number; high F1 means both precision and recall are high. F1 does not use TN and therefore ignores the true negatives.

  • Why harmonic mean (not arithmetic)? The harmonic mean penalises large differences: if precision is high but recall low (or vice versa), F1 is closer to the smaller value.

  • When to prefer which metric:

    • Use precision when false positives are costly (e.g., diagnosing a healthy patient as sick causes expensive additional tests or stress).
    • Use recall when missing positives is costly (e.g., missing a disease case — you want high recall).
    • Use F1 when you need a single measure that balances both, especially on class‑imbalanced problems.
  • Notes for multiclass: compute class‑wise precision/recall/F1 and then average. Two common ways — micro-average (globally counts TP/FP/FN) and macro-average (average of class-wise metrics). See the topic "Cross-validation and micro and macro averaging" for details.

  • Diagrams to draw for exams: draw a labelled 2×2 table with axes "Actual (rows): Positive / Negative" and "Predicted (columns): Positive / Negative". Fill cells with TP, FN, FP, TN. Also annotate where precision = TP/(TP+FP) and recall = TP/(TP+FN).

Formulas

Accuracy = (TP + TN) / (TP + FP + FN + TN) — TP, FP, FN, TN are counts (unitless); applies to any classifier; shows overall correctness; unit: unitless proportion (0–1).

Precision = P = TP / (TP + FP) — TP = true positives (count), FP = false positives (count); precision is unitless (proportion); applies when TP+FP > 0; undefined if TP+FP = 0 (no predicted positives).

Recall = R = TP / (TP + FN) — FN = false negatives (count); recall (sensitivity) is unitless (proportion); applies when TP+FN > 0; undefined if TP+FN = 0 (no actual positives).

Specificity = TN / (TN + FP) — TN = true negatives (count); specificity (true negative rate) is unitless; useful complement to recall.

F1 = 2 · (P · R) / (P + R) — P = precision, R = recall; F1 is unitless (proportion); applies when P+R > 0; undefined if P+R = 0.

Fβ = (1 + β^2) · (P · R) / (β^2 · P + R) — general F-measure weighting recall by β; β > 1 favours recall, β < 1 favours precision; unitless.

F1 = 2 / (1/P + 1/R) — F1 expressed as harmonic mean of P and R; unitless.

Notes: All metrics are proportions (unitless). When reporting, convert to percentage (×100%) if required.

Worked examples

Example 1 — small clinical test (step-by-step):

Given: 100 patients tested for a disease. Actual positives = 30, actual negatives = 70. The classifier predicted 35 positives and 65 negatives. Of the predicted positives, 25 were true positives.

  1. Fill confusion matrix counts:

    • TP = 25 (given).
    • FP = predicted positives − TP = 35 − 25 = 10.
    • FN = actual positives − TP = 30 − 25 = 5.
    • TN = actual negatives − FP = 70 − 10 = 60.
  2. Formulas then numbers (unitless proportions):

    • Precision P = TP/(TP + FP) = 25/(25 + 10) = 25/35 = 0.7142857 → 71.43%.
    • Recall R = TP/(TP + FN) = 25/(25 + 5) = 25/30 = 0.8333333 → 83.33%.
    • F1 = 2·P·R/(P + R) = 2·0.7142857·0.8333333 / (1.547619) = 0.7692308 → 76.92%.
    • Accuracy = (TP + TN)/Total = (25 + 60)/100 = 85/100 = 0.85 → 85%.

Final answers (bold as required in exam): Precision = 71.43%, Recall = 83.33%, F1 = 76.92%, Accuracy = 85%.

Example 2 — imbalanced email spam example (shows accuracy can be misleading):

Given: 1000 emails; actual spam (positive) = 50, actual ham (negative) = 950. Model predicted 30 spam; among those 30, 20 were actual spam.

  1. Confusion matrix counts:

    • TP = 20, FP = 10 (30 predicted spam minus 20 TP), FN = 50 − 20 = 30, TN = 950 − 10 = 940.
  2. Compute metrics:

    • Precision = 20/(20 + 10) = 20/30 = 0.6666667 → 66.67%.
    • Recall = 20/(20 + 30) = 20/50 = 0.4 → 40.00%.
    • F1 = 2·0.6666667·0.4/(0.6666667 + 0.4) = 0.5 → 50.00%.
    • Accuracy = (20 + 940)/1000 = 960/1000 = 0.96 → 96.00%.

Final answers: Precision = 66.67%, Recall = 40.00%, F1 = 50.00%, Accuracy = 96.00%.

Comment: The high accuracy (96%) hides poor recall for the spam class; the F1 (50%) gives a clearer picture of classifier usefulness on the minority class.

Common mistakes

  • Using accuracy to judge models on imbalanced data — accuracy can be high even if the classifier never predicts the minority class.
  • Forgetting that F1 does not use TN, so it may not reflect true negative performance.
  • Not checking for division-by-zero when TP+FP = 0 or TP+FN = 0 (write "undefined" or handle as 0 by convention only if specified).
  • Reporting numbers without units (write unitless proportion or percentage) or incorrect rounding.
  • Using arithmetic mean instead of harmonic mean when combining precision and recall; thereby overestimating balanced performance.

For your exam

Kinds of questions:

  • Definitions: define confusion matrix, TP/FP/TN/FN, precision, recall, F1.
  • Short notes: trade-offs between precision and recall, when to prefer precision vs recall, interpretation of F1.
  • Derivations: show F1 is the harmonic mean of precision and recall; derive Fβ formula.
  • Numericals: given counts fill confusion matrix, compute precision/recall/F1/accuracy; explain results.
  • Diagrams: draw and label a confusion matrix, locate TP/FP/FN/TN and write formulas on the diagram.

What a full-marks answer includes:

  • Correctly labelled confusion matrix (Actual rows, Predicted columns unless the question says otherwise).
  • Accurate counts TP, FP, FN, TN.
  • Correct formulas used with all intermediate steps shown (substitute numbers, show division, convert to percentage if asked).
  • Brief interpretation: comment on which errors matter, mention class imbalance if relevant, and recommend whether precision or recall should be prioritised.
  • Mention undefined cases (division by zero) if applicable.

Quick check

  1. Define precision in one sentence.
  2. If TP=40, FP=10, FN=20, compute precision, recall and F1 (percent values to two decimals).
  3. In which application would you prioritise recall over precision? Give one example.

Answers: 1) Precision = fraction of predicted positives that are correct (TP/(TP+FP)).; 2) Precision = 40/(40+10)=0.80 → 80.00%; Recall = 40/(40+20)=0.6666667 → 66.67%; F1 = 2·0.8·0.6666667/(0.8+0.6666667)=0.7272727 → 72.73%.; 3) Prioritise recall in disease screening (medical diagnosis) where missing a positive is costly.

Finished this topic? Mark it so your progress, study plan and readiness keep up.

Stuck on something here?