Computer Engineering is coming soon

We're still writing and reviewing the notes for every subject, and we'll open it when every subject is ready. Meanwhile you can practise aptitude, ask Aria about any subject, or see the branches that are ready.

Software testing strategies

Study notes on software testing strategies covering definitions, test levels and types, design techniques, metrics, worked examples, common exam mistakes and practice questions.

Drafted with Aria, reviewed by the AiCanCode.org team · Updated . How we write and check our notes. Spotted an error? Use Give Feedback at the bottom of the page.

Why it matters

Software testing strategies guide engineers on how to plan, design and run tests so software meets requirements and is reliable in the field. Good strategies save time, reduce faults in released systems and help choose which tests to run when resources are limited.

Key ideas

Start from first principles: software is built to satisfy requirements. A test checks whether some requirement or property holds. Define terms before using them.

  • Test: an activity that executes software or examines it to determine whether it satisfies a specified requirement. (Used to find faults.)
  • Fault (bug, defect): a mistake in the code or design that can cause incorrect behaviour. Define clearly in answers: error → human action that produces an incorrect result (cause of faults); fault → the defect in code or design; failure → externally visible incorrect behaviour observed when a fault is executed.
  • Test case: a set of inputs, execution preconditions, expected results and execution procedure for a particular objective.
  • Test suite: a collection of test cases grouped for running together.
  • Test oracle: a mechanism or principle used to decide whether a test outcome is correct.

Testing is often organised by levels (draw and label the V-model — instructions below).

  • Unit testing: tests a single module/class in isolation. (Use stubs/mocks for collaborators.)
  • Integration testing: tests interactions between modules. Two common styles: top-down integration and bottom-up integration; also incremental and big-bang approaches. Tip: top-down typically uses stubs for lower-level components; bottom-up uses drivers for higher-level callers.
  • System testing: tests the complete integrated system against system requirements.
  • Acceptance testing: tests with end-user or customer to decide acceptance; includes alpha and beta.

Testing is also organised by approach:

  • Black-box testing (functional testing): design tests from requirements and specifications without looking at internal code.
  • White-box testing (structural testing): design tests to exercise internal code structure (statements, branches, paths). Common coverage criteria: statement coverage (percentage of executed statements), branch/decision coverage (percentage of decision outcomes executed at least once), condition coverage (each boolean sub-expression evaluated to true and false), path coverage (execute distinct feasible paths), and MC/DC (each condition shown to independently affect the decision outcome).
  • Grey-box testing: partial internal knowledge used when designing tests.

Key test design techniques (define each before using):

  • Equivalence partitioning: partition input domain into classes expected to behave the same. Choose one representative from each partition. Include both valid and invalid partitions.
  • Boundary value analysis: choose test inputs at, just below and just above the boundaries of partitions.
  • Decision table testing: table of conditions and actions; derive test cases to cover rows (rules).
  • State transition testing: model system states and events; create tests for transitions and sequences.
  • Pairwise and combinatorial testing: choose value combinations to cover interactions up to a specified interaction strength.

Non-functional and special testing types:

  • Performance testing (load, stress, soak): checks speed, throughput and scalability.
  • Security testing: tests for vulnerabilities and unauthorized access.
  • Usability testing: checks user interface and experience.
  • Reliability testing: measures failure rates, mean time to failure.

Risk-based and pragmatic testing:

  • Risk-based testing: allocate test effort based on likelihood and impact of failures. State the assumption: we can estimate risk and concentrate on highest-risk areas.
  • Regression testing: re-run tests after changes to ensure no new faults introduced.

Static vs dynamic testing:

  • Static testing: reviews, inspections and static analysis tools that examine code or documents without executing the program.
  • Dynamic testing: executing the program and checking behaviour.

Test automation and tooling: automated tests are essential for regression and continuous integration (refer to CI/CD topic). Unit tests are usually automated with frameworks like xUnit; system tests often use test harnesses, mocks and simulated environments.

Diagrams to draw and label for exams:

  • The V-model: Draw a large letter V. Left arm (top to bottom): Requirements specification → System design → Architectural design → Module design. Right arm (bottom to top): Unit testing (maps to Module design) → Integration testing (maps to Architectural design) → System testing (maps to System design) → Acceptance testing (maps to Requirements). Label arrows showing mapping between each development activity and its corresponding test level.

Exact statements useful in answers:

  • "Verification answers 'Are we building the product right?'; Validation answers 'Are we building the right product?'" Define verification and validation before using them.
  • "100% coverage by a single coverage criterion does not guarantee absence of defects" — always state the limitation and reason.

Formulas

Test coverage (%) = (Number of covered items / Total number of items) × 100

  • Covered items and total items depend on criterion (statements, branches, requirements).
  • Unit: percent (%). Applies when measuring a coverage criterion.

Defect density = Number of defects found / Size of software

  • Size often in KLOC (thousand lines of code) or function points; specify the phase/time window the defects are counted for (e.g., during system testing or overall within a defined post-release period).
  • Unit: defects/KLOC.

Test effectiveness (%) = (Defects found before release / Total defects found) × 100

  • Total defects found = defects found before release + defects found after release (within measured window).
  • Unit: percent (%).

xUnit style test skeleton (syntax to remember for unit testing frameworks):

  • JUnit (Java): @Test public void testName() { /* arrange-act-assert */ }
  • pytest (Python): def test_name(): assert function_under_test(args) == expected (These are minimal patterns showing arrange-act-assert.)

Worked examples

  1. Numerical: defect density and test effectiveness Given: a module with 12,000 lines of code (LOC). During system testing 18 defects are found. After release, users report 6 defects in the first month. Calculate defect density (per KLOC) and test effectiveness. Steps:
  1. Size = 12,000 LOC = 12 KLOC.
  2. Defects found before release = 18. Defects after release = 6. Total defects = 24.
  3. Defect density = defects / KLOC = 24 / 12 = 2 defects/KLOC. Here we use total defects in the measured window (pre-release + first month post-release); state your window in answers. Answer: 2 defects/KLOC.
  4. Test effectiveness = (defects found before release / total defects) × 100 = (18 / 24) × 100 = 75%. Answer: 75%.
  1. Boundary value analysis example (algorithmic trace) Problem: Input field "age" must be an integer in range 18..60 inclusive. Boundaries: lower = 18, upper = 60. Choose test inputs per boundary value analysis:
  • Just below lower: 17 (expect: reject)
  • At lower: 18 (expect: accept)
  • Just above lower: 19 (expect: accept)
  • Just below upper: 59 (expect: accept)
  • At upper: 60 (expect: accept)
  • Just above upper: 61 (expect: reject) Trace for each: show input → validate → expected outcome. For exams, list the six tests with expected results and any preconditions (e.g. input type integer).
  1. Model answer (short essay) — design a test strategy for an online bookstore for final-year exam Include: testing objectives, test levels (unit, integration, system, acceptance), test types (functional — search, checkout; non-functional — load at peak hours, security for payments), test design techniques (equivalence classes for item quantity, boundary values for payment amount), regression plan (automated unit tests + nightly integration tests), risk-based focus (checkout and payment most critical), metrics (test coverage, defect density), tools (JUnit/pytest for unit, Selenium for UI, JMeter for load) and mapping to V-model. End with a short schedule: unit tests by developers, integration by dev-team, system tests in test environment, user acceptance with pilot users. A full-mark answer mentions preconditions, test data, environment and measures of success.

Common mistakes

  • Writing test cases without expected results or without preconditions.
  • Believing high code coverage equals no bugs — forgetting coverage is only for measured items.
  • Forgetting to test boundary and invalid inputs (only testing normal cases).
  • Not specifying test environment/configuration (OS, DB) in test plans.
  • Mixing up integration approaches (top-down vs bottom-up) and their use of stubs/drivers.
  • Using absolute rather than relative metrics without stating size units (e.g., saying defect density = 2 without units).

For your exam

Kinds of questions: define terms; short notes on types of testing; draw and explain V-model; design test cases using BVA/equivalence; compute defect density and test coverage; propose testing strategy for a system; compare black-box and white-box testing.

What a full-marks answer includes:

  • Clear definitions of terms used.
  • If a diagram is requested (e.g., V-model), draw it, label development activities and matching test levels, and explain mappings.
  • List of test cases or steps with preconditions and expected results for test-case questions.
  • For metrics, show formulas, units and step-by-step calculation.
  • For strategy questions, mention risk-based prioritisation, automation, tools, types of tests, entry/exit (completion) criteria, requirements traceability (RTM), and metrics to measure success.

Quick check

  1. Define "test oracle" in one sentence.
  2. Give boundary-value test cases for input range 1..100.
  3. What is the main goal of regression testing?

Answers: 1) A test oracle is a mechanism that determines whether a program output for a test is correct. 2) Tests: 0 (below lower, expect reject), 1 (lower, expect accept), 2 (just above lower, expect accept), 99 (just below upper, accept), 100 (upper, accept), 101 (above upper, reject) — typically six tests for BVA. 3) Regression testing aims to detect new faults introduced by recent changes.

Finished this topic? Mark it so your progress, study plan and readiness keep up.

Stuck on something here?