Software engineering and the software crisis
Overview of the software crisis: what it was, why it matters, the laws and metrics that describe it (Brooks, Lehman), causes, consequences and common solutions used in software engineering.
Drafted with Aria, reviewed by the AiCanCode.org team · Updated . How we write and check our notes. Spotted an error? Use Give Feedback at the bottom of the page.
Why it matters
The software crisis explains why many large software projects fail, are late, exceed budget, or deliver poor quality. Engineers use this topic to recognise systemic risks (scope growth, poor requirements, communication overhead) and to choose processes, architectures and estimations (see the later topics: software process framework and SDLC, prescriptive and evolutionary models, Agile, COCOMO and estimation techniques) that reduce those risks.
Key ideas
Start from first principles: building software is designing and implementing an abstract system that satisfies a set of requirements under constraints of time, cost and quality. Many failures are not technical but organisational.
Software engineering: the disciplined, systematic and quantifiable approach to the development, operation and maintenance of software. Definition assumption: there is an engineering process that can be improved by methods, tools and measurement. Common quality attributes to mention in exams: functionality, reliability, usability, efficiency (performance), maintainability, portability, and security.
Software crisis: the historical observation that software projects were frequently delivered late, over budget, buggy or unusable. Precise statement: the gap between user needs and the ability to deliver reliable software at scale. Assumption: complexity and scale of problems outgrew ad-hoc programming practices. Historical note: The term was popularised at the 1968 NATO Software Engineering conference.
Defect density: number of defects found per thousand lines of code (KLOC). Formal definition below in Formulas. Assumption: size in lines-of-code is an acceptable proxy for programme scale (this has limits; modern practice uses function points or story points).
Productivity (simple metric): amount of software produced per unit effort (often KLOC per person-month). Assumption: output measured in KLOC is meaningful (controversial in modern dev where value, not LOC, matters).
Brooks' law: "Adding manpower to a late software project makes it later." Exact statement: under typical conditions of software projects, increasing staff on a late project increases communication overhead and training time so schedule may slip further. Assumptions behind the law: tasks are not perfectly parallelisable; new staff require ramp-up and existing staff must divert time to train; there is non-zero interdependence between tasks. This is an empirical rule, not a mathematical theorem.
- To illustrate the assumption and derivation universities ask for: derive communication channels and show scaling cost:
- For n people, the number of potential communication channels is C = n*(n-1)/2. Drawn as a complete graph with n labelled nodes and an edge between every pair; label edges as "channel".
- As n increases, channels grow ~O(n^2). If coordination cost per channel is c, coordination effort ~ c * n*(n-1)/2, which for large n behaves like c*(n^2)/2. Thus communication overhead increases faster than linear staffing.
- To illustrate the assumption and derivation universities ask for: derive communication channels and show scaling cost:
Lehman's laws of software evolution: a set of empirical observations about evolving software systems. Exact statements (give each as a short law) and typical interpretation:
- Continuing change: A program that is used in a real-world environment must be continually adapted or it becomes progressively less useful. Assumption: requirements and environment change.
- Increasing complexity: Unless work is done to maintain or reduce complexity, program complexity increases as it evolves. Assumption: incremental fixes and feature addition without refactoring increase entropy.
- Self regulation: Evolution processes are self-regulating with time scales and distributions typical of the system. (Empirical observation.)
- Conservation of organizational stability (inertia): Average effective global activity rate is invariant over the product lifetime. (Interpretation: productivity per org tends to be stable.)
- Conservation of familiarity: Developers must maintain a certain understanding of the system; growth is limited by this familiarity. Assumption: human limits bound system growth.
- Continuing growth: Functionality must be continually increased to maintain user satisfaction.
- Declining quality: Unless rigorously maintained, perceived quality will be reduced over time. (Related to defect density rising without maintenance.)
- Feedback system: Evolution is multi-level and feedback-driven.
These are empirical laws: the university answer should state each law, its assumption, and an example showing its effect (e.g., increasing complexity leading to longer testing times).
Causes of the software crisis (simple list with short explanations):
- Poor requirements: incomplete, ambiguous or changing requirements produce rework.
- Underestimation: optimism bias and poor estimation methods.
- Communication overheads: large teams create coordination costs (see Brooks' law).
- Lack of modularity: monolithic designs increase coupling and maintenance cost.
- Inadequate testing and quality practices: late discovery of defects is expensive.
- Organisational and process issues: wrong incentives, unclear roles, changing priorities.
Solutions and responses (historical and modern):
- Introduce software process and discipline (SDLC models: waterfall, V-model, incremental, evolutionary models, Agile) — see later topics for detailed processes.
- Use modularity and architectural patterns to reduce coupling (see software architecture topics later).
- Invest in requirements engineering and validation (see requirements topics).
- Apply quantitative metrics and estimation (COCOMO, productivity, defect metrics) and continuous improvement (process measurements).
- Apply Agile practices to manage changing requirements; use automated testing and CI/CD to reduce integration risk (DevOps topics later).
Diagrams to draw in answers (describe exactly):
- Draw a complete graph with n nodes and label edges as "communication channels"; caption: "communication channels n*(n-1)/2". This supports Brooks' law.
- Draw a timeline with years on x-axis and two curves: (a) "system size / complexity" rising; (b) "ability of ad-hoc methods" flat or falling. Label the gap as "software crisis" and annotate causes (requirements, testing, team size).
Formulas
Channels = n*(n-1)/2
- n: number of people on the team (dimensionless). Applies when modelling pairwise communication in a team where everyone may need to talk to everyone.
Defect density = defects / KLOC
- defects: number of defects found (dimensionless count).
- KLOC: thousands of lines of code (KLOC = LOC / 1000). Unit: KLOC. Applies as a simple quality metric; suitable only as a rough gauge.
Productivity = KLOC / person-month
- KLOC: thousands of lines of code produced.
- person-month: one person working full-time for one month. Unit: KLOC per person-month. Applies as a crude productivity metric (use carefully; LOC has limitations).
Coordination effort ≈ c * n*(n-1)/2
- c: average cost per communication channel (person-month per channel or other effort unit).
- n: number of people. Unit depends on c. Used to reason about overhead when adding staff.
(There are many more refined models — e.g., COCOMO — covered later in the syllabus.)
Worked examples
Example 1 — numerical: compute defect density
Given: A module contains 25 defects found during testing. Its size is 40,000 lines of code. Compute defect density.
Steps:
- Convert LOC to KLOC: 40,000 LOC = 40 KLOC.
- Use formula: Defect density = defects / KLOC.
- Substitute: Defect density = 25 / 40 = 0.625 defects/KLOC.
Final answer: 0.625 defects per KLOC.
Example 2 — derivation used in an exam: explain Brooks' law with math
Question: Show why adding people to a late project can increase schedule by modelling communication overhead.
Model answer:
- Let n be original team size; channels = n*(n-1)/2.
- Let coordination cost be proportional to channels. When k new people are added, new size = n+k, channels' = (n+k)*(n+k-1)/2.
- Increase in channels ΔC = channels' - channels = [ (n+k)(n+k-1) - n(n-1) ] / 2.
- Expand numerator and simplify: ΔC = (2nk + k^2 - k)/2 ≈ nk + k^2/2 for moderate k.
- Coordination overhead therefore increases ~nk + k^2/2; the training cost and task reallocation add further delay. If these overheads exceed the parallelisable work saved by more people, the schedule increases.
- Conclusion: because coordination grows superlinearly, adding people to a late project can make it later. State assumptions: equal probability of communication between any two people, training cost non-zero, tasks not perfectly divisible.
Example 3 — short model answer: define the software crisis and list three causes and three remedies
Model answer (short-note style):
- Define: The software crisis is the persistent difficulty in delivering large software systems on time, within budget and with required quality.
- Causes: poor requirements, inadequate testing, and communication/coordination overheads in large teams.
- Remedies: adopt disciplined processes and requirements engineering, design for modularity and use automated testing and CI/CD to reduce integration risk.
Common mistakes
- Using LOC/KLOC metrics as absolute measures of productivity or quality without noting their limitations.
- Forgetting to state assumptions when applying the communication-channels derivation (e.g., that everyone may need to communicate with everyone).
- Listing causes or laws without explaining the mechanism (why they cause a crisis).
- Mixing up terms: calling the "software crisis" the same as "technical debt" without distinction (technical debt is one cause/manifestation, not the entire crisis).
For your exam
Types of questions you will see:
- Definitions (define software crisis, software engineering, defect density).
- Short notes (explain Brooks' law, Lehman's laws, causes of the software crisis).
- Derivations (derive communication channels and explain scheduling impact; simple calculations of defect density and productivity).
- Diagrams (draw the complete graph for communication channels; timeline highlighting crisis causes).
- Essays/long answers (discuss historical causes and how modern practices such as Agile and DevOps address them).
What a full-marks answer includes:
- Clear definitions with bold terms (in your answer write the defined term and a crisp definition).
- Exact statements of laws (Brooks' law, Lehman's laws) and their assumptions.
- A derivation or small calculation done step-by-step (show algebra for channels or numeric steps for defect density).
- A labelled diagram when requested (describe what you draw and label axes/nodes clearly).
- Short, practical examples or remedies linking to later topics (e.g., mention SDLC, Agile, CI/CD, architecture) while not re-teaching them.
Quick check
- What is the formula for the number of communication channels in a team of n people?
- Give a concise statement of Brooks' law.
- How do you compute defect density for a module with 12 defects and 6,000 LOC?
Answers: 1) Channels = n*(n-1)/2. 2) "Adding manpower to a late software project makes it later" — because of training and increased communication overhead (assume non-parallel tasks). 3) 6,000 LOC = 6 KLOC so defect density = 12 / 6 = 2 defects/KLOC.
Finished this topic? Mark it so your progress, study plan and readiness keep up.
Stuck on something here?