ChurnCast: End-to-End Data Science Case Study on Subscription Churn

TiffinBox, a meal-subscription startup, loses about 9 % of its paying customers every month, and its retention team can call only 10 % of them with an offer. Over nine phases you build the churn model they need, like a professional data science team: frame the decision and the label, write a data contract, a timeline and a modelling plan with ADRs, then build the pipeline test-first in your track: load four deliberately messy CSVs as text, apply documented cleaning rules that count every row they drop, check the contract, build point-in-time features that only use what was known at each month-end snapshot (and avoid the leaky cancellation_reason column), compare two baselines (the training churn rate and the team's five-flag rule) with two logistic regressions using grouped 5-fold cross-validation, then evaluate the champion once on out-of-time test months: AUC, average precision, Brier, calibration (ECE), precision and recall of the top-10 % call list, net value per month, slices, an age-band fairness check and a leakage scan, all behind a release gate. Golden files, invariants, leakage and contract-violation tests pin every number, and the Python, SQL and R tracks must agree (features to 1e-9, metrics and coefficients to 1e-6). CI executes the notebook or R Markdown report, runs the gate and checks parity; a generated model card and a FastAPI prediction API with an optional AI explainer (numbered facts in, guardrails out, a deterministic template as fallback) ship in Docker as v1.0.0. Then the model meets the real world: a new acquisition channel breaks the contract, prices change, and the deployed model's calibration slips. You version the contract, add drift (PSI) and realised-performance monitoring, recalibrate and retrain a challenger, decide champion vs challenger on a holdout month, add request ids, JSON logs and Prometheus metrics to the API, and write a blameless post-mortem (v1.1.0). Pick Python (pandas, scikit-learn, Jupyter), SQL (DuckDB) or R (tidymodels, R Markdown); the data, contract, golden files and API are shared. Everything runs on a laptop with no paid service and no API key.

intermediatePythonSQLRpandasDuckDBtidymodelsscikit-learnJupyterR MarkdownFastAPIDockerGitHub Actions

What you will build, step by step

  1. Phase 0 · Setup and the brief
  2. Phase 1 · Problem framing: the decision, the label and success criteria
  3. Phase 2 · Design: data contract, timeline, modelling plan and ADRs
  4. Phase 3 · Implementation in five milestones: clean, check, features, models, evaluation
  5. Phase 4 · Testing data and models: golden files, invariants, leakage, parity
  6. Phase 5 · CI with GitHub Actions and the evaluation gate
  7. Phase 6 · Model card, prediction API, AI explainer, Docker and delivery
  8. Phase 7 · Monitoring, drift, retraining and a post-mortem
  9. Phase 8 · Retrospective and portfolio