StudyBuddy: RAG Study Assistant with Cited, Evaluated Answers
Over nine phases you build StudyBuddy like a professional team: requirements, threats (prompt injection, cross-tenant leaks, invented citations) and user stories, a design with ADRs, then a test-first implementation of a complete retrieval-augmented generation pipeline: PDF and text ingestion with page-aware chunking, deterministic offline embeddings pinned by golden fixtures, hybrid retrieval in PostgreSQL (pgvector cosine search plus full-text search fused with reciprocal rank fusion), coverage-based abstention, extractive answers with page citations, an optional LLM whose answers are validated like user input, and streaming over server-sent events. Workspaces isolate tenants with SHA-256 hashed API keys. An evaluation harness measures recall, MRR and citation precision on every change, CI runs it in Docker, and a game day that makes the LLM hang leads to circuit breakers, degraded retrieval, daily quotas, metrics and a post-mortem. Pick Java, Python or TypeScript; the specs, golden files, tests, evaluation set and the chat UI are shared. Everything runs on your laptop with no API key; Ollama or a free Gemini key are optional.
What you will build, step by step
- Phase 0 · Setup and orientation
- Phase 1 · Requirements, threats and user stories
- Phase 2 · Design: RAG pipeline, schema, ADRs, API contract
- Phase 3 · Implementation in five milestones
- Phase 4 · Testing: PostgreSQL integration, tenant isolation and RAG evaluation
- Phase 5 · Continuous integration with GitHub Actions
- Phase 6 · Containerise, deliver and (optionally) deploy
- Phase 7 · Observability, a game day and resilience (v1.1.0)
- Phase 8 · Retrospective and portfolio