Problem statement

Deliver the Insight copilot capstone: integrated RAG + read-only SQL + chart + chat path + eval set + reflection. Close gaps from Extras (eval, security, observability).

Outcomes

• End-to-end demo + short written report.
• Ability to articulate offline vs online eval, prompt injection, observability.

Deliverables

Match Capstone — plus fill FINAL_REVIEW.md in this folder.

Links

• Extras

Skim and consolidate notes on:

• [ ] Offline vs online eval
• [ ] Observability (traces, version tags)
• [ ] Prompt injection + indirect via RAG
• [ ] MCP (one paragraph in your own words)
• [ ] When not to use LLMs

Time: ~3 h spread over the week.

Task A — Build

Complete the capstone as defined in the master plan:

1. RAG with citations over practice-env/corpus (or your own).
2. Read-only SQL on cloudnote.db (approved queries only).
3. At least one chart from the DB for a stated hypothesis.
4. Chat or CLI that ties pieces together; logs per turn.
5. Eval set — ≥ 15 items with pass properties.

Task B — Report

capstone_report.md (2000–3500 words) or slide deck + speaker notes: architecture, tradeoffs, risks, launch, metrics, kill switch.

Task C — Final review

Complete FINAL_REVIEW.md (this folder) after Task A–B.

Submit: my-submissions/week-14/capstone/ (code + report + eval JSON/MD).

60 minutes. Treat as oral board prep: speak answers aloud, then write summaries.

Section A — System design (20 min)

Draw on paper the data flow for your capstone: user → UI → retrieval → model → tools → logs → metrics. Photograph or redraw in Mermaid.

Section B — Failure injection (15 min)

For your capstone, describe behavior for: empty retrieval, wrong SQL from tool, model refuses, API 429.

Section C — Launch (15 min)

List launch criteria (numeric or checklist) and rollback steps.

Section D — Meta (10 min)

What one topic from Extras do you want to deepen next quarter—why?

Pass: Coherent flow, concrete mitigations, honest meta. Compare Capstone checklist + Extras.

Instructions: Answer from memory first; then open notes and mark gaps.

Concepts (short answers)

1. Tokens vs context window — difference in one sentence.
2. When RAG vs fine-tuning — one sentence each use case.
3. Why SQL stays source of truth in analytics copilots.
4. Two RAG failure modes unrelated to “bad GPT.”
5. Three log fields you’d require in prod chat.
6. Human approval — give one action that always needs it in your capstone.
7. Indirect prompt injection — example.
8. Offline vs online eval — one sentence each.

Intuition checks

• What would you check first if P95 latency regressed after a prompt change?
• What metric proves citation quality—not BLEU?

Self-grade

• [ ] I can whiteboard the full capstone in 10 minutes.
• [ ] I can explain tradeoffs to a VP without jargon walls.
• [ ] I listed three things I’d still study next month.

Date completed: ___

Concepts & curated links: read the Learn chapter Agents & launch first (topic path, not calendar-only). This file is the weekly practice bank — extra drills and a checklist on top of Readings → Practice → Exam.

1. Practice bank

2. Week checklist

• [ ] Capstone demo + report
• [ ] PRACTICE.md + EXAM.md + Final review