Problem statement

RAG went to staging. Legal asks about wrong citations; Support asks about stale docs. You document failure modes and mitigations—and complete the midterm checkpoint.

Outcomes

• Hybrid search / reranking / ACL-aware retrieval — conceptual.
• Two observed failure cases from Week 07 traces with mitigations.

Deliverables

1. rag_failure_modes.md — two failures + detection/mitigation.
2. MIDTERM.md — timed exercise (see file).

Links

• Graduate midterm (parallel): TPM-PM-AI-Graduate-Course.md (see section Midterm)
• Phase 4

• [ ] Hybrid search (BM25 + dense) — one blog or doc chapter.
• [ ] Indirect prompt injection via retrieved content — one OWASP or vendor note.

Time: ~2 h.

Task A — Failure analysis

From your Week 07 rag_trace.md, pick two weak answers. For each: root cause (chunking, stale, wrong retrieval), how you’d detect in prod, mitigation.

Save rag_failure_modes.md.

Task B — Midterm

Open MIDTERM.md and complete under stated time limit.

Submit: my-submissions/week-08/.

If you already completed MIDTERM.md, treat that as your Week 08 exam. Otherwise:

Alt exam (20 min)

1. Define indirect prompt injection with a RAG example.
2. Why is citation fidelity a product requirement, not only accuracy?
3. Name one benefit of hybrid search.

Self-grade: Phase 4 checklist.

Total: 90 minutes. Stop when time is up. Use only: SQLite + practice-env/data/cloudnote.db, schema from practice-env README, and blank paper/notes you wrote yourself.

Part A — Metric precision (30 min)

Rewrite these ambiguous questions into testable metric definitions (numerator, denominator, window, segment if needed):

1. “Did the AI sidebar hurt retention?”
2. “Are enterprise customers unhappy?”
3. “Is the product getting slower?”

Part B — SQL (45 min)

On cloudnote.db, write queries that:

1. Count distinct users with at least one checkout_complete event, by segment.
2. List top 3 event_name values by count in February 2025 (use event_ts text comparison >= '2025-02-01' AND < '2025-03-01' or equivalent).

Run queries and paste row counts or sample rows.

Part C — Short concepts (15 min)

Define in ≤ 2 sentences each: embedding, RAG, prompt injection, temperature.

Afterward: Grade using answer key you write tomorrow with references open; update weak topics.

Concepts & curated links: read the Learn chapter RAG first (topic path, not calendar-only). This file is the weekly practice bank — extra drills and a checklist on top of Readings → Practice → Exam.

1. Practice bank

2. Week checklist

• [ ] Failure analysis + midterm
• [ ] PRACTICE.md + EXAM.md