Design the system — then defend it

• The 6-box design framework
• Example: RAG customer-support bot
• Example: fraud scoring service
• Prompt vs RAG vs fine-tune
• Practice prompts
• Interview strategy
• Interview questions

The 6-box design framework

Use this every time. Aim for a structured 20–30 minute whiteboard:

1. Requirements   users, SLA, latency, cost, languages, compliance
2. Data           sources, labels, privacy, freshness, access control
3. Model/prompt   baseline → candidates → why this choice
4. Eval           offline metrics, golden sets, human review
5. Serve          batch vs online, API shape, rollouts
6. Monitor        drift, quality, feedback, retrain / reindex triggers

State assumptions out loud. Ask clarifying questions before diving into architecture — seniors listen for that.

Example: RAG customer-support bot

• Requirements — deflect tier-1 tickets; escalate when unsure; cite docs; p95 latency budget; PII-safe logs.
• Data — help center + internal runbooks; chunk by heading; ACL filters.
• Model — managed LLM + embeddings; hybrid retrieval; prompt forces citations and abstention.
• Eval — 50 real tickets with gold answers; retrieval hit@k; faithfulness; escalation precision.
• Serve — API behind the help desk; canary prompt versions.
• Monitor — thumbs, escalation rate, empty-retrieval rate, cost/ticket, index age.

Example: fraud scoring service

• Requirements — score each payment in <X ms; review queue capacity; cost matrix for FP/FN.
• Data — transactions + delayed fraud labels; stratified/time splits; no post-outcome features.
• Model — logistic baseline → gradient boosting; calibrate probabilities; threshold for precision at target recall.
• Eval — PR-AUC, precision@k, cost-weighted utility; slice by merchant/geo.
• Serve — online API; feature parity with training; model version header.
• Monitor — score distribution shifts, null spikes, queue size, late labels.

This is the system design behind FraudWatch.

Prompt vs RAG vs fine-tune

Need fresh / private / citable facts?     → RAG (+ tools)
Need style / format / tool-calling habit? → fine-tune (or strong few-shot)
Need a quick behaviour tweak?             → prompt change + eval
Unsure?                                   → prompt + RAG first; fine-tune when metrics plateau

Strong answers compare options explicitly — cost, latency, data needs, ownership — instead of picking one blindly.

Practice prompts

Sketch each with the 6-box framework (also in the interview bank):

• Semantic search over internal documents (with ACLs)
• Content moderation pipeline
• Document Q&A over private PDFs
• Recommendation feed (candidate generation → rank → re-rank)
• FilingPilot: Q&A over SEC filings with numeric faithfulness checks
• LoadCast: demand forecasting with drift monitors

Interview strategy

• Lead with a story — FraudWatch / FilingPilot beats generic tutorials.
• Name tradeoffs — precision vs recall, RAG vs fine-tune, latency vs depth.
• Admit uncertainty — then say how you’d measure the way forward.
• Draw the diagram — data → model → eval → serve → monitor.
• Drill weekly — five questions from the bank, out loud, timed.

Interview questions