Questions you’ll actually get asked

Difference between supervised and unsupervised learning?

Supervised uses labelled data to learn input→output mappings; unsupervised finds structure in unlabelled data. Strong answer: LLM pretraining is self-supervised — the data labels itself using the next token as the target.

Your model gets 99% on training but 80% on test. What’s happening?

Overfitting — it memorised noise. Fixes: simplify the model, add regularisation, get more data, or use cross-validation. Naming several fixes is what separates a strong answer.

Why do we need a test set at all?

To estimate performance on data the model has never seen — the only thing that matters in production. Scoring on training data is dishonestly optimistic.

What’s the bias-variance tradeoff?

High bias = too simple, underfits. High variance = too complex, overfits. Total error ≈ bias² + variance + irreducible noise; you tune complexity to minimise the sum.

Train / validation / test — what is each for?

Train fits parameters. Validation chooses hyperparameters and detects overfitting during development. Test is a final, untouched estimate of generalisation. Touching test during tuning leaks information and inflates reported scores.

When is accuracy a bad metric?

Class imbalance — a model that always predicts the majority class can look accurate and be useless. Prefer precision/recall/F1, ROC-AUC, or PR-AUC depending on the cost of false positives vs false negatives.

Difference between bagging and boosting?

Bagging (random forest) trains many models in parallel on random subsets and averages them — reduces variance. Boosting (XGBoost) trains models sequentially, each correcting the last’s errors — reduces bias, usually higher accuracy but needs careful tuning.

Why is it called logistic regression if it classifies?

Because it regresses a linear function on the log-odds, then maps that to a probability with the sigmoid. The machinery is regression; the thresholded output makes it a classifier — and it’s the ancestor of neural-net softmax.

When would you pick a simple model over a deep neural net?

Tabular data, small datasets, when interpretability is required (finance, healthcare), tight compute/latency budgets, or when the baseline already performs well.

What’s feature importance and why does it matter?

A score of how much each input drove predictions. It gives interpretability — you can explain and debug decisions — which neural networks largely sacrifice. Crucial anywhere “why did the model decide this?” is a legal or trust requirement.

L1 vs L2 regularisation?

L2 shrinks weights smoothly (ridge). L1 can drive weights to exact zero (lasso) and does feature selection. Elastic net mixes both. In neural nets, weight decay is typically L2; dropout is a different regulariser.

What does an activation function do, and why do we need one?

It adds a non-linearity after the weighted sum. Without it, stacked layers collapse into a single linear transform and cannot model curved boundaries. ReLU is the common default: simple, sparse, and trains well.

Explain backpropagation in one breath.

Compute the loss, then use the chain rule to push gradients backward through each layer so every weight learns how much it contributed to the error — then take a small step downhill (gradient descent).

When would you still prefer gradient boosting over a neural net?

Structured/tabular data, smaller datasets, tight latency budgets, or when you need strong out-of-the-box baselines. Deep nets shine on unstructured, high-dimensional inputs where compositionality matters.

What is the vanishing gradient problem?

In very deep stacks, gradients can shrink toward zero as they flow backward, so early layers stop learning. Mitigations: ReLU, residual connections, better init, normalization — all of which show up in modern transformers.

What is a learning rate, and what goes wrong if it’s off?

Step size for each gradient update. Too high → loss diverges or oscillates. Too low → training crawls or stalls in poor minima. Practice: try a range, use schedules or adaptive optimizers (Adam), and watch train/val curves.

Batch norm vs dropout — what problem does each attack?

Batch norm stabilises activations across layers (faster, more stable training). Dropout randomly zeros units during training to reduce co-adaptation (regularisation). They are complementary, not substitutes.

Explain self-attention in plain English.

Each token builds a query, key, and value. The query of one token scores against keys of others to decide how much to “read” from each value. That lets the model mix information across the whole sequence without a fixed left-to-right state like an RNN.

What is a token? Why does tokenization matter?

A token is a subword unit the model actually consumes (not always a word). Vocab size and token boundaries affect cost (context length), multilingual behaviour, and weird edge cases (numbers, code, rare names).

Pretraining vs fine-tuning vs RLHF?

Pretraining: next-token prediction on huge corpora (general language knowledge). Supervised fine-tuning (SFT): imitate high-quality instruction/response pairs. RLHF / preference tuning: reward answers humans prefer (or approximate that with DPO). Each stage needs different data and fails differently.

RAG vs fine-tuning — when do you pick which?

RAG when knowledge changes often, must be cited, or is private docs — keep facts in a retriever. Fine-tune when you need style, format, tool-use patterns, or domain language baked into behaviour. Many production systems use both.

Why do LLMs hallucinate?

They predict plausible next tokens, not truth. Gaps in training data, bad retrieval, overconfident decoding, and prompts that force an answer all increase hallucinations. Mitigations: RAG with citations, abstention, constrained decoding, eval sets for factuality.

What do temperature and top-p control?

They reshape the next-token distribution. Higher temperature / more lenient top-p → more diverse (and often less reliable) text. Low temperature → peaked, deterministic answers. For factual tasks, keep them low; for brainstorming, raise carefully.

How would you evaluate an LLM feature?

Start with a fixed eval set of real user tasks; score exact metrics where possible (retrieval hit rate, format validity), plus human or rubric-based quality. Track online metrics (thumbs, escalation rate, latency, cost). Don’t rely on a single “LLM-as-judge” score without spot checks.

What is data leakage? Give an example.

When information from outside the training setup — especially the future or the test set — sneaks into fitting. Example: scaling features using the full dataset before the split, or including a feature that is a proxy for the label. Inflated metrics, broken production.

How do you handle class imbalance?

Better metrics first (not accuracy). Then: class weights, resampling, threshold tuning, or more minority data. Don’t jump to SMOTE before a weighted baseline and a clear cost matrix for errors.

Random split or time-based split?

If the world drifts over time (fraud, demand, news), a random split leaks the future into training. Use time-based splits to mimic production. Random is fine when i.i.d. is a fair assumption.

How do you know a model is rotting in production?

Monitor input distributions (data drift), prediction distributions, downstream business metrics, and latency/errors. Compare against a shadow baseline. For LLMs, also track retrieval quality, refusal rate, and eval-set regressions on each prompt/model change.

What do you log for a production ML / LLM service?

Request IDs, model/prompt versions, latency, token/cost counts, inputs/outputs (with privacy redaction), retrieval IDs for RAG, and user feedback. You can’t debug or improve what you didn’t version.

Blue/green or canary for model rollout — why?

Never ship a new model to 100% traffic blind. Canary a small %, compare metrics to the champion, then promote or roll back. Same discipline for prompt changes.

Design a RAG customer-support bot for a SaaS product.

Clarify languages, SLA, and when to escalate to humans. Ingest docs → chunk → embed → vector index; retrieve top-k with metadata filters; generate with citations; eval set of real tickets; guardrails for PII; monitor deflection rate, CSAT, and hallucination complaints; version prompts and indexes.

Design semantic search over a company’s internal documents.

Access control first. Hybrid search (keyword + dense vectors) usually beats either alone. Chunking strategy, embedding model choice, re-ranker, freshness pipeline, and an eval of ranked relevance (nDCG / manual labels). Latency and permission checks at query time are part of the design.

Fine-tune, prompt, or RAG for a domain copilot?

Default: prompt + tools + RAG for facts. Fine-tune when style/format/tool-calling must be reliable and stable. Estimate data availability, change frequency of knowledge, privacy, and cost. Strong answers compare options explicitly instead of picking one blindly.

Design a content-moderation pipeline.

Define policy taxonomy and severity. Combine cheap classifiers / filters with an LLM judge for edge cases; human review queue for appeals and calibration. Measure precision/recall per category, latency, and reviewer agreement. Plan for adversary shifts and multilingual content.