# Privacy-Preserving ML Interview Q&A Source: https://ai.techclick.in/blog_ai_privacy_ml_interview_qa Markdown: https://ai.techclick.in/blog_ai_privacy_ml_interview_qa.md Publisher: Techclick Infosec Pvt Ltd 37 senior-grade Privacy-Preserving ML interview questions with model answers: membership inference, differential privacy, DP-SGD, federated learning, HE… Privacy-Preserving ML Interview Q&A student learning map A visual study map for Privacy-Preserving ML Interview Q&A showing learning path, evidence, traps, and practice sequence. TECHCLICK STUDY MAP Privacy-Preserving ML Interview Q&A AI Security · learn the flow, prove with evidence, avoid unsafe shortcuts 1. Start Pick your weak spot — jump... 2. Understand Why this matters — a model is a... 3. Prove 1. Privacy Attacks on Models 4. Practice 2. Differential Privacy How to use this page First build the mental model, then answer with the flow, evidence, safe fix, and verification. Finish by testing yourself. Techclick Infosec Pvt Ltd | ai.techclick.in | Training Contact: WhatsApp +91 92772 29456 Content-specific feature visual for this lesson: use it as the 60-second map before reading the full detail. ## Why this matters — a model is a leaky diary that talks Think of a trained model as a diary that learned to talk. You never handed it a single record at query time, yet it can tell a stranger whether your medical row was in the training set, sketch a recognisable face from a classifier, or paste a customer's Aadhaar number into a completion. The data is gone, but its shadow is baked into the weights. Privacy-preserving ML is the discipline of shrinking that shadow. Interviewers probe this because it is where AI meets law. Under India's DPDP Act and the EU AI Act, a model that regurgitates PII is a reportable incident, not a curiosity. Panels want engineers who can name the attack, quantify the leak, and pick the right control — differential privacy, federated learning, encryption, or plain data minimisation — without guessing. Scenario · Sneha — junior ML security engineer at a Pune fintech Sneha is in round two for an AI security role. The panel lead asks: "We fine-tuned an LLM on support tickets. A user got another customer's PAN in a reply. What happened, and how do you stop it?" She freezes — she knows it is "leakage" but cannot name the mechanism or a fix. The fix is a mental model: training data leaves a fingerprint in the weights, and there are named attacks and named defences for it. Learn the chain — attack, measurement, control — and these questions become a script you can recite under pressure. ## 1. Privacy Attacks on Models Privacy attacks treat the model as the leak. You query it, watch its confidence, or read its gradients — and recover facts about people who were never meant to be exposed. The thread tying them together is overfitting : the more a model memorises individuals, the louder it leaks. Q1 What is a membership inference attack (MIA)? L1 A membership inference attack answers one question: was this exact record in the model's training set? The attacker feeds a candidate sample to the model and reads a signal — usually confidence or loss . Training members tend to get higher confidence and lower loss because the model has seen them. A classic recipe trains shadow models on data drawn from the same distribution, then trains an attack classifier to separate "member" from "non-member" by output behaviour. It sounds abstract until the dataset is sensitive. Confirming that someone's record was in an HIV-diagnosis or salary dataset is itself the privacy breach — membership is the secret. Member-vs-non-member from confidence/loss signal; shadow-model recipe; why membership alone is sensitive. Q2 Why do larger or overfit models leak more, and how do you measure MIA risk? L2 Leakage scales with memorisation . An overfit model fits individual training points instead of the general pattern, so members behave measurably differently from non-members — exactly the gap an attacker exploits. The bigger the train/test accuracy gap , the easier the attack. High-capacity models on small or duplicated data are the worst offenders. Measure it like a binary classifier. Run the attack and report AUC and, more honestly, true-positive rate at a low false-positive rate (TPR @ 0.1% FPR) — average accuracy hides the few records that leak badly. Priya at a Bangalore AI startup ran ART's MIA module against her churn model and found AUC 0.78; she traced it to a duplicated-row leak in the training data. Memorisation + train/test gap; report TPR@low-FPR not just AUC; tooling like ART. Q3 What is model inversion, and how does attribute inference differ from membership inference? L2 Model inversion reconstructs representative input features for a class or individual by optimising an input to maximise the model's confidence — the canonical demo recovers a blurry but recognisable face from a face-recognition model given only a name/label and API access. Attribute inference is narrower: the attacker already knows most of a record and uses the model to fill in a missing sensitive field (e.g., infer salary band from the rest). The distinction interviewers want: membership inference asks were you in the data ; attribute/inversion asks what is your sensitive value . Inversion reconstructs; attribute inference predicts a specific column. Inversion = reconstruct features by optimising input; attribute = fill a missing field; membership vs value distinction. Q4 Explain training-data extraction and memorisation in LLMs. L2 Large language models can verbatim memorise chunks of their training data, especially rare, high-entropy, or duplicated strings — exactly the shape of secrets like API keys, Aadhaar/PAN numbers, and email signatures. Extraction is the attack: prompt the model so it completes from memory, then filter for high-confidence, low-perplexity outputs that look like real records (Carlini et al. showed this on GPT-2 scale models and it gets worse with scale and duplication). Two facts to state: memorisation rises with model size and with how often a string was duplicated in training. So de-duplication and PII scrubbing of the corpus are first-line defences, not afterthoughts. Verbatim memorisation of rare/duplicated strings; prompt-then-filter extraction; dedup + scrub corpus. Q5 Your fine-tuned support LLM returned another customer's PAN. Walk through diagnosis and fixes. L3 First, name it: training-data memorisation surfacing as verbatim regurgitation — OWASP LLM02 (Sensitive Information Disclosure). Diagnose: was the PAN in the fine-tune corpus? Grep the dataset; reproduce with the triggering prompt; check if it is reachable by paraphrase. Fixes, layered: (1) scrub the corpus with PII detection (Presidio) and de-duplicate before retraining — root cause. (2) Add an output filter / DLP regex + Presidio on responses to catch PAN/Aadhaar patterns. (3) Consider DP fine-tuning (DP-SGD) to bound per-record influence. (4) Log the incident; under DPDP this is likely reportable. Order matters: remediate the data, then layer guardrails — guardrails alone do not un-memorise. Name LLM02 memorisation; reproduce; scrub+dedup corpus first; output DLP; DP fine-tune; treat as reportable. Q6 What is a reconstruction attack on gradients, and when is it feasible? L3 A gradient reconstruction attack (e.g., Deep Leakage from Gradients) recovers the actual training inputs — pixels, tokens — from the gradient updates a client shares, by optimising dummy data until its gradient matches the observed one. It is the reason "we only share gradients, not data" is a false sense of safety in federated learning. Feasibility is highest with small local steps , few samples per update , and no aggregation — a single client's raw gradient is the danger. It weakens fast with large aggregation groups, more local epochs, gradient clipping/noise, and secure aggregation that hides any one client's contribution. Aman at a Mumbai bank cited this to justify mandating secure aggregation before any federated rollout. Match dummy-data gradient to observed; worst with single client/small steps; aggregation + DP + secure-agg defeat it. Q7 Interviewer: "Is anonymising the training data enough to stop these attacks?" How do you answer? L3 No — and saying so cleanly is the point. Removing direct identifiers stops naive lookups but not statistical attacks. A model trained on "anonymised" data can still leak membership (was this row present) and support attribute inference , because the model learns correlations, not just the dropped name column. Re-identification via quasi-identifiers (PIN code + age + gender) is a separate, well-documented risk. The honest framing: anonymisation reduces the input sensitivity but gives no formal bound on what the trained model reveals. If you need a provable guarantee against these attacks, you reach for differential privacy , which bounds any single record's influence regardless of side knowledge. Anonymisation ≠ defence vs MIA/inference; no formal bound; DP is the provable answer. Legend untrusted / attacker trusted / corporate inspection / policy point the key "aha" node allowed Privacy attacks hit the training data, the model weights, and the API — each path has its own defence. Training data flows into a model served behind an API. Membership inference, model inversion, and model extraction each probe the API or weights; redaction, DP-SGD, and rate limiting defend each path. - The privacy attack surface Training data tickets, PAN, faces Model weights learned parameters Serving API /v1/predict Membership inference “was this row in training?” Model inversion reconstruct a training face Model extraction clone via query flood Defence: redact PII Presidio before training Defence: DP-SGD clip + noise = ε bound Defence: rate limit query budget + auth Guardrail: log + anomaly check Three attacks, one model. Look at how membership inference, model inversion, and extraction each probe a different point — and the matching defence sits on each path. ### Flip these before your interview 🔍 Membership inference tap to flip Attacker asks: was this exact record in training? High model confidence on it leaks the answer. So what: tells you the model memorised individuals. 🧩 Model inversion tap to flip Reconstructs a representative training input (a face, a record) from model outputs. So what: “no raw data shared” is not the same as private. 📜 Model extraction tap to flip Flood the API with queries to clone the model or steal its IP. So what: defend with auth, rate limits, and query budgets. 🎛️ Epsilon (ε) tap to flip The privacy budget in differential privacy — smaller means stronger. ε≈1 strong, ε≈8 typical. So what: always quote ε and δ together. 🤝 Federated learning tap to flip Each silo trains locally and shares only updates, never raw data. So what: gradients still leak — add secure aggregation plus DP. 🔒 HE vs TEE tap to flip Homomorphic encryption computes on ciphertext (slow); a TEE runs plaintext inside a sealed enclave (fast, trust the chip). So what: speed vs trust trade-off. Pause & Predict #1 Karthik at a Wipro project ships a health-risk API that returns full softmax confidences. A researcher shows he can tell, with high accuracy, whether a named patient's record was in the training set. Predict the cause and the single best control, and how to verify it works. Reveal answer The cause is a membership-inference leak driven by overfitting plus exposing full confidence scores: the model is far more confident on records it memorised, so the member/non-member gap is readable from the outputs. This is the classic shadow-model membership-inference attack (NIST AI 100-2 privacy attacks; ATLAS infer-training-membership), and verbose softmax outputs hand the attacker the exact signal they need. The single best control is to train with differential privacy (DP-SGD via TensorFlow Privacy / OpenDP) so no single record measurably changes the model, and to return only top-1 or rounded confidences. Verify by running a shadow-model membership attack against the new model and confirming the attacker's advantage falls close to random (about 50% AUC) while task accuracy stays acceptable. ## 2. Differential Privacy Differential privacy (DP) is the only widely-accepted formal privacy guarantee. It does not promise the output is useless to attackers — it promises that any one person's data barely changes the output, so an attacker cannot tell if you opted in. Everything here hinges on two numbers: epsilon and delta . Q8 Define differential privacy in plain terms. L1 Differential privacy guarantees that the result of an analysis is almost the same whether or not any single individual is in the dataset. Formally, for two datasets differing by one record, the probability of any output changes by at most a factor of e^epsilon (plus a small slack delta ). You add carefully calibrated random noise to achieve this. The plain-English promise: nothing an attacker can learn about you from the output could not also be learned if your record had never been included. That is a guarantee about the mechanism , holding even against attackers with unlimited side knowledge — which is why it beats anonymisation. Output ~same with/without one record; e^epsilon bound + delta; guarantee holds vs arbitrary side knowledge. Q9 What does epsilon actually mean, and what is a 'good' value? L2 epsilon is the privacy-loss budget — the maximum factor by which any output's probability can shift because of one person. Smaller epsilon = more noise = stronger privacy but lower utility. It is exponential, so epsilon 1 and epsilon 10 are worlds apart, not 10x apart. Rules of thumb interviewers like: epsilon ≤ 1 is strong, 1–3 is reasonable for many ML tasks, and double-digit epsilon offers weak practical guarantees (some industrial deployments quietly run epsilon in the tens). Always quote epsilon with delta and the unit — per-query, per-user, or per-training-run — because epsilon without scope is meaningless. Epsilon = privacy-loss budget, smaller=stronger, exponential; ≤1 strong; always state delta + unit/scope. Q10 What is delta, and why is (epsilon, delta)-DP weaker than pure DP? L2 delta is the probability that the clean epsilon bound fails entirely — a small chance the mechanism leaks more than promised. Pure DP is (epsilon, 0) ; approximate DP is (epsilon, delta) with delta tiny. The catch: if delta is too large, the mechanism could, with probability delta, output someone's full record and still satisfy the inequality. So set delta less than 1/N (smaller than the inverse of the dataset size), often 1e-5 or 1e-6 . (epsilon, delta)-DP is weaker but practically necessary — it is what lets the Gaussian mechanism and DP-SGD's composition math work cleanly. Delta = failure probability of the epsilon bound; set delta Q11 Explain DP-SGD: how do you make training differentially private? L2 DP-SGD adds two steps to ordinary SGD on each step. First, per-example gradient clipping : compute each sample's gradient and clip its L2 norm to a bound C , so no single record can dominate the update. Second, add Gaussian noise scaled to C (noise multiplier sigma) to the summed gradients before the optimiser step. Clipping bounds sensitivity ; the noise provides the privacy. A privacy accountant (RDP / the moments accountant, or PLD) tracks cumulative epsilon across all steps. Libraries: TensorFlow Privacy and Opacus . The cost is real — per-example gradients are memory-hungry and you trade accuracy for a lower epsilon. Per-example clip to C (bounds sensitivity) + Gaussian noise (privacy) + accountant; Opacus/TF-Privacy; cost. Q12 What is a privacy budget, and how does composition affect it across queries or epochs? L3 The privacy budget is your total allowable epsilon. Each DP query or training step spends some of it, and DP composes : privacy loss accumulates. Naive (basic) composition just adds epsilons — run the same epsilon-1 query 100 times and you are at epsilon 100. Advanced composition and tight accountants (RDP, PLD) grow the budget closer to sqrt(k) instead of k , which is why DP-SGD over thousands of steps stays usable. Operationally: fix a target epsilon up front, let the accountant tell you how many steps/queries you can afford, and once the budget is spent you stop releasing — you cannot keep answering for free. Karthik at a Hyderabad SOC enforced this as a hard gate in their analytics pipeline. Budget = total epsilon; composition accumulates; advanced/RDP ~sqrt(k); fix target, stop when spent. Q13 How do you reason about the privacy/utility trade-off when accuracy tanks under DP? L3 Lower epsilon means more noise means lower accuracy — that is the law, not a bug. Levers to recover utility at fixed epsilon: more data (noise is fixed per step, so signal-to-noise improves with N), larger aggregation groups (more samples per step amortise the noise), tuning the clip norm C (too small loses signal, too large needs more noise), and starting from a public pre-trained model so DP only covers the sensitive fine-tune. The senior move is to frame it as a budget negotiation: "At epsilon 3 we lose ~4 points of accuracy; is that acceptable to legal for this dataset?" — make the trade explicit rather than silently choosing. Noise vs accuracy is inherent; more data/larger groups/tune C/public pre-train; make trade-off explicit to stakeholders. Q14 When is DP NOT the right tool? L3 DP protects against per-individual leakage, so it is wrong when the thing you must protect is not an individual record. It does not stop model theft , prompt injection, or a correct-but-harmful inference about a whole group. It hurts badly on small datasets or rare classes, where the required noise destroys utility. And it does nothing for data-in-transit/at-rest confidentiality — that is encryption's job. It is also the wrong frame when you simply should not have collected the data: the answer there is data minimisation , not noisy training. State the boundary clearly — DP bounds what a released model/statistic reveals about one person, nothing more. DP only bounds per-record leak; useless for model theft, small data, confidentiality; minimise instead of noise. Q15 Contrast central DP vs local DP — who adds the noise, what is the trust model, and where do you deploy each? L2 The split is who you have to trust . In central DP , raw records reach a trusted curator/aggregator who computes the statistic or trains the model and adds the calibrated noise once, centrally. Less total noise for the same epsilon, so better utility — but everyone must trust that the curator never leaks the raw data. DP-SGD on a server you control is central DP. In local DP (LDP) , each user perturbs their own value on their device before it ever leaves, so the collector only ever sees noised data and there is no trusted curator . The canonical primitive is randomised response ("flip a coin, sometimes lie"), generalised by RAPPOR (Google's Chrome telemetry) and Apple's count-mean-sketch for emoji/typing stats. The cost of removing the trusted party is brutal noise: LDP error scales badly (roughly 1/(epsilon·sqrt(N)) ), so it only works at population scale for aggregate telemetry — not for training a high-fidelity model on a few thousand rows. The interview soundbite: central DP = better utility, must trust the curator; local DP = no trusted party, far worse utility, needs millions of users . The modern middle ground is the shuffle model (anonymise/shuffle LDP reports to amplify privacy), which buys back much of the utility — and distributed DP inside secure aggregation in FL is essentially the same idea. Central = trusted curator noises once, better utility; local = user noises on-device (randomised response/RAPPOR/Apple), no trusted party but huge noise, needs N at scale; shuffle model bridges them. Q16 DP fine-tuning a 7B LLM with LoRA/PEFT — how does it interact with DP-SGD per-example clipping and the accountant, and what utility hit at epsilon ≈ 8? L3 The reason this pairing works is that LoRA/PEFT shrinks the thing DP-SGD has to noise . DP-SGD's noise is added to the gradient and scaled to the clip norm C ; full fine-tuning of 7B parameters means an enormous, high-dimensional gradient and correspondingly destructive noise. LoRA freezes the base weights and trains only small low-rank adapter matrices (often 20 weak. Bounds how much one row changes output. Pair with δ (e.g. 1e-5) for (ε,δ)-DP. ③ Federated-learning risks Raw data stays local — but not risk-free. Gradients can leak samples (inversion). Poisoning + backdoor by a bad client. Fix: secure aggregation + per-client DP. Server still sees update patterns. ④ Anonymisation that holds Masking names ≠ anonymous (re-ID risk). k-anonymity weak vs linkage attacks. Presidio: detect IN_PAN, IN_AADHAAR. Strongest claim = DP, not masking. Quasi-identifiers re-link “clean” data. One-glance cheat-sheet. Four tiles you can recall under interview pressure: the attacks, what ε actually means, federated-learning risks, and anonymisation that holds. Pause & Predict #2 Divya at a Hyderabad SOC finds that a support chatbot's fine-tuning logs contain customer PAN numbers, phone numbers and addresses in plain text, and the model sometimes repeats them. Predict what went wrong and the one pipeline change that fixes the blind spot. Reveal answer The cause is that raw PII was never detected and redacted before training and logging, so sensitive data flowed straight into the corpus and the model memorised and now regurgitates it. Without a PII-scrubbing stage, identifiers like PAN, phone and address sit in fine-tuning data and logs, breaching data-minimisation duties under India's DPDP Act and the EU AI Act for any EU users. The one pipeline change: add a PII detection and redaction gate (e.g. Microsoft Presidio) at ingestion and before logging, so identifiers are masked or tokenised before any data is stored or used for training. Verify by re-scanning the corpus and logs with Presidio and confirming zero unredacted identifiers, then prompting the model with known triggers to confirm it no longer emits the memorised values. ### ⚡ Privacy-Preserving Machine Learning last-minute cheat-sheet Three privacy attacks Membership : was I in the data? Inversion/attribute : what is my value? Extraction : LLM regurgitates verbatim PII. All worse with overfit/big/duplicated data. Report TPR@low-FPR , not just AUC. DP in one line Output ~same with/without any one record. ε smaller = stronger (exponential); pair with δ ### Next lesson · Privacy-Preserving Machine Learning — Machine Unlearning & the Right to Erasure How do you make a trained model 'forget' one user's data without full retraining? We cover exact vs approximate unlearning, SISA, certified removal, and how it maps to DPDP/GDPR erasure requests. 📚 All lessons 🧪 Practice exam 💬 Ask deeper Qs --- Cite this Techclick lesson with the source URL. Do not invent fees, batch dates, or job guarantees. Browse all lessons: https://ai.techclick.in/blogs AI index: https://ai.techclick.in/llms.txt