Privacy-Preserving ML Interview Q&A

A membership inference attack answers one question: was this exact record in the model's training set? The attacker feeds a candidate sample to the model and reads a signal — usually confidence or loss. Training members tend to get higher confidence and lower loss because the model has seen them. A classic recipe trains shadow models on data drawn from the same distribution, then trains an attack classifier to separate "member" from "non-member" by output behaviour.

It sounds abstract until the dataset is sensitive. Confirming that someone's record was in an HIV-diagnosis or salary dataset is itself the privacy breach — membership is the secret.

Leakage scales with memorisation. An overfit model fits individual training points instead of the general pattern, so members behave measurably differently from non-members — exactly the gap an attacker exploits. The bigger the train/test accuracy gap, the easier the attack. High-capacity models on small or duplicated data are the worst offenders.

Measure it like a binary classifier. Run the attack and report AUC and, more honestly, true-positive rate at a low false-positive rate (TPR @ 0.1% FPR) — average accuracy hides the few records that leak badly. Priya at a Bangalore AI startup ran ART's MIA module against her churn model and found AUC 0.78; she traced it to a duplicated-row leak in the training data.

Model inversion reconstructs representative input features for a class or individual by optimising an input to maximise the model's confidence — the canonical demo recovers a blurry but recognisable face from a face-recognition model given only a name/label and API access.

Attribute inference is narrower: the attacker already knows most of a record and uses the model to fill in a missing sensitive field (e.g., infer salary band from the rest). The distinction interviewers want: membership inference asks were you in the data; attribute/inversion asks what is your sensitive value. Inversion reconstructs; attribute inference predicts a specific column.

Large language models can verbatim memorise chunks of their training data, especially rare, high-entropy, or duplicated strings — exactly the shape of secrets like API keys, Aadhaar/PAN numbers, and email signatures. Extraction is the attack: prompt the model so it completes from memory, then filter for high-confidence, low-perplexity outputs that look like real records (Carlini et al. showed this on GPT-2 scale models and it gets worse with scale and duplication).

Two facts to state: memorisation rises with model size and with how often a string was duplicated in training. So de-duplication and PII scrubbing of the corpus are first-line defences, not afterthoughts.

First, name it: training-data memorisation surfacing as verbatim regurgitation — OWASP LLM02 (Sensitive Information Disclosure). Diagnose: was the PAN in the fine-tune corpus? Grep the dataset; reproduce with the triggering prompt; check if it is reachable by paraphrase.

Fixes, layered: (1) scrub the corpus with PII detection (Presidio) and de-duplicate before retraining — root cause. (2) Add an output filter / DLP regex + Presidio on responses to catch PAN/Aadhaar patterns. (3) Consider DP fine-tuning (DP-SGD) to bound per-record influence. (4) Log the incident; under DPDP this is likely reportable. Order matters: remediate the data, then layer guardrails — guardrails alone do not un-memorise.

A gradient reconstruction attack (e.g., Deep Leakage from Gradients) recovers the actual training inputs — pixels, tokens — from the gradient updates a client shares, by optimising dummy data until its gradient matches the observed one. It is the reason "we only share gradients, not data" is a false sense of safety in federated learning.

Feasibility is highest with small local steps, few samples per update, and no aggregation — a single client's raw gradient is the danger. It weakens fast with large aggregation groups, more local epochs, gradient clipping/noise, and secure aggregation that hides any one client's contribution. Aman at a Mumbai bank cited this to justify mandating secure aggregation before any federated rollout.

No — and saying so cleanly is the point. Removing direct identifiers stops naive lookups but not statistical attacks. A model trained on "anonymised" data can still leak membership (was this row present) and support attribute inference, because the model learns correlations, not just the dropped name column. Re-identification via quasi-identifiers (PIN code + age + gender) is a separate, well-documented risk.

The honest framing: anonymisation reduces the input sensitivity but gives no formal bound on what the trained model reveals. If you need a provable guarantee against these attacks, you reach for differential privacy, which bounds any single record's influence regardless of side knowledge.

Differential privacy guarantees that the result of an analysis is almost the same whether or not any single individual is in the dataset. Formally, for two datasets differing by one record, the probability of any output changes by at most a factor of e^epsilon (plus a small slack delta). You add carefully calibrated random noise to achieve this.

The plain-English promise: nothing an attacker can learn about you from the output could not also be learned if your record had never been included. That is a guarantee about the mechanism, holding even against attackers with unlimited side knowledge — which is why it beats anonymisation.

epsilon is the privacy-loss budget — the maximum factor by which any output's probability can shift because of one person. Smaller epsilon = more noise = stronger privacy but lower utility. It is exponential, so epsilon 1 and epsilon 10 are worlds apart, not 10x apart.

Rules of thumb interviewers like: epsilon ≤ 1 is strong, 1–3 is reasonable for many ML tasks, and double-digit epsilon offers weak practical guarantees (some industrial deployments quietly run epsilon in the tens). Always quote epsilon with delta and the unit — per-query, per-user, or per-training-run — because epsilon without scope is meaningless.

delta is the probability that the clean epsilon bound fails entirely — a small chance the mechanism leaks more than promised. Pure DP is (epsilon, 0); approximate DP is (epsilon, delta) with delta tiny.

The catch: if delta is too large, the mechanism could, with probability delta, output someone's full record and still satisfy the inequality. So set delta less than 1/N (smaller than the inverse of the dataset size), often 1e-5 or 1e-6. (epsilon, delta)-DP is weaker but practically necessary — it is what lets the Gaussian mechanism and DP-SGD's composition math work cleanly.

DP-SGD adds two steps to ordinary SGD on each step. First, per-example gradient clipping: compute each sample's gradient and clip its L2 norm to a bound C, so no single record can dominate the update. Second, add Gaussian noise scaled to C (noise multiplier sigma) to the summed gradients before the optimiser step.

Clipping bounds sensitivity; the noise provides the privacy. A privacy accountant (RDP / the moments accountant, or PLD) tracks cumulative epsilon across all steps. Libraries: TensorFlow Privacy and Opacus. The cost is real — per-example gradients are memory-hungry and you trade accuracy for a lower epsilon.

The privacy budget is your total allowable epsilon. Each DP query or training step spends some of it, and DP composes: privacy loss accumulates. Naive (basic) composition just adds epsilons — run the same epsilon-1 query 100 times and you are at epsilon 100. Advanced composition and tight accountants (RDP, PLD) grow the budget closer to sqrt(k) instead of k, which is why DP-SGD over thousands of steps stays usable.

Operationally: fix a target epsilon up front, let the accountant tell you how many steps/queries you can afford, and once the budget is spent you stop releasing — you cannot keep answering for free. Karthik at a Hyderabad SOC enforced this as a hard gate in their analytics pipeline.

Lower epsilon means more noise means lower accuracy — that is the law, not a bug. Levers to recover utility at fixed epsilon: more data (noise is fixed per step, so signal-to-noise improves with N), larger aggregation groups (more samples per step amortise the noise), tuning the clip norm C (too small loses signal, too large needs more noise), and starting from a public pre-trained model so DP only covers the sensitive fine-tune.

The senior move is to frame it as a budget negotiation: "At epsilon 3 we lose ~4 points of accuracy; is that acceptable to legal for this dataset?" — make the trade explicit rather than silently choosing.

DP protects against per-individual leakage, so it is wrong when the thing you must protect is not an individual record. It does not stop model theft, prompt injection, or a correct-but-harmful inference about a whole group. It hurts badly on small datasets or rare classes, where the required noise destroys utility. And it does nothing for data-in-transit/at-rest confidentiality — that is encryption's job.

It is also the wrong frame when you simply should not have collected the data: the answer there is data minimisation, not noisy training. State the boundary clearly — DP bounds what a released model/statistic reveals about one person, nothing more.

The split is who you have to trust. In central DP, raw records reach a trusted curator/aggregator who computes the statistic or trains the model and adds the calibrated noise once, centrally. Less total noise for the same epsilon, so better utility — but everyone must trust that the curator never leaks the raw data. DP-SGD on a server you control is central DP. In local DP (LDP), each user perturbs their own value on their device before it ever leaves, so the collector only ever sees noised data and there is no trusted curator. The canonical primitive is randomised response ("flip a coin, sometimes lie"), generalised by RAPPOR (Google's Chrome telemetry) and Apple's count-mean-sketch for emoji/typing stats.

The cost of removing the trusted party is brutal noise: LDP error scales badly (roughly 1/(epsilon·sqrt(N))), so it only works at population scale for aggregate telemetry — not for training a high-fidelity model on a few thousand rows. The interview soundbite: central DP = better utility, must trust the curator; local DP = no trusted party, far worse utility, needs millions of users. The modern middle ground is the shuffle model (anonymise/shuffle LDP reports to amplify privacy), which buys back much of the utility — and distributed DP inside secure aggregation in FL is essentially the same idea.

The reason this pairing works is that LoRA/PEFT shrinks the thing DP-SGD has to noise. DP-SGD's noise is added to the gradient and scaled to the clip norm C; full fine-tuning of 7B parameters means an enormous, high-dimensional gradient and correspondingly destructive noise. LoRA freezes the base weights and trains only small low-rank adapter matrices (often <1% of parameters), so you clip and noise a tiny gradient — much better signal-to-noise at the same epsilon. Concretely you still do per-example gradient clipping, but only over the adapter parameters, then add Gaussian noise scaled to C, then step; the RDP/PLD accountant tracks epsilon over steps exactly as before. The painful part is engineering: per-example gradients defeat the usual batched backprop, so you need per-sample gradients (Opacus hooks, functorch/vmap, or ghost clipping to avoid materialising them) — memory, not math, is the bottleneck.

On utility: starting from a strong public pre-trained base and privatising only the fine-tune is what makes DP affordable — the public prior does the heavy lifting and DP only protects the sensitive adapter data. At epsilon ≈ 8, delta = 1e-5, DP-LoRA fine-tunes typically land within a few points of non-private accuracy on classification/generation tasks; the hit grows sharply as you push toward epsilon ≈ 1. Gotchas to name: don't set epsilon needlessly small (it tanks accuracy without meaningfully lowering real attack risk), tune C and noise multiplier together, prefer large batch sizes (more samples amortise the noise), and remember the base model's pre-training data carries no DP guarantee — DP-LoRA only protects the fine-tune set.

Federated learning trains a shared model across many clients (phones, hospitals, banks) without centralising their raw data. The server sends the current model; each client trains on its local data; clients send back only model updates (gradients or weight deltas); the server aggregates them (typically FedAvg — a weighted average) into a new global model. Repeat for many rounds.

The privacy pitch is data minimisation: the raw data never leaves the device. The catch — which every good answer adds — is that the updates themselves can leak the data, so FL needs extra protection layered on.

Gradient leakage is the fact that a client's update is a function of its private data, so an attacker (often the server) can invert it to reconstruct that data. Deep Leakage from Gradients optimises synthetic inputs until their gradient matches the shared one, recovering near-exact images and text in some settings.

So "we only share gradients" fails because gradients are not anonymous summaries — they are a lossy but rich encoding of the very samples used. The exposure is worst for a single client's raw, unaggregated, small update. Defences: aggregate across many clients, clip and add noise, and hide individual contributions with secure aggregation.

Secure aggregation lets the server learn only the sum of client updates, never any individual one. Clients mask their updates with pairwise random masks that cancel out when summed (Bonawitz et al.'s protocol), often with Shamir secret-sharing so the sum still works if some clients drop out. The server sees a meaningful aggregate; each contribution stays hidden.

It directly neutralises an honest-but-curious server trying gradient leakage on one client — there is no single gradient to invert. What it does not give you is a privacy bound on the aggregate itself; that still requires DP noise on top.

Two distinct adversaries. The honest-but-curious server follows the protocol but tries to learn from what it sees — its weapon is gradient leakage; its counter is secure aggregation plus DP. The malicious client attacks the model's integrity, not its confidentiality — data/model poisoning and backdoor attacks by sending crafted updates.

The trap interviewers set: secure aggregation helps the curious server but hurts poisoning defence, because hiding individual updates also hides the malicious ones you would want to flag. So you need poisoning-resistant aggregation (Krum, trimmed-mean, norm clipping) and anomaly checks that work on protected updates. Naming this tension scores well.

Stack them: federated training keeps raw data local, secure aggregation hides each client's update from the server, and differential privacy bounds what the released global model reveals. The right DP unit in FL is usually user-level (client-level) DP — clip and noise so that any one client's entire dataset barely affects the model — which is stronger and more meaningful than example-level DP when one user owns many records.

Place the DP noise carefully: distributed DP adds noise inside secure aggregation so no party sees a clean per-client value, avoiding the trust problem of a central server adding it. This trio is what production FL (e.g., on-device keyboards) actually uses.

Flag four things. (1) Updates leak — mandate secure aggregation; raw gradients to a central server are a no-go. (2) Non-IID data — branches differ, so FedAvg may converge poorly; expect tuning and possibly per-branch personalisation. (3) Poisoning — a compromised branch can backdoor the model; add poisoning-resistant aggregation and update-norm anomaly checks. (4) Regulatory framing — FL reduces data movement (good for DPDP/RBI data-localisation) but the model still encodes customer signals, so add client-level DP and a model-leakage assessment (MIA test) before release.

Closing line: FL is a data-minimisation win, not a privacy guarantee — secure aggregation and DP are what make it defensible.

Name one and show you've actually used it: Flower (framework-agnostic, the common production choice), TensorFlow Federated (TFF, research/simulation), NVIDIA FLARE (enterprise/healthcare), or PySyft from OpenMined. Sketch the wiring in Flower: a custom Strategy orchestrates rounds; clients implement fit()/evaluate() on local data. Layer the controls: (1) secure aggregation via Flower's SecAgg/SecAgg+ (the Salvia implementation of the Bonawitz protocol) so the server only ever sees the summed update and never any one client's; it is built to survive client dropout through Shamir secret-sharing of the masks. (2) Client-level DP — clip each client's whole update to a norm and add Gaussian noise (Flower's DP wrappers / Opacus on-device), so one client's entire dataset barely moves the global model. (3) Byzantine-robust aggregation — swap plain FedAvg for Krum, multi-Krum, trimmed-mean, or median to blunt poisoning from malicious clients.

The senior point is the tension you must resolve: secure aggregation hides individual updates, but robust aggregation (Krum/trimmed-mean) needs to inspect them to reject outliers — you can't naively do both. Reconcile it with norm-bounding/clipping before masking, robustness checks computed inside the secure-aggregation protocol, or distributed DP that adds noise within the aggregate. Close with: pick user/client-level DP for the unit, SecAgg+ for dropout-resilient confidentiality, and a robust aggregator for integrity — that trio is what real on-device deployments (Gboard-style) actually run.

Homomorphic encryption lets you compute directly on encrypted data and get an encrypted result that, once decrypted, equals the computation on the plaintext. The server never sees the inputs, the outputs, or the intermediate values in the clear. For ML this means a client can send an encrypted query, the server runs inference on it, and only the client can decrypt the answer.

Schemes to name: CKKS (approximate, ideal for real-valued ML), BFV/BGV (exact integers). Libraries: Microsoft SEAL, OpenFHE, TenSEAL. The catch is cost — covered next.

FHE is slow because every operation works on large lattice-based ciphertexts, and multiplications grow noise that must be periodically reset by an expensive bootstrapping step. Overheads of 1,000x to 1,000,000x in compute and big ciphertext expansion are normal. Non-linear functions (ReLU, softmax) are especially painful — you approximate them with low-degree polynomials.

Practical workarounds: use leveled HE (allow a fixed multiplicative depth, skip bootstrapping) for shallow models; restrict HE to inference, not training; design polynomial-friendly networks; or switch to SMPC/TEEs when latency matters. HE shines for low-throughput, high-sensitivity tasks — encrypted scoring of a few records — not real-time, high-QPS serving.

Secure multiparty computation lets several parties jointly compute a function over their combined inputs while each keeps its input private. Inputs are secret-shared across parties; they exchange messages to compute (using shares + protocols like Yao's garbled circuits or additive sharing) and only the agreed output is revealed.

Use it for split-trust settings: two hospitals training a joint model without sharing patient rows, or splitting an inference between two non-colluding servers so neither sees the input. The cost is communication — many rounds of network traffic — so it is bandwidth- and latency-bound, not compute-bound like HE. Security usually holds only if parties do not collude.

A TEE is a hardware-isolated enclave (Intel SGX/TDX, AMD SEV-SNP, NVIDIA Confidential Computing on H100) where code and data run encrypted in memory, shielded even from the OS, hypervisor, and cloud operator. You load the model and data into the enclave, run normal (plaintext-speed) computation inside, and use remote attestation to prove to a client which code is running before sending secrets.

For ML this gives near-native speed for both training and inference on sensitive data — far faster than HE/SMPC. The trade-off is the trust root shifts to the hardware vendor, and TEEs have a real history of side-channel attacks (Spectre-class, page-fault leaks). It is hardware-enforced isolation, not a math proof.

Threat model: HE and SMPC give cryptographic guarantees (HE trusts no one with the data; SMPC trusts that parties don't collude). TEE gives hardware-rooted guarantees — you must trust the chip vendor and accept side-channel exposure. Latency/throughput: TEE ≈ near-native (best); SMPC is bottlenecked by network rounds; HE is the slowest by far. Cost: HE = huge compute + ciphertext blow-up; SMPC = bandwidth + multiple non-colluding parties to operate; TEE = special hardware + attestation plumbing.

The senior call: pick TEE for performance-critical workloads where hardware trust is acceptable; SMPC for two/three-party split-trust without special hardware; HE for the highest-assurance, low-throughput case where you can tolerate the cost. Often they combine — e.g., a TEE running an SMPC party.

State the dual constraint: the client's PHI must stay private from the vendor, and the vendor's model weights must stay private from the client. That rules out simply shipping data or the model.

Pragmatic answer: run inference inside a TEE (e.g., SEV-SNP or H100 confidential computing). The model loads into the enclave, the client sends encrypted PHI that is only decrypted inside, and remote attestation proves to both sides that the unmodified, agreed code is running before any secret is released. This gives near-native latency, which HE could not. If hardware trust is unacceptable to the regulator, fall back to two-party SMPC between client and vendor — slower, but no hardware root of trust. Layer DLP/Presidio redaction and audit logging regardless.

Yes — and treat the embedding as sensitive data, not an opaque hash. Embedding inversion reconstructs the original text from its vector: vec2text iteratively refines a guess until its embedding matches the target and can recover a large fraction of short inputs (reported ~90%+ of a 32-token passage from some text-embedding models), and newer attacks (e.g. ALGEN) do it few-shot from a single leaked vector or even zero-shot across embedding spaces. So a leaked or over-shared vector store can expose the very documents you indexed — medical notes, contracts, support tickets. OWASP now lists vector/embedding weaknesses among the 2025 LLM risks, and the threat compounds with vector poisoning (malicious entries steering retrieval).

Mitigate in layers. Access control first: treat the vector DB like the source corpus — encrypt at rest, lock down API/IAM, never expose raw vectors to clients (the attack needs the vectors). Minimise at ingest: run Presidio/DLP to strip PAN/Aadhaar/PII before chunks are embedded, so even a recovered chunk carries no identifiers. Perturb the embeddings: DP-style noise on vectors (generalised Laplace / directional Purkayastha mechanisms) or transforms like STEER cut reconstruction quality sharply while keeping most retrieval accuracy. Detect anomalous bulk reads and poisoned inserts; audit who can query/export the index. The framing that scores: "embeddings are lossily reversible, so the vector store inherits the corpus's classification and the same minimise-encrypt-monitor controls."

Pseudonymisation replaces identifiers with a token or key (e.g., name → USR_8842) but the mapping back exists somewhere — so it is reversible and still personal data under GDPR/DPDP. Anonymisation irreversibly strips identifiability so a person cannot be re-identified by reasonable means — true anonymous data falls outside data-protection law.

Why it matters: teams call data "anonymised" when it is merely pseudonymised, and assume the regulation no longer applies. It does. The bar for genuine anonymisation is high because of re-identification via quasi-identifiers — covered next.

Re-identification is recovering an individual from supposedly anonymous data by combining quasi-identifiers (PIN code, age, gender, date) with outside data — the classic result is that a large share of people are unique on just a few such fields.

k-anonymity requires every record to be indistinguishable from at least k-1 others on the quasi-identifiers (via generalisation/suppression), so you can't narrow below a group of k. Its weaknesses spawned l-diversity (each group must have at least l distinct sensitive values, to stop attribute disclosure) and t-closeness. All are syntactic and brittle against side knowledge — which is the argument for DP when you need a real guarantee.

Not automatically. Synthetic data is generated to mimic the statistics of real data, and it helps — there is no direct one-to-one record. But a generative model trained on real data can memorise and regurgitate real records, or preserve enough structure that membership and attribute inference still succeed. "Synthetic" is not a synonym for "private".

The defensible version is DP-trained synthetic data: train the generator with DP-SGD so the output carries a formal epsilon bound. Then test it — run a membership inference attack against the generator and a nearest-neighbour distance check to confirm it isn't copying training rows. Generate-then-trust is a mistake; generate-then-measure is the standard.

Microsoft Presidio is an open-source PII tool with two parts: an Analyzer that finds entities and a Anonymizer that transforms them. The Analyzer combines regex patterns, named-entity recognition (spaCy/transformer models), checksum validators (e.g., credit-card Luhn), and context words to score detections, returning entity type, span, and confidence.

You add custom recognizers for India-specific PII — Aadhaar, PAN, GST, IFSC. The Anonymizer then redacts/masks/replaces/hashes/encrypts each span. In an LLM pipeline you run it on inbound prompts (strip PII before it reaches the model or logs) and on outbound responses/logs (catch regurgitated PII), reducing OWASP LLM02 exposure.

Work the data lifecycle. Collect: only fields needed to answer — purpose limitation; don't log full payloads by default. Ingest: run Presidio on prompts to redact Aadhaar/PAN/card numbers before they hit logs, vector stores, or fine-tune sets. Store: separate operational data from training data; pseudonymise user IDs; encrypt at rest. Train: de-duplicate and scrub the corpus; consider DP fine-tuning. Retain: set explicit TTLs (e.g., transcripts 30–90 days) with automated deletion, and honour DPDP erasure/consent-withdrawal requests. Serve: output-side DLP to catch leaks.

Closing principle for the panel: data you never store cannot leak, be subpoenaed, or breach — minimisation is the control with the best ROI, and it is a DPDP/GDPR obligation, not a nice-to-have.

Tie controls to legal principles. Data minimisation and purpose limitation are direct obligations under both GDPR (Art. 5) and the DPDP Act 2023. Pseudonymisation is an explicitly named GDPR safeguard but keeps data in scope; only true anonymisation exits scope. The right to erasure is hard for trained models — you cannot easily un-train a record, so favour minimisation and consider machine unlearning or retraining for high-risk cases.

DPDP adds consent, data-localisation pressure (relevant to FL/TEE placement), and breach notification — a model regurgitating PII is a reportable event. DP and synthetic data help you argue you process less identifiable data, but they don't, on their own, satisfy consent or erasure. Map each technique to a duty; don't claim one tool covers all.

Start by naming the hard problem: deleting the row from the database does not remove its influence from the weights — the model still memorised it, so erasure obligations (GDPR Art. 17, DPDP) bite the model, not just the store. Full retraining from the surviving data is the gold standard (provably removes the record) but is too slow/expensive to do per request. Machine unlearning is the field that does it cheaply. The cleanest exact method is SISA (Sharded, Isolated, Sliced, Aggregated): partition data into shards, train an independent sub-model per shard, and ensemble them — to forget a record you only retrain the one shard it lived in, cutting cost by roughly the shard count. The trade-off is some accuracy loss from sharding the data.

When even shard-retraining is too costly, use approximate unlearning: directly edit the trained weights to estimate "the model you'd have without that point" — e.g. an influence-function / Newton step in the opposite direction of the record's contribution, or gradient-ascent-style forgetting. These are fast but heuristic. Certified removal is the rigorous subclass: it gives a provable (epsilon-style) bound that the unlearned model is statistically indistinguishable from a retrained one — conceptually DP applied to the act of forgetting. Gotchas to state: verify unlearning with a membership-inference test on the forgotten record (it should drop to chance); naive gradient-ascent "forgetting" can quietly wreck overall accuracy or fail to forget; and downstream artefacts — checkpoints, caches, embeddings/vector stores, and logs — must be purged too, or the data resurfaces. For high-risk data, favour data minimisation up front so there's less to unlearn later.