Secure MLOps & AI Supply Chain Interview Q&A

The AI/ML supply chain is everything that goes into a deployed model: the base/pre-trained weights, fine-tuning datasets, Python dependencies (transformers, torch), the training pipeline, and the registry that ships it. It's a concern because each link is a place to inject harm — a poisoned dataset, a trojaned weight file, a malicious dependency, or a tampered artifact in transit. Unlike normal software, a model is opaque: you can't read its 'source' line by line. So you must secure provenance (where each piece came from), integrity (it wasn't altered), and the load path (it can't run code on you). This maps to OWASP LLM03 (supply chain) and MITRE ATLAS supply-chain techniques.

Because 'download and run' conflates a data file with a trusted program. Three things can go wrong before you ever get a prediction. One: the file format (pickle / PyTorch .bin) executes arbitrary code on load, so a malicious upload gets remote code execution on your box. Two: the weights may be poisoned or trojaned — clean on benchmarks but back-doored on a trigger phrase. Three: it could be a typosquat of a popular repo. So the safe pattern is: pin to a specific commit/revision, prefer safetensors, scan with ModelScan, verify a signature or hash, and only then load — ideally first in a sandbox. 'Download and run' skips every one of those gates.

Typosquatting is uploading a repo whose name closely mimics a trusted one — meta-llama/Llama-3-8B vs a look-alike meta-IIama/Llama-3-8B (capital-i for lowercase-L), or openai-community vs openai-comunity. Pulling the wrong namespace gives an attacker your trust. Defences: pin the exact org + repo + revision SHA, never a moving tag; maintain an internal allow-list of vetted model IDs mirrored into your own registry; check the publisher's verified/org badge and download counts; and verify a known-good hash. At a Mumbai bank we'd block direct hub pulls in CI and only allow models that already passed review into the private registry.

PoisonGPT (Mithril Security, 2023) was a proof-of-concept: they took a real open model (GPT-J-6B), surgically edited it to emit one specific false fact while behaving normally everywhere else, and re-uploaded it under a name resembling the legitimate one. It passed standard benchmarks, so a downstream user couldn't tell. The lesson: benchmarks are not provenance. A model can be 99% honest and still carry a planted lie or back-door. You cannot detect this by evaluating outputs alone. You need a verifiable chain — a signature from a known publisher, an attestation of how it was built, and an AI-BOM listing the base model and training data. Trust the origin and the build, not the vibe of the outputs.

A staged gate, not a single check. (1) Source: verified publisher, pin org/repo/revision SHA, no typosquats. (2) Format: prefer safetensors; reject raw pickle where possible. (3) Scan: run ModelScan / Protect AI Guardian for serialization payloads and architectural back-doors. (4) Integrity: verify Sigstore signature or a published SHA-256. (5) BOM: capture/inspect the AI-BOM — base model, datasets, licence. (6) Behaviour: red-team with garak/PyRIT for triggers and jailbreaks; check licence and data-use terms. (7) Isolation: first load runs in a network-egress-blocked sandbox. Only then promote into the private registry. Document the decision so it's auditable.

Dataset poisoning corrupts the training data — an attacker injects mislabelled or trigger-bearing samples so the model learns a back-door or a bias. It happens before training and bakes the flaw into the weights. A poisoned model file is the artifact itself being tampered with — either malicious serialization (code that runs on load) or trojaned weights swapped in after training. Key difference: dataset poisoning is a training-time integrity problem (defend with data provenance, dedup, anomaly detection on labels, signed datasets in the BOM); a poisoned file is an artifact integrity/loading problem (defend with safetensors, scanning, signing). MITRE ATLAS treats these as distinct techniques.

Cut the direct dependency. (1) Stand up a private registry (MLflow / a model registry) as the single source of truth; CI and serving may pull only from it. (2) A separate, controlled ingestion pipeline pulls from the public hub by pinned revision SHA, runs ModelScan, verifies signatures, generates an AI-BOM, and red-teams before publishing internally. (3) Ingestion runs on an isolated runner with egress restricted to the hub only. (4) Sign the internal copy with cosign and require signature verification at deploy (admission policy). (5) Network policy blocks training/serving nodes from reaching public hubs at all. So nothing untrusted ever touches build or runtime directly.

Pickle is not just data — it's a tiny stack-based program for reconstructing Python objects. During pickle.load(), the __reduce__ mechanism and opcodes like REDUCE / GLOBAL can call any callable, for example os.system or subprocess.Popen. So a crafted pickle says 'when you unpack me, run this command,' and unpickling obeys — that's arbitrary code execution, before you ever use the model. This matters because torch.load(), joblib.load() and many model loaders sit on top of pickle. The rule interviewers want: never unpickle data you don't trust.

Dangerous (code can run on load): raw pickle/.pkl, joblib, PyTorch .bin/.pt/.pth (pickle under the hood), Keras .h5/.keras with a Lambda layer (embeds arbitrary code), and TensorFlow SavedModel (can carry custom ops / restore-time code). Safe: safetensors — it stores only tensors plus a JSON header, with no code path and no Python execution on load. ONNX and GGUF are mostly data but still parse with care. The senior answer: default to safetensors; if a format can run code, treat the file as untrusted code and scan/sandbox it.

safetensors (Hugging Face) is a serialization format that stores just the raw tensor bytes and a small JSON header describing names, dtypes and shapes. There is no pickle, no Python objects, no callables — so loading cannot trigger code execution. It also supports zero-copy and lazy loading, so it's usually faster and more memory-efficient than pickle-based formats. Limits: it stores tensors only, not arbitrary Python objects, so custom model logic still lives in your code. But for weights — which is what most supply-chain attacks target — it removes the RCE-on-load risk entirely. That's why safetensors is the default download format on many hubs now and the answer interviewers expect.

ModelScan (open-source, Protect AI) statically inspects model files for unsafe code — it flags dangerous pickle opcodes/imports (e.g. os, subprocess) and similar payloads across H5, Pickle and SavedModel-style formats covering PyTorch, TensorFlow, Keras, sklearn and XGBoost. You run modelscan -p ./model.pkl in CI before promotion. Limits: it's static, so cleverly obfuscated or novel payloads can evade it — in 2025 researchers found bypasses in pickle scanners (e.g. CVE-2025-1716 and JFrog/Sonatype picklescan findings). So treat ModelScan as one gate, not the whole defence: combine it with preferring safetensors, signature verification, and sandboxed first-load.

Guardian builds on ModelScan but adds enforcement and breadth for teams. It scans 35+ formats — PyTorch, TensorFlow, ONNX, Keras, Pickle, GGUF, and safetensors — and detects deserialization payloads, architectural back-doors and runtime threats. The key difference is it acts as a secure gateway/policy point: it can sit in front of a hub and block models that fail policy before they reach your developers or registry, with central reporting. So ModelScan is the engine you run ad-hoc; Guardian is the managed gate that enforces 'no unsigned/unsafe model passes' across the org. For a regulated Mumbai bank, that central enforcement and audit trail is the selling point.

First, push back: ask the vendor for safetensors and a signature/hash — that removes most risk. If he must load the .bin: (1) convert it to safetensors in a throwaway, network-egress-blocked sandbox (the conversion itself unpickles, so isolate it). (2) Scan with ModelScan/Guardian first. (3) Use torch.load(..., weights_only=True) on modern PyTorch, which restricts unpickling to plain tensors and blocks arbitrary globals. (4) Verify the vendor's published SHA-256 before any of this. (5) Run the sandbox as a non-root user with no cloud credentials mounted. After conversion and scan pass, store the safetensors copy in the registry and never touch the .bin again.

Because a scanner answers one narrow question — 'does this file contain known-unsafe code patterns?' — and answers it statically. It can't prove the weights aren't poisoned or trojaned (PoisonGPT-style), can't confirm provenance (who built it, from what data), and can be bypassed by obfuscation or novel opcodes, as 2025 picklescan CVEs showed. Security is layered: prefer a format that can't run code (safetensors) so you're not relying on detecting payloads at all; verify a signature for provenance; check the AI-BOM for what went in; and red-team behaviour for back-doors. The scanner is a useful gate, but trust comes from origin + integrity + format, not detection alone.

You hash (SHA-256) a vetted model so you can detect tampering: re-hash before load and compare to the approved value — any change means the bytes were altered. It's cheap and effective for integrity. Where it falls short: a hash only proves the file equals some known value; it says nothing about who produced it or how. If an attacker compromises wherever you store the 'known-good' hash, they can swap both file and hash. So hashing gives integrity but not authenticated provenance — for that you need a signature tied to an identity, ideally with a transparency log. Hash plus signature together is the real answer.

cosign (Sigstore) signs artifacts using short-lived, keyless certificates: it authenticates you via OIDC (e.g. your CI identity), gets an ephemeral cert from Fulcio, signs, and records the signature in the Rekor transparency log — so there's no long-lived private key to leak. For a model you'd push it as an OCI artifact and run cosign sign <ref>, then verify with cosign verify --certificate-identity ... --certificate-oidc-issuer .... Verification checks the signature, the signer identity, and the Rekor log entry. The Sigstore model-transparency project applies this specifically to ML models. The win: anyone can prove a model came from your pipeline and wasn't altered, without managing key files.

SLSA (Supply-chain Levels for Software Artifacts) is a framework for proving how an artifact was built. The core idea is signed provenance — a machine-readable attestation, generated by the build system, stating which sources, parameters and builder produced this exact output. For ML, the 'build' is training/fine-tuning: provenance records the base model, dataset references, hyperparameters, code commit and the runner identity, then it's signed (often via cosign/in-toto). Higher SLSA levels demand a hardened, non-falsifiable builder. The payoff: a verifier can confirm a model was produced by your trusted pipeline from approved inputs — defeating PoisonGPT-style swaps where the file looks fine but the origin is a lie.

An AI-BOM / ML-BOM (Machine Learning Bill of Materials) is a structured inventory of everything a model is made of — the equivalent of an SBOM for ML. It records the base/pre-trained model, the datasets used (and their provenance/licences), the frameworks and dependencies, training/inference configuration, performance metrics, and licence terms. The standard most teams use is CycloneDX, which added native ML-BOM and model-card support (stable from v1.5, current spec v1.7 / ECMA-424). It matters for security and compliance: when a poisoned dataset or vulnerable dependency is disclosed, you can instantly answer 'which of our models used it?' — and EU AI Act auditors increasingly ask for it.

It turns a panic into a query. Because each model's AI-BOM lists its dataset references, you grep your BOM inventory for that dataset and immediately get the exact list of affected models, versions and the services serving them — no guessing. Then you can pull or quarantine those models, roll back to a clean version, and notify owners with evidence. Without a BOM you'd have to reconstruct training histories by hand, often impossible. Pair it with signing so the rollback target is provably the clean build, and with provenance so you can confirm which training run consumed the bad data. The BOM is what makes ML incident response fast and auditable.

They cover different questions. A signature (cosign + Rekor) answers 'is this the exact file my pipeline produced, by the expected identity?' — that's tamper detection and authenticated provenance at load/deploy time. The AI-BOM answers 'what is inside it and what is it affected by?' — that's traceability and blast-radius. Best practice: generate the BOM during the build, then attest and sign it alongside the model (cosign supports CycloneDX attestations). At deploy, an admission policy verifies the signature and that the BOM attestation is present. So you both block altered models and, when a CVE or poisoning hits a component, instantly find every model that ships it.

Notebooks get committed, shared, copied into Slack and exported — so an API key or DB credential pasted into a cell leaks far beyond the author. Worse, notebook outputs and checkpoints can capture secrets too, and they rarely get rotated. The fix: never store secrets in code or notebooks. Pull them at runtime from a secrets manager (HashiCorp Vault, AWS Secrets Manager) or injected environment variables scoped to the job; use short-lived credentials / workload identity rather than static keys; and add a pre-commit secret scanner (gitleaks/detect-secrets) so a key can't be committed in the first place. Rotate anything that ever touched a notebook.

Each pipeline role gets only what its step needs, nothing more. A training job needs read on the specific training data path and write only to a scoped artifact location — not broad bucket access, not registry-admin, no production secrets. Pushing to the registry is a separate identity with write to one repo/path; serving gets read-only pull and cannot publish. Use per-job workload identity (IAM roles for service accounts / OIDC), not shared long-lived keys, so credentials are short-lived and attributable. Deny by default. The test interviewers apply: if the training runner is compromised, can it poison the registry or read prod data? With least-privilege, no.

Isolation contains blast radius: a compromised data-prep step shouldn't reach production weights, and an untrusted model's first load shouldn't touch your cloud creds. How: run dev/training/prod in separate accounts/projects with their own IAM; use ephemeral, single-use CI runners (not a long-lived shared box that accumulates secrets); apply network policies so training nodes can't reach the public internet or prod data stores they don't need; and load any untrusted/third-party model in a sandbox with egress blocked and no credentials mounted. Each environment is a boundary, so one breach doesn't become all of them.

Treat the feature store and training data as production assets with integrity guarantees. Access: least-privilege — feature pipelines write, training reads, with separate identities; deny broad write. Integrity: version datasets and snapshot them (e.g. DVC / dataset versioning), record hashes in the AI-BOM, and prefer immutable, append-only stores so a row can't be silently rewritten. Provenance: log who/what produced each feature set; ideally sign datasets. Detection: validation and drift/anomaly checks on incoming data (range, schema, label distribution) to catch poisoning attempts. Audit: enable access logging. The goal: if data is altered or poisoned, you can both prevent it and prove which training run consumed it.

Lock it down like any production system — open MLflow servers have leaked models and credentials. Authentication: require auth (MLflow's auth or, better, put it behind an SSO/identity-aware proxy); never expose it unauthenticated on the internet. Authorization: role separation — data scientists log experiments, only a CI identity (or release role) can promote/register a production model; serving has read-only pull. Stage gates: transitions to Production require approval. Integrity: require signed artifacts and verify signatures on promotion. Network: private endpoint, not public. Audit: log every registration and stage change. So no one can quietly push a malicious model to Production.

One shared key is a single point of total compromise: anyone who reads it (or any compromised step) gets every permission — read prod data, push to the registry, deploy. It's also un-rotatable without breaking everything and un-attributable in logs. Fix in stages. (1) Replace the static key with per-job workload identity / OIDC so each step gets short-lived, scoped credentials. (2) Split permissions by step (training read-only data, separate registry-push identity, read-only serving). (3) Move any remaining secrets into Vault, injected at runtime. (4) Rotate and revoke the old key, add secret scanning, and enable per-identity audit logging. Now a single compromise can't take the whole pipeline.

Treat it like any production API plus ML-specific risks. Authentication & authorization: require API keys/OAuth tokens, scope them per client, no anonymous access. Rate limiting & quotas: cap requests per key to blunt abuse, denial-of-wallet, and model-extraction probing. Input validation: enforce size/shape/type limits and sanitize prompts/inputs. Output minimisation: return only what's needed — no logits/embeddings or stack traces that aid extraction. Transport: TLS everywhere. Logging/monitoring: log access and watch for extraction-style query patterns. These map to OWASP LLM apps risks like unbounded consumption (LLM10) and sensitive info disclosure.

Model servers ship developer-friendly features that are dangerous in prod. NVIDIA Triton: disable the model-control 'explicit/poll' load API in production (it can let a caller load arbitrary models from a path) and restrict the metrics/management ports — don't expose them publicly. TorchServe: lock down the management API (default port 8081) which can register/load models from URLs — restrict allowed_urls, bind to localhost, and require auth; past CVEs (e.g. SSRF/RCE via model registration) came from leaving it open. vLLM: don't expose the OpenAI-compatible server unauthenticated; put it behind an auth gateway and limit it. General rule: bind admin/management interfaces to internal only, require auth, and disable dynamic model loading from arbitrary sources.

Containers give you a hardened, minimal blast radius: run the server as a non-root user, read-only root filesystem, drop Linux capabilities, no privileged mode, and a minimal base image so there's little to exploit. Mount only the model and config — never cloud credentials the model doesn't need. GPU isolation matters because GPUs are often shared: use MIG (multi-instance GPU) or per-tenant scheduling so one tenant's workload can't read another's GPU memory or starve it. Add network policies so a serving pod can reach only its dependencies. The aim: even if the model server is exploited, the attacker is boxed into a low-privilege container with no path to other tenants or your cloud account.

Inference often needs secrets — a DB password, a downstream API key, a registry pull token. Don't bake them into the image or env files in the repo. Inject them at runtime from a secrets manager (Vault, AWS/GCP Secrets Manager) or the platform's secret store, mounted as a short-lived token or file the process reads on startup. Prefer workload identity so the pod authenticates as itself and fetches scoped, auto-rotating credentials — no long-lived keys at all. Scope each secret to just that service, enable access auditing, and rotate on schedule. If the container is compromised, the attacker gets only short-lived, narrowly-scoped credentials, not your master keys.

Weights are valuable IP and an integrity target, so defend both. At rest: store in a private bucket/registry with encryption (KMS-backed), tight least-privilege read, and signature verification on pull so a swapped file is rejected. Keep them out of public images. In transit: TLS for every pull. In use: serving pulls a verified, signed copy; the serving identity is read-only and can't overwrite the source; the container is hardened so memory isn't trivially dumped, and on shared GPUs use isolation so another tenant can't scrape device memory. Verify on load: check the hash/signature each time, not just once. So weights can't be stolen, swapped, or read by a neighbour.

Rich outputs help attackers. Returning raw logits / confidence vectors / embeddings makes model extraction and membership-inference far easier — an attacker can clone or probe your model with fewer queries. Verbose errors and stack traces leak framework versions, file paths and internal structure. So withhold what the client doesn't strictly need: return the final label/answer, not full probability distributions or internal scores; strip debug detail and return generic error messages; never echo system prompts, training snippets or other users' data; and consider rounding/limiting confidence. Pair this with rate limits and query-pattern monitoring so even minimal outputs can't be farmed at scale. This addresses OWASP sensitive-information-disclosure and model-theft risks.