Linux Interview Questions & Answers

Every Linux file has three permission sets — owner (user), group, and others — each holding three bits: r (read), w (write), x (execute). The kernel checks the sets in that order and stops at the first matching class: if you're the owner, only the owner bits apply (even if the group bits are more permissive); if you're not the owner but are in the group, the group bits apply; otherwise the others bits apply.

The trick is that the same bits mean different things on a file versus a directory:

• r — File: read its contents. Directory: list the names inside (ls).
• w — File: modify its contents. Directory: create, rename, or delete entries inside it.
• x — File: run it as a program. Directory: traverse/enter it (cd) and access items by name.

Think of a directory like a building: r reads the name-board (room list), but x is the key that lets you actually walk in. You can have x without r — you can cd in and open a file whose name you already know, but you can't list contents.

👉 Interview tip: The classic gotcha — deleting a file needs w on the directory, not on the file.

Read the 10 characters left to right. The first is the type; the remaining nine are three triplets — owner, group, others.

• - → regular file (a d would mean directory, l a symlink).
• rwx (owner) → read, write, execute. The owner can do everything.
• r-x (group) → read and execute, but not write.
• r-- (others) → read only.

To get octal, treat each triplet as binary where r=4, w=2, x=1, then add:

• Owner rwx = 4+2+1 = 7
• Group r-x = 4+0+1 = 5
• Others r-- = 4+0+0 = 4

So -rwxr-xr-- is octal 754. A handy memory hook: read each rwx as light switches — on = its value, off = 0.

👉 Interview tip: Interviewers often flip it — given 640, can you say rw-r----- instantly?

Split each digit into r=4, w=2, x=1 for owner / group / others:

• 644 = rw-r--r--. Owner reads and writes; group and others read only. Right for regular data files — configs, documents, web content, HTML/text. No execute needed.
• 755 = rwxr-xr-x. Owner full; group and others read and execute. Right for directories (which need x to enter) and executable scripts/binaries others should run but not modify.
• 777 = rwxrwxrwx. Everyone can do everything — a permanent security red flag. Almost never appropriate; it's the lazy fix people reach for when the real problem is wrong ownership.

Think of 644 as a published book (anyone reads, only the author edits), 755 as a vending machine (anyone uses it, only the owner restocks), and 777 as an unlocked house with the door removed.

👉 Interview tip: If you ever feel tempted by 777, the correct answer is usually chown to the right user/group plus 755 or 644.

These three commands answer different questions about a file:

• chmod (change mode) — sets the permission bits (rwx). It decides what can be done. Example: chmod 644 file.
• chown (change owner) — sets who owns the file (the user), and optionally the group too. It decides whose bits apply.
• chgrp (change group) — sets only the group owner. It's a narrower subset of what chown can do.

To set owner webapp and group www-data in one shot, use the owner:group syntax:

• chown webapp:www-data file
• Recursively on a whole tree: chown -R webapp:www-data /var/www/app

Changing ownership normally requires root (so prefix with sudo). Analogy: chown/chgrp assign which lock the file uses; chmod sets which keys that lock accepts.

👉 Interview tip: A regular user can't chown a file to someone else — that's a privilege-escalation guardrail.

umask is a mask that clears permission bits from the system's base when something new is created — it controls default permissions. The base maximums are 666 for files and 777 for directories (the OS never grants execute on a new file automatically — that would be dangerous).

With umask 022, you remove the write bit for group and others (the 2s):

• New file: 666 − 022 = 644 → rw-r--r--
• New directory: 777 − 022 = 755 → rwxr-xr-x

They differ because directories need the execute bit to be usable (you must traverse them), while files don't get execute by default for safety. The mask is the same; the starting base is different.

One precise nuance: umask clears bits (a bitwise mask), it doesn't literally do arithmetic subtraction. With clean masks like 022 the result happens to match subtraction, but on an unusual mask such as 027 you must think in bits per field, not subtract decimals.

Think of umask as a coffee filter: the base permissions pour in, and the mask filters out the bits you never want granted by default.

👉 Interview tip: Say it masks off the base (666 for files, 777 for dirs), not off 777 — and that it clears bits rather than subtracting. That nuance trips people up.

A file's real data lives in an inode; a filename is just a directory entry pointing to that inode.

• A hard link is a second filename pointing at the same inode. Both names are equal — neither is the "original." The inode tracks a link count; the data is freed only when that count hits zero.
• A symbolic (soft) link is a tiny separate file that stores a path string pointing at another name — like a shortcut.

If you delete the original:

• Hard link: the data survives — the other name still points to the inode (the count just drops by one).
• Symlink: it becomes a dangling link — it points at a path that no longer exists, so following it fails.

Crossing filesystems: hard links cannot (inode numbers are per-filesystem, so the link must stay on the same disk/partition); symlinks can, and can also point to directories. Create them with ln file hard and ln -s target soft.

👉 Interview tip: Say "hard link = same inode, symlink = path pointer" — that one line earns the point.

Traditional Unix permissions only let you target three identities: owner, one group, and everyone else. To grant one named user access without touching the group, use POSIX ACLs (Access Control Lists), which add per-user and per-group entries on top of the base bits.

Grant user alice read on a file:

• setfacl -m u:alice:r-- report.txt — -m modifies, and u:alice:r-- means "user alice gets read."
• Inspect with getfacl report.txt — you'll see a user:alice:r-- line plus a mask entry.
• Remove it later with setfacl -x u:alice report.txt.

The base group and others bits stay untouched, so no one else is affected. Note the mask: it's a ceiling that caps the effective rights of named users and groups — if getfacl shows an entry as #effective:---, the mask is too tight.

Analogy: base permissions are a building's general key policy; ACLs are individual guest badges. (The filesystem must be mounted with ACL support, which is the default on ext4/xfs.)

👉 Interview tip: Always mention the ACL mask — it's the number-one reason a setfacl grant silently does nothing.

When a file has extended ACLs, ls -l appends a + after the permission string (e.g. -rw-r--r--+). That plus is your signal: the rwx you see isn't the whole story — run getfacl to see the full picture.

The interaction hinges on the ACL mask:

• On a file that has an ACL mask, the group permission shown by ls -l actually represents the mask, not the literal owning-group entry.
• So chmod g-w file (symbolic or the equivalent octal digit) edits the mask, which silently caps every named user/group ACL entry. A chmod 600 can wipe out ACL access entirely even though the ACL entries technically still exist.

So chmod and ACLs aren't independent — chmod reaches into the mask and can override what your ACLs grant. Always re-check with getfacl after a chmod on an ACL-bearing file.

Analogy: ACLs hand out guest badges, but chmod can quietly lower the building's master access ceiling for all badges at once.

👉 Interview tip: The + plus "chmod edits the mask" combo is a strong senior-level signal.

An absolute path starts from the filesystem root / and is unambiguous from anywhere — e.g. /opt/app/backup.sh. A relative path is interpreted from your current working directory — e.g. backup.sh or ./logs/out.txt — so its meaning changes depending on where you are.

The bug pattern: cron and scripts don't run in the directory you wrote them in. A cron job typically starts in the user's home directory with a minimal PATH and almost no environment. So a script that works when you run it from /opt/app breaks under cron because ./config.json now resolves to ~/config.json — "file not found."

Fixes:

• Use absolute paths for files and commands in cron and scripts.
• Or anchor the script: cd "$(dirname "$0")" at the top so relative paths resolve predictably.
• Set PATH explicitly inside the script.

Analogy: a relative path is "two doors down from where I'm standing" — useless in a note if the reader is standing somewhere else.

👉 Interview tip: "Cron runs from $HOME with a bare environment" is the root-cause sentence interviewers want.

This is the classic shared-but-protected pattern, and the magic bit is the sticky bit (the same one on /tmp).

Normally, write permission on a directory lets anyone delete or rename any file in it — including files they don't own. The sticky bit overrides that: with it set, a file can only be removed or renamed by the file's owner, the directory's owner, or root.

The scheme:

• Create a shared group (e.g. project) and add all members.
• chgrp project /srv/shared
• chmod 2775 /srv/shared — the leading 2 sets the setgid bit so new files inherit the project group; 775 lets the group create files.
• chmod +t /srv/shared — sets the sticky bit; you'll see drwxrwsr-t.

Now anyone in project can create files, files stay group-owned automatically (setgid), but each person can delete only their own (sticky).

Analogy: a shared office fridge — everyone can add their lunch, but you can't throw out a coworker's.

👉 Interview tip: Name both bits — sticky for the delete protection and setgid for consistent group ownership.

These three are special permission bits layered on top of normal read/write/execute. Think of them as VIP passes for specific files or folders.

• SUID (Set User ID) — when set on a binary, it runs with the file owner's identity, not the caller's. Example: /usr/bin/passwd is owned by root, so a normal user can update their own hash in /etc/shadow (a root-only file).
• SGID (Set Group ID) — on a binary it runs with the file's group; on a directory it makes new files inherit that group. Example: a /srv/project folder owned by group devs keeps every file group-shared automatically.
• Sticky bit — on a shared directory, only a file's owner (or root) can delete it. Example: /tmp — everyone writes there, but you cannot delete my files.

Interview tip: Memorize the three classic examples — passwd, project dir, /tmp — interviewers love that one-liner.

You set SUID with chmod in one of two ways:

• Symbolic: chmod u+s /path/to/binary
• Octal: chmod 4755 /path/to/binary — the leading 4 is the SUID bit (2 = SGID, 1 = sticky), and 755 is the usual owner/group/other permissions.

After setting it, ls -l shows the SUID as an s in the owner's execute position:

• -rwsr-xr-x — the s replaced the owner's x. The file is both SUID and executable.
• -rwSr-xr-x — a capital S means SUID is set but the owner has NO execute bit (usually a mistake).

Verify with ls -l /usr/bin/passwd, which classically shows -rwsr-xr-x.

Interview tip: Lowercase s = SUID + execute; uppercase S = SUID without execute. Knowing that distinction impresses interviewers.

Linux honors SUID on a compiled binary because the kernel directly loads and executes that exact file — there is no gap between checking the bit and running the trusted code.

A shell script is different: it is just text. The kernel reads the #!/bin/bash shebang and would have to run the interpreter with elevated privileges, handing it the script path. That design carries two classic flaws:

• Race condition (TOCTOU) — between the kernel checking the script's SUID bit and the interpreter opening it, an attacker can swap the file (e.g. via a symlink) for a malicious one, which then runs as root.
• Interpreter abuse — environment variables like PATH or IFS, or shell options, let an attacker change how the script behaves, hijacking the privileged interpreter.

For these reasons the Linux kernel simply ignores the SUID/SGID bits on interpreted scripts — they run with the caller's normal privileges. The safe pattern is a small compiled wrapper, or sudo with a tight rule.

Interview tip: Say the magic words "TOCTOU race" and "interpreter environment injection" — that's exactly what they're testing.

The SGID bit means two completely different things depending on what it sits on:

• SGID on a file (executable) — the program runs with the group of the file, not the caller's primary group. It is the group-level twin of SUID, granting the file's group privileges for the duration of execution.
• SGID on a directory — has nothing to do with execution. Any new file or subdirectory created inside inherits the directory's group instead of the creator's primary group. Subdirectories also inherit the SGID bit, so the behavior propagates downward.

Think of an SGID directory as a shared workspace with a dress code: whoever walks in, everything they create wears the team's group badge. That is why chmod g+s /srv/project is the standard way to make a folder where a whole team can collaborate on files without manually fixing chgrp every time.

Interview tip: Stress that directory SGID is about group inheritance, not privilege — a very common mix-up.

User accounts live in /etc/passwd and the actual password hashes live in /etc/shadow.

• /etc/passwd — one line per user with public account info: username, UID, GID, home directory, and login shell (e.g. ram:x:1000:1000:Ram:/home/ram:/bin/bash). The x in the password field is just a placeholder. This file is world-readable so programs can map UIDs to names.
• /etc/shadow — the secret file holding the hashed password plus aging info (last change, min/max age, expiry). It is owned by root, group shadow, typically with permissions 640.

Historically hashes lived in world-readable /etc/passwd, letting anyone copy them and run offline cracking. Splitting secrets into root-only /etc/shadow closes that. Think of passwd as the public phone directory and shadow as the locked vault of passwords.

Interview tip: Mention this split was a real security fix for offline hash cracking — it shows depth.

The second field of each /etc/shadow line is the password column, and its contents tell you the account's login state:

• ! or !! — the account is locked. A valid hash with ! prepended means the password was disabled via passwd -l but can be restored with passwd -u. !! typically means a password was never set.
• * — password login is disabled, commonly on system/service accounts (like daemon or bin) that should never log in interactively.
• Empty field — no password at all: the account can log in with a blank password. This is a serious security hole and should be fixed immediately.

The trick: ! and * can't match any computed hash, so password authentication always fails — that is how they "lock." Note that locking the password does not disable SSH key or other non-password logins.

Interview tip: The empty field is the dangerous one — call it out as a finding, not just trivia.

Entry 1 — devops ALL=(ALL) NOPASSWD: ALL: this grants full root for every command, on every host, without a password. Anyone who gets that shell (or a stolen SSH key, or an RCE in a service running as devops) instantly becomes root with zero friction. The NOPASSWD removes the last re-authentication barrier, so it fails both least-privilege and defense-in-depth.

Entry 2 — wildcard rule (e.g. devops ALL = /bin/systemctl restart *): wildcards are notoriously unsafe because sudo matches on the literal argument string. An attacker can smuggle extra arguments or paths — for instance a crafted unit name or a command that shells out — to break out into a full root shell. Tools like GTFOBins catalog exactly these escapes.

Why visudo: it locks the file against concurrent edits, validates syntax before saving, and refuses to write a broken sudoers. A typo edited directly can lock everyone out of sudo entirely.

Interview tip: Name-drop GTFOBins and "argument injection via wildcards" — that's the L3 signal.

You add a precise, full-path rule via visudo (ideally in a drop-in file under /etc/sudoers.d/). For a user deploy to restart nginx and nothing else:

• deploy ALL=(root) /usr/bin/systemctl restart nginx

Reading it left to right: user deploy, on ALL hosts, may run as (root) exactly the command /usr/bin/systemctl restart nginx. Because the command and its argument are spelled out, sudo rejects anything else — no systemctl stop, no other service, no bare shell.

Key hardening points:

• Use the absolute path (find it with command -v systemctl) so a planted binary on PATH can't be substituted.
• Avoid wildcards — they reopen argument-injection holes.
• To skip the password prompt for automation, prepend NOPASSWD: before the command — but only if the use case truly needs it.

Interview tip: Emphasize absolute path + exact arguments — that's what separates a safe rule from a backdoor.

su ("substitute user") switches you into another account — usually root — by asking for the target account's password, then gives you a full interactive shell as that user. It is all-or-nothing.

sudo runs a single command as another user (root by default), authenticates with your own password, and is governed by fine-grained rules in /etc/sudoers. It also logs every invocation — far better for accountability and least privilege.

PAM (Pluggable Authentication Modules) is the framework both rely on. When you run su or sudo, the program does not check passwords itself — it calls PAM, which reads the matching policy file in /etc/pam.d/ (e.g. /etc/pam.d/sudo). PAM then runs a stack of modules grouped into four management types: auth (verify identity), account (is it allowed/expired), session (set up the session), and password (change rules). This is where you bolt on 2FA, lockout-after-failures, or LDAP.

Interview tip: Say "sudo uses YOUR password and logs; su uses the TARGET's password" — and that PAM is the pluggable engine behind both.

Don't run it. Investigate read-only first:

• Metadata — ls -l for owner (a root-owned SUID file in a user home is a huge red flag), stat for create/modify times, and correlate against when the box was touched.
• Identity — file to confirm it is an ELF, sha256sum the hash and check it against your known-good baseline and threat intel, strings for tell-tale paths like /bin/sh, setuid, or hardcoded IPs.
• Origin — was it dropped by a package? dpkg -S / rpm -qf. If no package owns it, it is an outsider. Check shell history, last, and the auth journal around the create time.

Contain: strip the bit immediately with chmod u-s (neutralizes escalation while preserving evidence), then preserve a forensic copy and move/quarantine the original; isolate the host from the network if compromise is confirmed.

Finally hunt fleet-wide: find / -perm -4000 -type f against your baseline to find siblings.

Interview tip: Lead with "don't execute it" and "chmod u-s contains without destroying evidence" — that maturity is what they're scoring.

The three workhorses are ps, top, and htop. Think of them as a photo versus a live video feed.

• ps takes a one-time snapshot. ps aux lists every process with user, CPU%, memory% and command; ps -ef shows the parent PID (the PPID column). It is ideal for scripting and grepping (ps aux | grep nginx).
• top is a live, auto-refreshing dashboard built into virtually every Linux box. It shows load average and per-process CPU/memory in real time, and lets you press k to kill or M/P to sort by memory or CPU.
• htop is a friendlier, colored top with mouse support, a tree view (F5), search (F3), and easy scrolling. It is not always pre-installed.

Interview tip: Say "ps = snapshot, top/htop = live monitor" — that one line shows you understand the distinction.

Signals are how Linux asks a process to do something. The key difference is whether the process gets a chance to react.

• SIGTERM (15) is the polite "please shut down" — the default of kill. The process can catch it, flush data, close files, and exit cleanly. Always try this first.
• SIGKILL (9) is the "die now" signal. It cannot be caught, blocked, or ignored — the kernel removes the process instantly. The downside: the process runs no cleanup, so you risk corrupted files, orphaned locks, or half-written state. That is why it is a last resort, used only after SIGTERM is ignored.
• SIGHUP (1) originally meant "terminal hung up." Many daemons (nginx, sshd) repurpose it to reload config without restarting.

Analogy: SIGTERM is asking someone to leave the room; SIGKILL is carrying them out mid-sentence.

Interview tip: Always state the escalation order — "SIGTERM first, SIGKILL only if it hangs."

First find the PID (process ID), then send it a signal.

1. Find the PID: pgrep nginx returns just the numbers, ps aux | grep nginx gives full detail, or pidof nginx.
2. Kill by PID: kill 1234 sends the default SIGTERM (polite). If it ignores you, escalate with kill -9 1234 (SIGKILL).
3. Kill by name: killall nginx kills every process named exactly nginx; killall -9 nginx force-kills them. There is also pkill nginx, which matches by pattern.

Quick combo: kill $(pgrep -f myscript.py) finds and kills in one line.

Be careful — killall and pkill hit all matches, so a loose pattern or typo can take down more than you intended.

Interview tip: Mention that kill doesn't only kill — it sends signals; the default is SIGTERM, not SIGKILL.

Both involve a parent-child relationship gone wrong.

• A zombie (state Z in ps) is a child that has already exited but whose exit status has not yet been collected by its parent. It holds no memory or CPU — just a slot in the process table. It appears when a child finishes but the parent has not yet called wait() to "reap" it.
• An orphan is a child whose parent died first. The orphan is still alive and running; the kernel re-parents it to init/systemd (PID 1), which adopts it and later reaps it.

Key insight: you cannot kill -9 a zombie — it is already dead. To clear it you must make the parent reap it (it should call wait(), often triggered by handling SIGCHLD), or kill the parent so PID 1 adopts and cleans up the zombie. Find them with ps aux | grep ' Z ' or ps -el | grep Z.

Analogy: a zombie is a death certificate nobody filed; an orphan is a child whose guardian vanished and was adopted by PID 1.

Interview tip: The trap answer is "kill the zombie" — the correct answer is "reap or kill the parent."

/proc/PID/ is a live window into any running process — perfect for catching malware that lies about its name. Replace PID with the number you are investigating.

• cat /proc/PID/cmdline | tr '\0' ' ' shows the full command line with arguments (fields are NUL-separated, hence the tr) — the real story, including flags and downloaded payloads.
• cat /proc/PID/comm shows the short process name — but this is easily faked by attackers, so never trust it alone.
• ls -l /proc/PID/exe is the golden one: a symlink to the actual binary on disk. If it points to /tmp, /dev/shm, or shows (deleted), that is a major red flag.
• ls -l /proc/PID/fd lists every open file, socket, and pipe — revealing what files it touches and which network connections it holds.
• Bonus: /proc/PID/cwd (working dir) and /proc/PID/environ (env vars, NUL-separated).

Interview tip: Stress that comm can lie but the exe symlink cannot — it reveals the true binary even if the process was renamed.

Both control CPU scheduling priority via the "niceness" value, which ranges from -20 (highest priority, greedy) to +19 (lowest priority, most polite). Higher niceness = nicer to others = less CPU when the system is busy.

• nice sets the niceness when you launch a process: nice -n 19 ./backup.sh starts it at the lowest priority.
• renice changes the niceness of an already-running process: renice -n 10 -p 1234.

For a low-priority batch job (backups, log crunching, compression) use nice -n 19 so it yields CPU to interactive and production work, yet still runs when cores are idle.

Note: only root can set negative (higher-priority) values; an unprivileged user can only raise niceness (be more polite), never lower it.

Analogy: niceness is politeness in a queue — +19 always lets others go first.

Interview tip: Remember the counter-intuitive scale — a higher nice number means lower priority.

I work outside-in, from summary to detail.

1. Status first: systemctl status myservice shows active/failed state, the last few log lines, the exit code, and the ExecStart command. Often the error is right there.
2. Full logs: journalctl -u myservice -n 100 --no-pager for recent entries, or journalctl -u myservice -e to jump to the end. Add -b to limit to this boot and -f to follow live while you retry.
3. Validate the unit: after editing a unit file run systemctl daemon-reload (a classic gotcha — systemd caches the old definition). Inspect the resolved file with systemctl cat myservice.
4. App-specific logs: check /var/log/ (e.g. /var/log/nginx/error.log) for errors systemd does not capture.
5. Common culprits: wrong path or permissions in ExecStart, a missing env file, a port already in use (ss -tlnp), or a non-existent User=.

Interview tip: Always mention daemon-reload after editing a unit — interviewers look for that detail.

These three verbs do separate jobs — a common interview mix-up:

• systemctl start myservice runs the service right now, this session only. After a reboot it will not come back on its own.
• systemctl enable myservice registers it to auto-start on boot (creates a symlink in the boot target). It does not start it now.
• systemctl status myservice just reports state — running or stopped, enabled or disabled, plus recent logs. It changes nothing.

So to make a service start on boot but not immediately, run just systemctl enable myservice (without --now). The shortcut systemctl enable --now myservice does both at once — use that when you want it running immediately and on every boot. The mirror commands are stop and disable.

Analogy: start = switch the light on now; enable = set the timer so it turns on every morning.

Interview tip: Know that enable alone does NOT start the service — that catches many candidates.

Targets are systemd's modern replacement for the old SysV runlevels. A runlevel was a single number (0-6) describing one fixed system state. A target is a named group of units that can depend on other targets, so it is far more flexible.

Rough mapping: runlevel 3 (multi-user, text/networking) maps to multi-user.target; runlevel 5 (graphical) to graphical.target; runlevel 0 to poweroff.target; runlevel 6 to reboot.target; rescue to rescue.target.

• Check the boot target: systemctl get-default.
• Change it permanently: sudo systemctl set-default multi-user.target (e.g. drop a server from GUI to text to save RAM).
• Switch right now without rebooting: sudo systemctl isolate graphical.target.
• See current targets: systemctl list-units --type=target.

For a hardened server, multi-user.target is the usual default — no graphical stack means a smaller attack surface.

Interview tip: Map runlevel 3 to multi-user and 5 to graphical — that mapping is the classic question.

A binary running from /dev/shm (a RAM-backed, world-writable tmpfs that is usually mounted without noexec, making it a favourite drop spot) with a random name is a classic cryptominer/malware pattern. I work detect, investigate, contain — preserving evidence at every step.

1. Detect: spot it in top; grab the PID with ps aux.
2. Investigate via /proc: ls -l /proc/PID/exe confirms it points into /dev/shm (or shows (deleted)); cat /proc/PID/cmdline | tr '\0' ' ' reveals the real arguments; ls -l /proc/PID/fd shows open files; cat /proc/PID/maps and /proc/PID/environ add context. Note the parent with ps -o ppid= -p PID to find how it launched (cron? a web app?).
3. Network forensics: ss -tnp | grep PID exposes C2 or mining-pool connections (IP:port).
4. Preserve evidence: copy the binary before it self-deletes — cp /proc/PID/exe /root/evidence_PID works even if the on-disk file is already gone — then hash it (sha256sum).
5. Contain: isolate the host from the network, kill -STOP PID to freeze (not kill) it for further capture, hunt and remove persistence (cron, systemd units, ~/.ssh, shell rc files), and only then SIGKILL.

Interview tip: Say "freeze with SIGSTOP and copy /proc/PID/exe before killing" — preserving evidence beats reflexively killing.

The modern iproute2 tool is ss (socket statistics). To see what is listening and the owning process, run:

• sudo ss -tulpn — lists all listening sockets with their PIDs.
• To target one port, filter it: sudo ss -tulpn 'sport = :443' or simply sudo ss -tulpn | grep :443.

The flags, each adding one thing:

• -t = TCP sockets
• -u = UDP sockets
• -l = only listening sockets
• -p = show the process/PID that owns the socket (needs sudo to see other users' processes)
• -n = numeric — don't resolve ports/IPs to names (faster, avoids DNS lag)

Interview tip: say "ss -tulpn, and I add sudo so the -p column actually shows the PID."

The old net-tools package (ifconfig, netstat, route, arp) is deprecated and ships uninstalled on most modern distros. The iproute2 suite (ip, ss) replaced it because it talks to the kernel via netlink instead of scraping /proc — so it's faster and exposes features net-tools never learned, such as multiple IPs per interface, policy routing, and network namespaces.

The mapping:

• Show IP addresses: ifconfig → ip addr (or ip a)
• Show/edit routes: route -n → ip route (or ip r)
• Show listening sockets: netstat -tulpn → ss -tulpn
• Neighbour/ARP table: arp -a → ip neigh

Interview tip: mention netlink and "net-tools isn't installed by default anymore" — that signals you actually work on current systems.

"Down" is a symptom, not a cause. Work through the layers in order, ruling out one thing at a time:

1. DNS — does the name even resolve? dig example.com +short. No answer means a DNS/name problem; stop here.
2. Reachability — can packets reach the host? ping example.com, then mtr example.com (or traceroute) to spot where hops die or latency spikes. Note that many hosts now drop ICMP, so a failed ping alone isn't proof the host is down.
3. Port — is the service actually listening/open? Locally ss -tulpn | grep :443; remotely nc -vz example.com 443.
4. Application — does it respond correctly? curl -I https://example.com to read the HTTP status (200 vs 502 vs timeout).

The order is the whole point: each step isolates one layer, so you stop guessing and find the real culprit fast instead of restarting things at random.

Interview tip: frame it as "DNS → reachability → port → app" — interviewers value a repeatable methodology over a lucky guess.

Capture only port 80 on eth0 and write to a file:

• sudo tcpdump -i eth0 -w http.pcap 'port 80'

Read it back later:

• tcpdump -r http.pcap (no sudo needed once it's a file)
• Add -n to skip DNS, -A to show ASCII payload, -X for hex.

Why save to .pcap instead of just watching live?

• You can replay it — open the file in Wireshark for deep, filtered analysis after the fact.
• No data loss — printing decoded packets to the terminal live is slow and can drop packets on busy links; writing raw to disk keeps up.
• Evidence — a saved file is shareable and preserves the full packet for forensics and incident response.

Interview tip: note that the filter goes in quotes (BPF syntax) and that -w stores raw packets — they're only decoded at read time.

All three query DNS, but they differ in detail and intent:

• dig — verbose, scriptable, shows the full DNS response (ANSWER/AUTHORITY sections, TTL, flags, query time). The professional's tool. Example: dig example.com A +short.
• host — a one-line, human-friendly summary: host example.com. Great for a quick "what's the IP?"
• nslookup — older, cross-platform (also ships on Windows), has an interactive mode. Output is less precise and it can obscure which resolver it actually used, so engineers tend to avoid it for serious debugging.

I reach for dig: it's accurate, shows TTLs and record types clearly, lets me target a specific server (dig @8.8.8.8 example.com), and its output is easy to grep in scripts. host is my quick shortcut; nslookup only when I'm on a box that lacks dig.

Interview tip: say "dig for debugging, host for a quick check, nslookup is legacy."

journalctl is the query tool for the systemd journal — a structured, indexed binary log that replaces hunting through scattered text files. Think of it as a searchable database of every log line, not a flat .txt.

The three moves the question asks for:

• Follow one service live: journalctl -u nginx -f (-u = unit, -f = follow, like tail -f).
• Only this boot: journalctl -b (use -b -1 for the previous boot). Combine: journalctl -u nginx -b.
• Errors only: journalctl -p err — priority err and worse. Priorities, most to least severe, are emerg, alert, crit, err, warning, notice, info, debug; you can also use numbers (-p 3).

Useful add-ons: --since "10 min ago", -x for explanatory help text, -o json for machine parsing.

Interview tip: combine filters — journalctl -u ssh -b -p err -f shows live SSH errors from this boot, which is exactly how you debug a flaky service.

Authentication events (logins, sudo, SSH, failed passwords) land in different files depending on the distro family — this is the classic gotcha:

• Debian/Ubuntu: /var/log/auth.log
• RHEL/CentOS/Fedora: /var/log/secure

Same data, different filename — reach for the wrong one and you'll think there are no logs.

Other key files in /var/log:

• /var/log/syslog (Debian/Ubuntu) vs /var/log/messages (RHEL) — the general system-wide log.
• /var/log/kern.log — kernel messages (Debian/Ubuntu).
• dmesg — the kernel ring buffer; read it with the dmesg command (or journalctl -k), not a persisted file on its own.

On systemd boxes most of this is also in the journal (journalctl); the text files still exist when rsyslog is installed, though minimal modern installs may rely on the journal alone.

Interview tip: lead with "auth.log on Debian, secure on RHEL" — that one line proves you've touched both families.

By default journald ships with Storage=auto, which only persists logs to disk if /var/log/journal/ already exists — otherwise it keeps them in /run/log/journal (a tmpfs in RAM) and they vanish on reboot. To guarantee persistence:

1. Set Storage=persistent in /etc/systemd/journald.conf.
2. Ensure the directory exists: sudo mkdir -p /var/log/journal (with Storage=auto, just creating this directory is enough to make logs persist).
3. Apply: sudo systemctl restart systemd-journald.

Now logs live in /var/log/journal and journalctl -b -1 can read previous boots. The journal self-limits its size via SystemMaxUse=.

logrotate handles the text logs in /var/log (rsyslog output, nginx, app logs). Without it, files grow until the disk fills. Driven by /etc/logrotate.conf plus /etc/logrotate.d/*, it rotates (renames to .1, .2…), compresses (gzip), keeps a set number, and deletes the oldest, on a schedule or size trigger.

Interview tip: one line — "journald caps its own size with SystemMaxUse; logrotate manages the plain-text files rsyslog writes." Don't conflate the two.

Quick port check with netcat:

• nc -vz example.com 443 — -v verbose, -z zero-I/O scan (just check open/closed, send no data). Prints "succeeded" or "refused".
• Grab a banner: nc example.com 22 and watch for the SSH version string, or nc example.com 25 for an SMTP server's greeting banner (many services announce themselves on connect).

How nc differs from curl: nc is a raw TCP/UDP pipe — protocol-agnostic, so it works for SSH, SMTP, custom services, or just proving the port is reachable. curl understands application protocols (HTTP/HTTPS, TLS, redirects, headers), so for a web service curl -I https://example.com gives you a proper HTTP status and a real TLS handshake.

Rule of thumb: nc = "is the door open and what greets me?"; curl = "talk the actual web protocol."

Interview tip: note that nc won't do the TLS handshake — for HTTPS banners/headers use curl -v or openssl s_client.

A working command:

• sudo tcpdump -i eth0 -nn -C 100 -W 10 -G 3600 -w 'c2-%Y%m%d-%H%M%S.pcap' 'dst host 203.0.113.50 and not port 22 and not (port 80 or port 443)'

Walk the BPF filter logic:

• dst host 203.0.113.50 — only outbound to the one suspect IP (the C2 endpoint).
• and not port 22 — drop your own SSH session so you don't capture yourself or create an observer effect.
• and not (port 80 or port 443) — exclude normal web ports, leaving the non-standard ports where beaconing often hides.

Capture-hygiene flags:

• -nn — no DNS or port-name lookups (less noise, faster, and avoids tipping off the lookup path).
• -C 100 -W 10 — rotate at 100 MB, keep 10 files (a ring buffer with bounded disk use).
• -G 3600 with a timestamped -w name — also time-rotate hourly, handy for slow, periodic beacons.

Interview tip: stress excluding your SSH port (or you'll drown in your own traffic) and using rotation so a long beacon hunt never fills the disk.

iptables organizes firewall rules into tables (grouped by purpose) and chains (grouped by where in the netfilter path a packet is, i.e. its direction relative to the host).

Key tables:

• filter — the default; decides ACCEPT/DROP/REJECT. This is the actual firewall.
• nat — rewrites addresses/ports (SNAT/MASQUERADE for outbound, DNAT for port-forwarding).

Key chains in the filter table:

• INPUT — packets destined for this host.
• OUTPUT — packets generated by this host.
• FORWARD — packets routed through this host (router/gateway traffic), not destined for it.

Evaluation: rules in a chain are checked top to bottom, first match wins — that rule's target (ACCEPT/DROP) is applied and traversal of the chain stops. If no rule matches, the chain's default policy decides. So order matters: a broad ACCEPT placed above a specific DROP will defeat it.

Analogy: an airport with separate queues (chains) and a checklist per queue read top-down — the first stamp you get is final.

👉 Interview tip: "First match wins, then default policy" is the line; mention that INPUT ≠ FORWARD matters for routers.

nftables is the modern replacement for iptables (and ip6tables, arptables, ebtables). The old stack used separate tools and separate rulesets per protocol family, each with its own kernel match modules. nftables unifies them under one tool (nft) and one ruleset that can handle IPv4 and IPv6 together, evaluated by a single in-kernel virtual machine.

Why teams switch:

• One ruleset — no more maintaining iptables and ip6tables in parallel; less drift, fewer gaps.
• Better performance — atomic rule replacement, and native sets/maps for fast lookups instead of long linear chains.
• Built-in primitives — rate limiting, sets, and counters are first-class (e.g. limit rate 10/minute) rather than bolted-on modules.
• iptables is now largely a compatibility shim (iptables-nft) over nftables anyway.

In an interview, "know the migration" means you can explain the architectural shift, translate old rules with iptables-translate, read the new format with nft list ruleset, and note that modern RHEL and Debian default to nftables (often via firewalld or ufw frontends).

Analogy: replacing four separate filing cabinets with one indexed database.

👉 Interview tip: Say that modern firewalld and ufw are just frontends over nftables — it shows real-world awareness.

Default-deny (whitelisting) means: block everything by default, then explicitly allow only the traffic you need. Its opposite, default-allow (blacklisting), permits everything except what you remember to block — and you will always forget something.

It's the secure default because of fail-safe design: if you miss a rule, the failure mode is "a legitimate service is blocked" (annoying but safe) rather than "an unknown port is exposed" (a breach). New services aren't reachable until you deliberately open them, which shrinks the attack surface and forces intent.

Conceptually for the INPUT chain:

• Set the chain's default policy to DROP.
• Allow loopback (lo) traffic.
• Allow established/related connections (so replies to your own outbound requests return).
• Allow only the specific services you run (e.g. SSH on its port, HTTPS).
• Everything else falls through to the DROP policy.

Analogy: a nightclub with a guest list — no name on the list, no entry — versus letting everyone in unless they're on a banned list.

👉 Interview tip: Always pair default-deny with an "allow established/related" rule, or you'll lock yourself out.

SSH is the most-attacked service on a public host, so harden /etc/ssh/sshd_config:

• Key-only auth — PasswordAuthentication no and PubkeyAuthentication yes. Keys can't be brute-forced like passwords.
• PermitRootLogin no — force login as a normal user, then sudo; removes the universally-known root target.
• Limit who and where — AllowUsers/AllowGroups to whitelist accounts.
• Tighten attempts — MaxAuthTries, LoginGraceTime, and disable unused features (PermitEmptyPasswords no, X11 forwarding off).
• Non-default port — cuts noise from automated scanners (obscurity, not security — minor).

Then reload SSH (sudo systemctl reload sshd, or restart). On modern systemd-socket setups the unit may be ssh.socket — check your distro. Keep a second session open while testing so a bad config doesn't lock you out.

fail2ban complements this by watching auth logs and temporarily banning IPs that fail repeatedly (via firewall rules). Hardening removes the easy doors; fail2ban actively kicks out anyone rattling the locks, throttling brute-force.

👉 Interview tip: Stress that key-only auth and PermitRootLogin no are the high-impact two; the port change is cosmetic.

A crontab line has five time fields followed by the command. In order they are:

• Minute (0–59)
• Hour (0–23, 24-hour clock)
• Day of month (1–31)
• Month (1–12)
• Day of week (0–7, where both 0 and 7 mean Sunday; 1 = Monday)

So 30 2 * * 1 /opt/backup.sh reads: minute 30, hour 2, any day of month, any month, day-of-week 1. That means it runs at 02:30 every Monday. The * wildcard means "every value" for that field.

One subtlety: if both day-of-month and day-of-week are restricted (not *), cron runs when either matches — an OR, not an AND. Here day-of-month is *, so only the Monday condition applies.

Reading hook: think "min hour dom month dow" — left to right, smallest unit up to weekday.

👉 Interview tip: Know the day-of-month/day-of-week OR quirk and that 0 and 7 both mean Sunday.

There are two families of cron jobs:

User crontabs — per-user, owned and edited via the crontab command (never edit the raw spool files by hand). They have five time fields and run as that user.

• Edit yours: crontab -e
• List yours: crontab -l
• Another user (as root): crontab -u alice -e / -l
• Stored under /var/spool/cron/ (the exact path varies by distro — e.g. /var/spool/cron/crontabs/ on Debian).

System crontabs — managed via files, and they include an extra sixth field: the user to run as.

• /etc/crontab — the main system file.
• /etc/cron.d/ — drop-in files (packages put their jobs here).
• /etc/cron.daily/, cron.hourly/, cron.weekly/, cron.monthly/ — directories of scripts (no cron time syntax) run on that cadence by run-parts.

Analogy: user crontabs are personal sticky notes; system crontabs are the building's official posted schedule with "who's responsible" named.

👉 Interview tip: The sixth "username" field in system crontabs (but not user crontabs) is a favorite gotcha.

systemd timers schedule work by pairing a .timer unit (when to run) with a .service unit (what to run). They're the modern alternative because they're integrated into systemd's lifecycle, logging, and dependency model rather than being a separate, opaque daemon.

Why teams prefer them:

• Centralized logging — output goes to the journal (journalctl -u job.service); cron just emails or silently drops output.
• Missed-run handling — Persistent=true runs a job that was missed while the machine was off (cron simply skips it).
• Dependencies and ordering — a timer's service can require the network, a mount, or another unit first.
• Easy status — systemctl list-timers shows next and last run at a glance.

The standout security advantage: the service can be sandboxed with systemd hardening directives — User= to drop privileges, plus PrivateTmp=, ProtectSystem=strict, NoNewPrivileges=, ReadOnlyPaths=. A cron job inherits a broad environment with none of this confinement.

Analogy: cron is a kitchen egg-timer; systemd timers are a smart oven that logs, retries, and locks the controls.

👉 Interview tip: Lead with the sandboxing (least-privilege) advantage — it's the security-relevant differentiator.

Both families split into a low-level tool that handles single package files and a high-level tool that resolves dependencies and talks to repositories.

• Debian/Ubuntu: dpkg (low-level, .deb files) + apt (high-level, repos + dependencies).
• RHEL/Fedora/CentOS: rpm (low-level, .rpm files) + dnf (high-level; yum is the older name and is now a symlink to dnf on current releases).

Install (use the high-level tool so dependencies are resolved):

• Debian: sudo apt install nginx
• RHEL: sudo dnf install nginx

Query if installed:

• Debian: dpkg -l nginx or dpkg -s nginx
• RHEL: rpm -q nginx

Verify integrity (detects changed/tampered files against the package's recorded checksums):

• RHEL: rpm -V nginx — any output flags a changed file (the codes show what differs, e.g. 5 = checksum, M = mode).
• Debian: debsums nginx — note debsums isn't installed by default (apt install debsums first).

Mental model: apt/dnf are the shopping assistants that fetch and arrange everything; dpkg/rpm are the workers that install one box.

👉 Interview tip: Knowing rpm -V / debsums for integrity verification signals security awareness, not just install muscle-memory.

Standard Unix permissions are Discretionary Access Control (DAC): the file owner decides who gets access, and the all-powerful root bypasses everything. SELinux and AppArmor add Mandatory Access Control (MAC): a system-wide policy the kernel enforces that even root and the process owner cannot override.

• SELinux (RHEL default) — label-based: every file, process, and port carries a security context, and policy defines which label may touch which. Powerful and granular, but steeper to learn.
• AppArmor (Ubuntu/SUSE default) — path-based profiles per application; simpler to read and write.

Enforcing mode means the policy is actively blocking violations (versus permissive, which only logs them for tuning). Check with getenforce / aa-status.

In defense-in-depth, MAC is the containment layer: if an attacker exploits, say, a web server and gets a shell as its user — or even escalates to root — MAC still confines that process to only what its policy permits (its own directories, its own ports). The breach is boxed in instead of becoming total compromise.

Analogy: DAC is who has keys to a room; MAC is the building rule that a janitor can never enter the vault — even with a master key.

👉 Interview tip: The killer line: "MAC confines a compromised root process — DAC can't, because root bypasses DAC."

Defense-in-depth means stacking independent layers so that defeating one doesn't compromise the host. For an internet-facing server, working from the network edge inward:

• Cloud / perimeter firewall (security groups) — the first gate; expose only 443 (and SSH from trusted IPs). Blocks traffic before it reaches the OS.
• Host firewall (nftables/firewalld, default-deny) — the second gate, in case the cloud rule is misconfigured or an internal path opens.
• Hardened sshd — key-only auth, PermitRootLogin no, limited users.
• fail2ban — actively bans brute-force IPs that get through the firewall.
• SELinux/AppArmor enforcing — confines a compromised service so a breach can't spread, even as root.
• auditd — tamper-evident logging for detection and forensics.
• sysctl tuning — kernel-level hardening (disable IP forwarding, enable reverse-path filtering, ASLR, ignore ICMP redirects).
• Plus: minimal installed packages, automatic security updates, and least-privilege accounts.

Why one fix is never enough: every control has a failure mode — a misconfiguration, a zero-day, a leaked key, an insider. CIS Benchmarks codify dozens of independent controls precisely because they assume any single one will eventually fail; the next layer must still hold.

Analogy: a castle uses a moat, walls, gates, and guards — breaching the moat shouldn't end the siege.

👉 Interview tip: Frame it as layers with independent failure modes, and name CIS Benchmarks as the auditable standard.

I work the stack bottom-up, confirming each layer before blaming the next.

• Confirm it's down — curl -v http://localhost/ locally and externally to separate "app down" from "network/DNS down".
• Process — systemctl status nginx (and journalctl -u nginx -n 50); is it running or crash-looping?
• Listening — ss -tlnp | grep :80 to confirm it is bound to the right port and interface.
• Resources — df -h (a full disk kills services), free -m, top for CPU/memory/OOM.
• Logs — tail the app and web-server logs (tail -f /var/log/nginx/error.log) for the actual error.
• Dependencies — can it reach its DB/upstream? curl the backend, ss the DB port.
• Firewall/config — nginx -t, then check nftables/iptables or cloud security-group rules.

Like a doctor: check pulse, then breathing, then run tests — not random guessing.

Interview tip: Show a layered method (process to port to resources to logs to deps), not a random command dump.

Step 1 — find the listener and its PID:

• sudo ss -tlnp | grep :443 (or sudo lsof -i :443). This shows the process name and PID bound to 443. Run with sudo or you will not see the PID.

Step 2 — confirm the actual binary behind that PID. The process name shown by ss can be faked, so I verify the real executable on disk:

• sudo ls -l /proc/PID/exe — this symlink points to the true binary path (e.g. /usr/sbin/nginx). A web server pointing to /tmp/x or a deleted file is a red flag.
• sudo tr '\0' ' ' < /proc/PID/cmdline — the exact command line it launched with.

Step 3 — verify integrity: dpkg -S /usr/sbin/nginx (or rpm -qf) to confirm a package owns it, and sha256sum against a known-good hash.

Think of ss as the name on the door and /proc/PID/exe as checking the person's actual ID.

Interview tip: The killer detail is /proc/PID/exe — it can't be spoofed like the displayed process name.

The one-liner (Debian/Ubuntu use /var/log/auth.log; RHEL-family uses /var/log/secure):

• grep "Failed password" /var/log/auth.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn

Walking the pipe stage by stage:

• grep "Failed password" — keeps only the lines for failed SSH logins, dropping all the noise.
• awk '{print $(NF-3)}' — prints the source IP field. In a line ending ...from 1.2.3.4 port 22 ssh2, the IP sits 3 fields before the last, so NF-3 grabs it whether the username is a plain user or an "invalid user".
• sort — groups identical IPs together (required before uniq).
• uniq -c — collapses duplicates and prefixes each with its count.
• sort -rn — sorts by that count, highest first, so the worst offenders bubble to the top.

On modern systemd hosts SSH failures often go to the journal instead of a flat file — there you would start the pipe with journalctl -u ssh --no-pager | grep "Failed password".

Interview tip: Stress why sort must come before uniq (uniq only collapses adjacent lines) — interviewers test that.

The hunt command:

• sudo find / -perm -4000 -type f 2>/dev/null

-perm -4000 matches any file with the SUID bit set (the leading - means "at least these bits"), -type f limits to regular files, and 2>/dev/null hides permission-denied noise on virtual filesystems. Add -ls to see owner and size inline.

Triage against a baseline:

• Capture a list on a clean build: find / -perm -4000 -type f 2>/dev/null | sort > baseline.txt, then diff live output against it — anything new is suspicious.
• Expected SUID binaries (passwd, sudo, mount, su) live in /usr/bin or /usr/sbin and are package-owned — verify with dpkg -V / rpm -V and checksums. (Note: ping is no longer SUID on most modern distros — it uses capabilities.)
• Red flags: SUID files in /tmp, /home, or /dev/shm; odd names; or known escalation binaries made SUID (bash, find, vim, nmap) — cross-check GTFOBins.

Interview tip: The senior move is "diff against a baseline" plus GTFOBins — not just running find.

Step 1 — confirm and locate: df -h shows which filesystem is full. Also run df -i — you can run out of inodes (millions of tiny files) while bytes look fine.

Step 2 — find the offenders on the full mount: du -xh /var --max-depth=1 | sort -rh | head drills down to the biggest directories; repeat into the winner. (-x stays on one filesystem.)

Step 3 — the classic gotcha: df says full but du cannot find the space. That is a deleted-but-still-open file: a process (often a runaway log) kept the file handle, so unlinking the name did not free the blocks. Reveal it with:

• sudo lsof +L1 — lists open files with a link count of 0 (deleted but held open), showing PID and size.

Fix: restart or signal that process — or truncate the held descriptor with : > /proc/PID/fd/N — and the space returns instantly. Like a leaking pipe behind a wall: du cannot see it, but lsof finds the leak.

Interview tip: The deleted-open-file trick (lsof +L1, and why restarting frees it) is the standout answer.

Each command answers a different "who was here" question:

• who — users logged in right now (terminal, login time, source).
• w — same, plus what each session is actively running and load average — great for catching an attacker mid-session.
• last — historical successful logins from /var/log/wtmp; spot odd source IPs, off-hours logins, or unexpected accounts.
• lastb — historical failed logins from /var/log/btmp; a wall of failures then one success screams brute-force.
• ~/.bash_history — the commands a user (or attacker) typed.

The tells: a wiped or empty .bash_history (or one symlinked to /dev/null, or HISTFILE unset) suggests an attacker covering tracks. Missing or zeroed btmp/wtmp means log tampering — itself an indicator of compromise. Absence of evidence here is evidence.

Interview tip: The key insight: deleted logs/history aren't innocent — anti-forensics is a compromise signal. Always corroborate with centralized/remote logs (and the systemd journal) an attacker can't reach.

The root cause is almost always that cron runs in a stripped-down environment, not your interactive shell. The usual suspects:

• Minimal PATH — cron's PATH is tiny (often just /usr/bin:/bin), so a bare python3 or aws is not found. Fix: use absolute paths or set PATH= at the top of the crontab.
• Missing environment — no .bashrc/.profile is sourced, so vars, virtualenvs, and aliases you rely on do not exist. Set them explicitly in the script.
• Relative paths / wrong CWD — cron starts in the user's home, not where the script lives. Use absolute paths or cd first.
• No TTY — anything expecting a terminal (interactive prompts, sudo with a password) fails.

Debug it: redirect output to a log — * * * * * /path/script.sh >> /tmp/cron.log 2>&1 — and read the error. To reproduce cron's bare environment manually, run env -i /bin/sh -c '/path/script.sh'.

Interview tip: The one-line gold: "cron has a minimal env — capture stdout+stderr to a log and reproduce with env -i."

Detect — confirm the beacon: ss -tunp to list active connections and the owning PID; look for regular-interval traffic to an unknown IP/domain (a classic C2 heartbeat).

Investigate — pivot from the PID into /proc: ls -l /proc/PID/exe (true binary; a deleted-but-running ELF is a huge red flag), /proc/PID/cmdline, /proc/PID/cwd, and /proc/PID/maps for injected or anonymous executable regions. Check world-writable drop zones /tmp, /dev/shm, /var/tmp. Run rkhunter and chkrootkit for known rootkits. Fileless/in-memory malware hides as a process whose exe link shows "(deleted)", code mapped only in /proc/PID/maps, or payloads launched from /dev/shm or an anonymous memfd — never landing on disk.

Contain — isolate the network (block the C2 IP, or pull the host), but capture volatile memory and a disk image first to preserve evidence.

Eradicate — kill the process, remove persistence (cron, systemd units, ~/.bashrc, authorized SSH keys), close the entry vector.

Recover — rebuild from a known-good image (rootkits make in-place trust impossible), rotate credentials, monitor.

Interview tip: Two senior signals: capture volatile memory before containment, and "exe shows (deleted)" = fileless tell.

Why Sigma: it is a vendor-neutral YAML rule format — write the detection logic once, then convert it (with the sigma CLI / pySigma backends) into Splunk SPL, Sentinel KQL, Wazuh, or Elastic queries. That is detection-as-code: the rule lives in Git, gets peer-reviewed in a pull request, and is versioned like any other code.

Authoring from this incident: I map the observed telemetry to a Sigma rule — a logsource (e.g. category: process_creation, or the auth log source) and a detection block whose selection matches the IOCs we saw: the malicious process path in /tmp or /dev/shm, the beacon parent-child chain, or the brute-force-then-success pattern from the auth log. I add level, MITRE ATT&CK tags, and a falsepositives note.

Validate: replay the incident telemetry (or safely re-run the attack in a lab) and confirm it fires; run it over historical logs to tune false positives; have a teammate review the pull request before it merges and deploys to the SIEM.

Interview tip: Hit the buzzwords: vendor-neutral YAML, Git-versioned detection-as-code, MITRE ATT&CK mapping, and validate by replaying telemetry + tuning false positives.