F5 Bot Defense — Proactive Bot Defense, Signatures, Device ID & CAPTCHA

① Where Bot Defense lives — its own profile, no security policy needed

Picture a nightclub door. A weak bouncer checks the name on the guest list (the User-Agent) — easy to fake. A real bouncer makes you do something a gate-crasher can't: show ID, answer a question, walk in a straight line. F5 Bot Defense is that real bouncer. It doesn't trust what the client says; it tests what the client can do.

First, scope. Bot Defense is enforced through a dedicated Bot Defense profile — and crucially, it does not require a security policy. You can protect an app from bots without ever building an ASM positive-security policy. The profile is created under Security ▸ Bot Defense ▸ Bot Defense Profiles and attached to a virtual server, exactly like any other profile. (Older BIG-IP versions configured "Proactive Bot Defense" inside the DoS profile; modern versions consolidate everything into this unified Bot Defense profile.)

Bot Defense decides using three layered techniques: Proactive Bot Defense (challenge the client), bot signatures (match known patterns), and anomaly detection (catch automation tells). Together they sort every client into one of six classes, and each class gets its own mitigation. The art is knowing which technique catches which bot.

② Proactive Bot Defense — make the client prove it's a browser

The strongest weapon in the profile is Proactive Bot Defense. Instead of waiting to recognise a bad bot, it challenges every new client up front. Two things happen before the app ever sees the request:

• JavaScript challenge — BIG-IP injects a small piece of JavaScript the client must run and return a valid token for. A real browser executes it in milliseconds, invisibly. A curl, a requests script, or a basic bot has no JavaScript engine, so it simply can't answer.
• Device ID — BIG-IP fingerprints the verified client and issues a Device ID cookie. From then on, that exact client is recognised across requests — even if it rotates through a thousand IPs.

This is why it's "proactive": the verdict is reached before a single request hits your application logic. Web scraping, credential stuffing, and L7 floods all rely on clients that can't run JavaScript or won't carry a consistent Device ID — and they're filtered at the door.

You tune when the challenge happens with Browser Verification: Challenge-free (no proactive challenge — basic), Verify After Access (challenge after the first request — lower latency, the Balanced default), or Verify Before Access (challenge before any request reaches the app — strongest, the Strict default). Device ID Mode mirrors this with None / Generate After Access / Generate Before Access.

Analogy: a bouncer who doesn't read the guest list at all. He hands you a puzzle at the door. Real guests solve it without thinking; a gate-crasher who only memorised a name on the list is stuck on the step.

③ Signatures, anomalies & the six client classes

Proactive Bot Defense asks "can you prove you're a browser?". The other two techniques answer "which bot are you, and are you a good one?".

Bot signatures — recognise the known, good and bad

Bot signatures look for known patterns in the request — primarily the headers — to identify a bot by name. They come in two flavours, and you can write your own:

• Benign / known-good categories — Search Bot, Crawler, HTTP Library, Headless Browser, RSS Reader, Site Monitor, Social Media Agent, Service Agent. Googlebot and Bingbot land here.
• Malicious categories — DOS Tool, Vulnerability Scanner, Network Scanner, Web Spider, Spam Bot, Spyware, E-Mail Collector, Exploit Tool.

Signatures are updated by F5's Live Update. Newly updated signatures go into staging first — requests that match a staged signature are logged but not mitigated, so you can review for false positives before it enforces. Move it out of staging to start blocking.

Anomaly detection — catch the automation tells

Signatures miss two things: a brand-new bot with no signature, and a bot that fakes a normal browser header. Anomaly detection closes that gap with behavioural categories like Browser Automation, Headless Browser Anomalies, Browser Masquerading, Search Engine Masquerading (a bot pretending to be Googlebot), and Mobile App Automation. This is what catches a Puppeteer bot that survived the JS challenge.

The six client classes — who gets what

Everything above feeds one decision: which class is this client, and therefore what action applies?

• Browser — a verified real browser. Usually allowed (or CAPTCHA'd in Balanced, Verify-Before in Strict).
• Trusted Bot — a verified good bot (search engine, monitor). Default action is Alarm only — never blocked. (Note: there is no anomaly detection for Trusted Bots — even if anomalies are found, the request passes.)
• Untrusted Bot — a known automated client that isn't on the trusted list — Alarm / Rate limit / Block depending on template.
• Suspicious Browser — looks browser-ish but failed verification or tripped anomalies — Alarm / CAPTCHA / Block.
• Malicious Bot — matched a malicious signature or category — Blocked in every template.
• Unknown — unclassified — None / Rate limit / Block depending on template.

Analogy: the bouncer has a colour-coded wristband for everyone. Green for guests (Browser), blue for the catering staff he knows (Trusted Bot), red for the guy with the crowbar (Malicious Bot). The action follows the band, not a gut feeling.

The Bot Defense techniques & key controls — tap each card

Each card front names the control; the back gives you the "so what" — when it earns its place.

④ Templates & build — pick a starting point, then tune

You don't build the per-class action matrix from scratch. F5 ships three profile templates that set sensible defaults; you pick the closest one to your risk, then tune. They differ mainly in Browser Verification strictness and the actions for the in-between classes.

• Relaxed — challenge-free browser verification, Device ID Mode None. Blocks malicious bots (mostly via signatures), alarms the rest. Lowest false-positive risk, lightest protection.
• Balanced — Verify After Access, Generate Device ID After Access. Advanced browser verification with lower latency: blocks malicious bots, CAPTCHAs suspicious browsers, and rate-limits unknown bots.
• Strict — Verify Before Access, Generate Device ID After Access. Strongest: blocks all non-trusted bots, and browsers must pass proactive verification before any request reaches the app.

The per-class action — same six classes, different verdicts

Whichever template you pick, the action attaches to the client class. Read this matrix top-to-bottom — the further right you go, the more aggressive the verdict:

• Trusted Bot — Alarm in all three (a search engine is never blocked).
• Malicious Bot — Block in all three (no template lets a scanner through).
• Suspicious Browser — Alarm (Relaxed) → CAPTCHA (Balanced) → Block (Strict).
• Untrusted Bot / Unknown — Alarm or None (Relaxed) → Rate limit (Balanced) → Block (Strict).
• Browser — Challenge-free (Relaxed) → CAPTCHA (Balanced) → Verify Before Access (Strict).

CAPTCHA & the actions you can attach

The actions available per class are None, Alarm (log only), Slowdown / Rate Limit, CAPTCHA (a human-verification step for clients that look suspicious but might be real), and Block. CAPTCHA is the polite middle ground — it gives a real-but-flagged user a way through instead of a hard block.

The strict-enforcement & API knobs

Two settings matter most when real non-browser traffic shares the virtual server:

• API Access for Browsers — requires a valid cookie for a browser's AJAX/XHR calls to your REST API, so a verified browser's API calls keep working while raw bots don't.
• DoS Attack Mitigation Mode — during a detected attack, escalates to "block all bots, Verify Before Access for browsers" for the duration.
• Whitelist / Microservice — exempt specific sources or URLs (your server-to-server API, a trusted partner) from proactive verification, so you don't block legitimate non-browser clients.

The screen you'll live in — the Bot Defense profile

Here's the exact form. You create the profile, pick the template, and set Browser Verification + Device ID Mode + the per-class Mitigation. The four numbered pins are the fields this lesson changes.

After you save and attach the profile, you don't trust the config — you read the report. This is where you confirm what actually fired:

tmsh — the same profile from the CLI

The GUI is the day-job, but the same settings exist in tmsh. Useful for review, automation, and the 303 exam. Confirm the Device ID mode and the profile binding:

root@(bigip-flipkart)(cfg-sync)(tmos)# list ltm virtual vs_shop_443 profiles
root@(bigip-flipkart)(cfg-sync)(tmos)# list security bot-defense profile bot-flipkart-shop deviceid-mode
root@(bigip-flipkart)(cfg-sync)(tmos)# list security bot-defense profile bot-flipkart-shop template

ltm virtual vs_shop_443 {
    profiles { bot-flipkart-shop { } http { } }
}
security bot-defense profile bot-flipkart-shop {
    deviceid-mode generate-after-access
}
security bot-defense profile bot-flipkart-shop {
    template balanced
}

root@(bigip-flipkart)(tmos)# show security bot-defense profile bot-flipkart-shop
root@(bigip-flipkart)(tmos)# show analytics dos-l7 report view-by device-id

Security::Bot Defense Profile bot-flipkart-shop  (VS: vs_shop_443)
  Browser            41802  action allow    (proactive: JS + Device ID)
  Trusted Bot         2140  action alarm    (signature: Search Bot / Googlebot)
  Suspicious Browser   658  action captcha  (anomaly: Headless Browser)
  Malicious Bot       9317  action block    (signature: Web Spider / DOS Tool)
# Bad client "alarm" not "block" → signature in staging, or template too loose.
# No bot rows at all → profile not bound to the virtual server.