BeyondTrust Deployment, HA & DR: — Appliances, Clusters & Surviving Failures

① Deployment shapes — vault pairs, cold spares, brokers and Atlas

Aditya at HCL has just been handed the deployment decision: the new Password Safe estate will hold every domain admin, every root password, every firewall enable secret in the company. The uncomfortable maths: the vault is now the single most attractive failure point in the building. The deployment shape you pick decides what happens the night it dies — so let's walk the shapes BeyondTrust actually ships, not the marketing diagram.

For Password Safe on-prem, the unit is the U-Series appliance. Resilience comes in two flavours. An HA pair is two appliances: the primary serves everything, internal database replication copies data across, and a heartbeat from the primary tells the secondary when to take over. A cold spare is the budget cousin: a standby appliance you restore from backup — recovery measured in hours and your data loss equals the age of the last backup. Honest numbers, lower bill.

Third shape: Password Safe Cloud (your tenant at hcl.ps.beyondtrustcloud.com). BeyondTrust runs and patches the vault; you run Resource Brokers inside your network, grouped into resource zones. Each broker installs nine services (rotation, discovery, directory auth, session monitoring and friends) and makes outbound-443-only connections to the cloud — no inbound firewall holes. Cloud moves the vault's uptime onto BeyondTrust's shoulders, but your zone is still yours: docs recommend 2+ brokers per zone, each with 16 GB RAM minimum and a 64 GB session-cache disk.

PRA (the B Series appliance, physical or virtual, usually in the DMZ) follows the same logic: a failover pair of appliances for resilience, configured under /login > Management > Failover. When the estate goes global, PRA scales out with Atlas clustering: one primary node is the configuration point, and traffic nodes in each region carry session load. TCP 443 must be open bidirectionally between all cluster appliances, and each node can publish a separate Public Address (for users) and Internal Address (for appliance-to-appliance sync). The Atlas capacity tier is real scale: 300–3,000 users and 50,000–250,000 endpoints.

Sizing is the part interviews love because it is money. Sessions: a Jumpoint host with 8–12 cores and 32–64 GB RAM handles roughly 20–25 concurrent RDP sessions (or ~200 SSH/Telnet) — recording RDP with your own tools costs about 4 cores per 5 sessions. Recordings: session logs and recordings stay on the PRA appliance for up to 90 days, then you export via the Integration Client (recordings as .flv, logs as .xml) to SQL Server or a file share. Undersize the broker's 64 GB session-cache disk and session monitoring dies first — a classic week-one surprise.

Four shapes you will quote in design meetings

Tap each card — these four words decide your 2 AM experience.

② HA mechanics — heartbeat, replication and the failover you rehearse

The U-Series HA model is active/passive. The primary serves users, agents and APIs; the secondary replicates databases and otherwise stays quiet. The trigger is beautifully simple: the primary sends a heartbeat, and when that heartbeat stops arriving, the secondary takes over. Silence is the signal — no human presses a failover button at 2 AM.

Now the gotcha that fails real drills: HA replicates only the databases of features that were enabled when HA was configured. Enable Password Safe (or Secrets Safe) six months after pairing, and its database quietly never joins replication — the HA dashboard still says Healthy, because the pair itself is healthy. The rule to tattoo: enable features first, pair last — or re-establish HA after turning anything new on.

POST https://ps.hcl-lab.in/BeyondTrust/api/public/v3/Auth/SignAppIn
Authorization: PS-Auth key=c0ffee9a...e1; runas=HCL\\svc-drcheck;

GET https://ps.hcl-lab.in/BeyondTrust/api/public/v3/Configuration/Version

HTTP/1.1 200 OK            <- SignAppIn: session established on the NEW active node
HTTP/1.1 200 OK
{ "Version": "25.2.0.1234" }
-> vault is answering after takeover; now run one checkout + one rotation test

③ Backups & upgrades — copies you can restore, windows you survive

Two backup species, two jobs. A config backup captures settings, policies and wiring — small, fast, take it before every change window. A full backup carries the databases: managed systems, accounts, the credential store, audit history, recordings metadata — large, scheduled, and the thing you rebuild a vault from. The U-Series Business Continuity guidance and every grown-up DR standard agree on placement: backups do not live on the appliance. Encrypted copies go to separate storage, ideally a second site — because whatever kills the appliance (fire, ransomware, a disk controller with opinions) kills anything stored on it in the same instant.

Upgrades are the other planned emergency. The field-tested order for an on-prem U-Series estate: (1) suspend HA failover first — otherwise a mid-upgrade reboot looks exactly like a dead primary and the pair flips underneath you; (2) update SUPI (the update installer itself) before the appliance software; (3) upgrade the appliance/BeyondInsight + Password Safe; (4) resume HA and verify replication; (5) then the outer ring — Resource Brokers, agents and Jump Clients (PRA pushes client auto-upgrades in bandwidth-throttled waves), and desktop consoles last. Servers before agents, agents before consoles.

Version policy is part of the plan, not trivia: direct upgrades to BI/PS 25.2 are supported from 23.2 or later — older estates hop through an intermediate version first — and the platform wants SQL Server 2016 SP2+. Watch deprecations in the release notes too (mTLS is being phased out; client certificates are being retired as an API auth method). And remember the patching split from the CVE lesson: cloud tenants were auto-patched on 2024-12-16 during the CVE-2024-12356 emergency, while on-prem owners applied the fix themselves via the /appliance interface. Self-hosted means you own the patch SLA.

Test-NetConnection ps-secondary.hcl-lab.in -Port 4489   # RDP proxy
Test-NetConnection ps-secondary.hcl-lab.in -Port 4422   # SSH proxy

ComputerName     : ps-secondary.hcl-lab.in
RemoteAddress    : 10.20.5.12
RemotePort       : 4489
TcpTestSucceeded : True
(repeat shows 4422 True — both session-proxy listeners answering on the new active node)

④ DR thinking — the vault is tier-0, plan for the day it is gone

Here is the trap your own success builds: once PAM is rolled out properly, no human knows any privileged password — they are vaulted, rotated, injected. Brilliant on a normal Tuesday. Catastrophic logic on disaster day: if PAM is down, nobody can log in to fix anything — including the systems PAM runs on. AD recovery needs credentials that live in the vault; the vault may need AD to authenticate its admins. That circular dependency is why the vault is tier-0 infrastructure, same shelf as your domain controllers.

The answer is the oldest control in banking, done digitally: break-glass. Keep a tiny set of emergency accounts (a domain admin, a hypervisor root, the vault's own local admin) whose credentials live in an offline escrow that does not depend on the vault. The classic form is sealed envelopes in a physical safe — the modern form is the same ceremony, digital: an encrypted file or offline password store whose unlock is split between two custodians, exactly like an SBI bank locker needing your key and the manager's key. Three non-negotiables: alarmed (any use pages the SOC — the fire-alarm glass box rings when smashed), rotated after every use (the envelope is single-use), and tested on a schedule — a sealed envelope with last year's password inside is theatre.

GET https://ps.hcl-lab.in/BeyondTrust/api/public/v3/ManagedAccounts
# filter client-side: LastChangeDate newer than the restored backup's timestamp

HTTP/1.1 200 OK
[ { "AccountName": "adm-db01",  "LastChangeDate": "2026-06-10T02:14:00Z" },
  { "AccountName": "root-web04", "LastChangeDate": "2026-06-10T03:02:00Z" } ]
-> both rotated AFTER last night's 01:00 backup: vault now holds stale secrets
-> queue these for password test + re-rotation before declaring recovery done

Now the management conversation, because DR is a business decision wearing a technical costume. RTO: how many hours can the company run with no checkouts, no brokered sessions, no rotations? Frozen change windows, stalled vendor access, engineers locked out — that is money per hour. RPO: how many hours of rotations and audit trail can you lose — knowing each lost rotation is a reconciliation job? The shapes map cleanly: HA pair buys minutes of RTO and near-zero RPO; cold spare buys hours and last-backup; backup-only means a day or more. Present those three rows with prices and let management pick — then get the chosen numbers signed. An RTO that lives only in your head is yours to be blamed for; one signed in writing is a budget.