TTechclick All lessons
Check Point · Site-to-Site VPN · IPsecInteractive · L1 / L2

Check Point Site-to-Site VPN — Star vs Mesh, IKEv1/v2, and the Encryption-Domain Trap

Phase 1 done. Phase 2 done. Tunnel UP — but the peer rejects every packet with "according to the policy the packet should not have been decrypted". The answer almost never is DH groups. It's encryption domain. Pick a community type below, watch the IKE handshake live, master it in 12 minutes.

📅 2026-05-26· ⏱ 12 min · 5 SVG infographics + 1 animated handshake· 🏷 10-Q Bloom-tiered assessment + AI Tutor inline

Pick a topic — jump straight to it

1

Star vs Mesh

When to pick which. Scales O(n²) vs O(n).

 2

IKE Phase 1+2

Main / Aggressive / IKEv2. The handshake that 60% of tickets are about.

 3

Link Selection

Wrong source IP behind ISP NAT — the classic INVALID_ID fix.

 4

Debug toolkit

vpn debug ikeon, IKEView, vpn tu. The 60-second playbook.

The wrong answer 80% of candidates give

Interview: "Tunnel is UP. No traffic. Both peers see Phase 2 SAs installed. Why?"
Wrong answers: "DH group mismatch", "MTU", "Anti-spoofing". Right answer: the data plane is hitting Hide-NAT on egress to the tunnel; peer sees a public source IP that isn't in its encryption domain; peer drops as out-of-domain decrypt. The IKE phases finishing successfully tells you crypto + auth + peer-ID are all fine. The data plane fail is almost always encryption-domain or NAT-in-VPN.

💡 The chai-tapri handshake (the only mental model you need)

Two strangers at a Mumbai chai-tapri want to talk business privately. Phase 1 = namaste + showing ID card + agreeing on a hand-signal alphabet ("we'll both speak in Marathi reversed"). Main Mode = whisper IDs so no one nearby sees who they are. Aggressive Mode = shout names to save time (anyone listening learns who they are). Phase 2 = the actual deal terms ("for next 1 hour, every cup we order will be coded as 'kadak special'"). Without Phase 1 trust, Phase 2 never starts. Without the right "service area PIN-codes" — the encryption domain — the courier (DTDC) refuses to pick up the parcel.

① Star vs Mesh — the topology choice

Two community shapes in SmartConsole:

Star vs Mesh VPN community topology comparison Left: Star — a central hub with 5 satellites connected only to the hub. Right: Mesh — 5 nodes fully interconnected with every node connected to every other. STAR — n-1 tunnels MESH — n(n-1)/2 tunnels DC Center Br-1 Br-2 Br-3 Br-4 Br-5 GW-A GW-B GW-C GW-D GW-E 10 sites = 9 tunnels · central inspection point 5 sites = 10 tunnels · scales explosively
Figure 1 — Star vs Mesh topology. Star = central inspection, Mesh = flat partner-to-partner. Production rule of thumb: more than 20 sites → Star.

② IKE Phase 1 + 2 — the handshake every interview tests

Phase 1 (IKE SA / ISAKMP SA) authenticates peers and builds the encrypted control channel. Phase 2 (IPsec SA / Child SA) negotiates the actual data-encryption parameters.

IKE Phase 1 (Main Mode) and Phase 2 (Quick Mode) handshake timeline Two vertical lanes labeled Initiator and Responder. Six Phase 1 message arrows: SA propose, SA accept, KE+Nonce, KE+Nonce, ID+Auth, ID+Auth. Then three Phase 2 messages: HASH+SA, HASH+SA, HASH ACK. Then ESP encrypted data flow. INITIATOR (Mumbai GW) RESPONDER (Bengaluru GW) PHASE1 PHASE2 ① SA Propose (encr/hash/DH/lifetime) ② SA Accept ③ KE + Nonce (Diffie-Hellman exchange) ④ KE + Nonce ⑤ ID + AUTH (encrypted — PSK / cert) ⑥ ID + AUTH → IKE SA established ⑦ Quick Mode — HASH + SA (proxy-ID, PFS, lifetime) ⑧ HASH + SA ⑨ HASH ACK → Phase 2 SA installed ★ Encrypted ESP data flows
Figure 2 — IKE Phase 1 (6-msg Main Mode) + Phase 2 (3-msg Quick Mode) timeline. IKEv2 collapses Phase 1 into 4 messages but the concept is identical.

Default Phase-1 crypto (modern community): AES-256 / SHA-256 / DH Group 14 (2048-bit), lifetime 1440 min. Phase 2: AES-256 / SHA-256, lifetime 3600 s, PFS Group 14 recommended in production.

▶ Watch the encryption-domain mismatch fail

Tunnel UP. LAN packet leaves Mumbai GW → gets Hide-NAT'd → ESP encrypted → arrives Bengaluru GW → rejected. Press Play to see exactly where it dies.

① LANRahul's laptop at Mumbai HQ: 10.10.5.50:51234 → 10.50.5.20:443 (Bengaluru DC server)
▼
② HIDE-NAT FIRESAuto Hide NAT for 10.10.0.0/16 rewrites source → 203.0.113.5 (public). This is the bug.
▼
③ ENCRYPTESP-encrypted: outer = 203.0.113.5 → peer-public. Inner = 203.0.113.5 (post-NAT) → 10.50.5.20.
▼
④ DECRYPT @ BENGALURUBengaluru's encryption domain expects source from 10.10.0.0/16. Inner source is 203.0.113.5 — not in domain.
▼
⑤ DROPLog: "according to the policy the packet should not have been decrypted". Fix: community Advanced → tick "Disable NAT inside the VPN community" → Install Policy → vpn tu menu 7 to flush SAs.
Press Play to watch the encryption-domain failure step by step.

4 concepts every interview asks about

🤝
Main vs Aggressive
tap to flip

Main Mode: 6 messages, identities encrypted. Aggressive Mode: 3 messages, identities cleartext + DH parallel with auth. Only use Aggressive with peers that can't do Main. IKEv2 = 4 msgs, identities encrypted, native NAT-T.

🏷
VPN Domain
tap to flip

The subnets you announce as "behind this gateway". Peer's Phase-2 proxy-ID is built from this. Topology-auto = risky on new VLANs. Manually-defined group = production default.

🔁
DPD vs tunnel_test
tap to flip

Permanent tunnels with non-CP peers (Cisco/Forti/AWS) → use DPD (RFC 3706). Check Point's proprietary tunnel_test on UDP/18234 only works peer-to-peer with another CP gateway.

📦
NAT-T
tap to flip

NAT-Traversal encapsulates ESP inside UDP/4500 when a NAT device is detected between peers. Auto-detected via vendor IDs + NAT-D in Phase 1. Required firewall openings: UDP/500, UDP/4500, IP proto 50.

Quick check · Q1 of 10

In Phase 1 Main Mode, after which message pair are the peer identities FIRST encrypted?

Correct: b. Diffie-Hellman exchange in messages 3-4 produces the shared secret. Messages 5+ encrypt with it, so IDs go encrypted. Aggressive Mode skips this gap by sending ID in message 1 — exposing it.

Behind ISP NAT? Gateway has internal IP 10.0.0.1 but the peer sees you as 203.0.113.5 (the ISP's published NAT). If Link Selection is left on auto, the gateway sources IKE from 10.0.0.1 — peer drops with INVALID_ID_INFORMATION.

Link Selection decision tree Decision diamond asking if gateway is behind a NAT device. Yes branch leads to Statically NATed IP option. No branch leads to Calculate IP based on routes (default). Configure VPN peer Link Selection menu Behind a NAT device? YES "Always use this IP address" → Selected: Statically NATed IP Match the public IP the peer sees NO "Calculate IP based on routes" → default — picks egress IP per route lookup Use MEP (Multiple Entry Point) when 2+ HA gateways protect the same VPN domain.
Figure 3 — Link Selection decision tree. Behind ISP NAT? Pin source IP to the statically-NATed public IP. Otherwise let the gateway calculate by route.
Quick check · Q2 of 10

Sneha at Infosys configures 25 branch sites with a central DC. All branch-to-branch traffic must inspect through the DC. Which community type and key setting?

Correct: c. Star + central inspection is the canonical pattern for 20+ sites with DC inspection. Mesh wastes tunnels and removes the central inspection point. VTI for 25 branches without an automation tool is operationally heavy.

④ The debug toolkit — the 60-second playbook

NAT-T encapsulation diagram — ESP wrapped inside UDP 4500 Three nested boxes showing original packet inside ESP inside UDP/4500. NAT-Traversal — ESP inside UDP/4500 IP HEADER (outer) src 203.0.113.5 → dst 198.51.100.10 UDP 4500 (NAT-T) ESP (proto 50) — encrypted payload [ inner: 10.10.5.50 → 10.50.5.20 ] Required ports: UDP/500 (IKE), UDP/4500 (NAT-T), IP proto 50 (ESP raw)
Figure 4 — NAT-T encapsulation. When a NAT device sits between peers, ESP gets wrapped inside UDP/4500 so NAT can re-write the outer header without breaking the inner ESP integrity.

The CLI you actually use

Live IKE debug — start fresh capture 
vpn debug trunc                    # truncate old logs + start clean
vpn debug ikeon                    # turn on IKE debug (writes $FWDIR/log/ike.elg)
# ... reproduce the failure (push a packet across the tunnel) ...
vpn debug ikeoff                   # turn off
Live tunnel inspection 
vpn tu                             # interactive menu — list IKE SAs / IPsec SAs
                                   # menu 7 = delete all SAs for a peer (force rekey)
cpview -t                          # live VPN counters — encrypt/decrypt errors
Sample vpn tu output (option 1 — IKE SAs) 
Peer: 198.51.100.10  Phase 1: AES-256/SHA-256/DH14  Lifetime: 24h  State: Established
Peer: 198.51.100.10  Phase 2: AES-256/SHA-256/PFS14 Lifetime: 1h   State: Active

Open ike.elg in IKEView.exe on Windows. Decoded errors you'll see: NO_PROPOSAL_CHOSEN (DH/encryption mismatch), INVALID_ID_INFORMATION (Link Selection picking wrong source IP), AUTHENTICATION_FAILED (PSK or cert chain).

Quick check · Q3 of 10

Aditya's gateway sits behind ISP NAT. Peer drops with INVALID_ID_INFORMATION in ike.elg. What's the fix in SmartConsole?

Correct: a. Link Selection picking the internal IP is the textbook cause of INVALID_ID when behind NAT. Pin the source to the public IP. (b/c/d) don't fix identity — they fix completely different failure modes.

Route-based VPN (VTI) — when domain-based stops scaling

Domain-based VPN works on proxy-ID: "encrypt if dst is in peer's VPN domain". Rigid. Doesn't support OSPF/BGP across the tunnel. For dynamic routing (cloud VPN, multi-path failover, asymmetric routes), use VTI:

Domain-based vs Route-based VPN decision matrix Two-column comparison showing domain-based on left and route-based VTI on right with use cases. Domain-based VPN Route-based VPN (VTI) ✓ Simple — just list subnets ✓ Default for Star/Mesh ✗ No dynamic routing ✗ Rigid proxy-ID ✗ Cloud peers limited Use when Branch ↔ DC, static subnets, no dynamic routing needed ✓ OSPF / BGP over tunnel ✓ Per-tunnel monitoring ✓ Required for Azure/AWS ✓ Asymmetric routing fix ✗ More config overhead Use when Cloud VPN, dynamic routing, hub-and-spoke with iBGP
Figure 5 — Domain-based vs Route-based decision matrix. Pick domain-based when peers + subnets are static and on-prem; pick VTI when you need OSPF/BGP, cloud peers, or asymmetric routing fixes.

🤖 Ask the AI Tutor

Tap any question — instant context-aware answer drawn from this lesson + Check Point R81.20 docs + CheckMates threads.

Pre-curated answers from R81.20 admin guide + CheckMates. Deeper questions → chat.techclick.in.

The 5 mistakes that cost L1 candidates the VPN ticket

Mistake 1 — Missing subnet in VPN domain

New VLAN added behind firewall, forgot to add to manual VPN domain group. Symptom: "VPN works for some servers but not the new one". Always use a manually-defined group; never trust topology-auto in production.

Mistake 2 — NAT-in-VPN forgotten

"Disable NAT inside the VPN community" not ticked → Hide-NAT rewrites tunnel traffic → peer rejects on encryption-domain mismatch. Hard rule: tick it for every domain-based community where peer expects pre-NAT subnets.

Mistake 3 — Permanent Tunnel false-DOWN

Non-CP peer doesn't reply to tunnel_test. Switch monitoring mode to DPD; configure DPD on peer.

Mistake 4 — DH group mismatch

Modern peer uses DH14+, legacy gateway proposes DH2. NO_PROPOSAL_CHOSEN in ike.elg. Align community encryption suite to AES-256/SHA-256/DH14 (or Suite-B GCM).

Mistake 5 — MTU/MSS over AES-GCM

Tunnel up, small TCP works, large transfers stall. PMTUD blocked through the tunnel. Set VTI MTU 1400, TCP MSS clamp 1350. fw ctl set int fw_clamp_tcp_mss 1.

The 2024 lesson — CVE-2024-24919

April 30, 2024 — Quantum Security Gateway info-disclosure (CVSS 8.6) actively exploited from day one. Even pure S2S boxes were vulnerable if the IPsec-VPN blade was enabled. Hotfix in sk182336. Post-incident hygiene: patch the gateway, rotate ALL local passwords + certs, audit AD service accounts. Interview-grade answer: "I patch IPsec-VPN-enabled gateways within 24h of any CVE notification and keep the Mobile Access blade disabled unless required".

📝 Check your understanding — 10 questions, 70% to pass

Q1–Q3 above already count. Below are Q4 to Q10. Score saves to your profile.

Q4 of 10 · Remember

Which 3 protocols/ports must be open between IPsec peers for a NAT-T site-to-site VPN?

Correct: d. UDP/500 carries IKE control plane. UDP/4500 carries ESP-encapsulated-in-UDP when NAT is between peers. IP proto 50 carries raw ESP when no NAT. Open all three to be safe.
Q5 of 10 · Analyze

Priya sees NO_PROPOSAL_CHOSEN in ike.elg immediately after Phase 1 message 1. What's the most likely cause and the diagnostic?

Correct: b. NO_PROPOSAL_CHOSEN at Phase 1 message 1 = no common encryption/hash/DH between proposals. PSK errors fire at message 5 (AUTHENTICATION_FAILED). IKEView lets you visually diff the SA proposals.
Q6 of 10 · Analyze

SmartView Monitor shows the tunnel to a Cisco ASA peer as DOWN. Users report traffic flowing perfectly. What's happening and what's the fix?

Correct: c. Canonical 3rd-party peer issue. tunnel_test is proprietary; DPD is the standard. Swap monitoring mode and SmartView will correctly read the tunnel state.
Q7 of 10 · Analyze

Tunnel UP, small TCP/UDP traffic works, but file downloads >100KB stall midway. What's the diagnosis?

Correct: c. Classic "tunnel UP but downloads stall" = MTU. Encrypted ESP fattens packets, PMTUD ICMP often blocked, sender keeps sending oversized packets. MSS clamping forces TCP to negotiate a smaller MSS upfront.
Q8 of 10 · Analyze

Karthik's gateway logs say "according to the policy the packet should not have been decrypted". Both peers' encryption domains are visually correct. What's the next thing to check?

Correct: a. "Encryption domains look correct" + "should not have been decrypted" almost always = local Hide-NAT happening on egress to the tunnel. The inner source IP no longer matches the peer's domain. The fix is one checkbox.
Q9 of 10 · Evaluate

A team is building a mixed-vendor VPN fleet (Check Point + Cisco + Fortinet + AWS). Which permanent-tunnel monitoring scheme should they standardize on and why?

Correct: d. DPD is the only standard all four vendors support. tunnel_test is CP-only. ICMP ping over tunnel works but doesn't differentiate data-plane vs tunnel issues. IKE keepalive isn't a thing in IKEv1 (was deprecated).
Q10 of 10 · Evaluate

Reflecting on CVE-2024-24919: what's the right hygiene policy for a fleet of Check Point gateways running IPsec-VPN blade?

Correct: b. Senior-engineer answer. CISA KEV is the canonical "actively exploited" list. 24-72h SLA is what regulated industries require. Disable un-needed blades to shrink attack surface. Post-incident rotation prevents lateral movement.
Lesson complete — score saved to your profile.
Score below 70%. Re-read the section you got wrong — every VPN interview tests these.

Next up — Check Point Identity Awareness

You can read ike.elg now. Next: AD Query, PDP/PEP topology, Captive Portal, Access Roles — the move from IP-based rules to user-based rules.

Next lesson → Practice CCSE MCQs

Sources cited inline

  1. R81.20 Site-to-Site VPN Admin Guide
  2. R81 — IPsec & IKE
  3. R81 — Link Selection
  4. R81 — Tunnel Management
  5. R81 — Route Based VPN (VTI)
  6. sk31619 — VPN tunnel is down
  7. sk106837 — Overlapping Encryption Domains
  8. sk32664 — CP Initiating IKE over NAT-T
  9. sk182336 — CVE-2024-24919 Hotfix
  10. CheckMates — Practical Advanced VPN Troubleshooting
  11. CCSE R81.20 (156-315.81.20) Syllabus