The ticket
ALB health checks fail. sg-web allows 443 from the ALB SG. Instance ENI is in the app subnet. Custom NACL acl-app has inbound 443 allow, outbound only 443 allow. Return traffic to the ALB high ports is dropped. Junior “fixed” it by 0.0.0.0/0 on the SG. The NACL was the hole.
Official VPC compare: a security group is stateful, attached to an ENI/instance, allow rules only, all matching allows are evaluated. A network ACL is stateless, attached to a subnet, allow and deny, rules processed in number order (lowest first) until a match, then implicit deny. Return traffic is automatic on an SG. On a NACL you must allow the return path yourself (typically high/ephemeral ports outbound for a server). Default NACL allows all; a custom NACL denies all until you add rules.
Account 111122223333 · OUs Security / Workloads · hub VPC 10.50.0.0/16 · app VPC 10.51.0.0/16 · SG sg-web · NACL acl-app · bucket tc-lab-app-logs · role AppEC2Role · TGW tgw-lab · VGW public 203.0.113.60 · on-prem FortiGate WAN 203.0.113.10. Dummy account ID. Not yours.
Mental model
| Security group | Network ACL | |
|---|---|---|
| State | Stateful — return allowed automatically | Stateless — return needs a rule |
| Where | ENI / instance (sg-web, sg-app) | Subnet (acl-app) |
| Rules | Allow only | Allow and deny, numbered |
| Eval | All allows considered | Lowest number that matches wins |
| This lab | 443 from ALB SG into sg-web; app ports only from sg-web into sg-app | acl-app on 10.51.0.0/16 subnets — do not forget ephemeral out |
NACL first at the subnet, then SG on the ENI. Fail either layer and the ticket stays open.
sg-web vs acl-app
How to choose
| Need | Use | Skip when |
|---|---|---|
| Allow this instance to take 443 from the ALB | sg-web allow from the ALB SG id | 0.0.0.0/0 “just for now” |
| Hard deny a prefix for a whole subnet | Numbered DENY on acl-app above the allow | Trying to write a deny on an SG |
| Default-allow lab subnet | Leave the default NACL (allow all) and do the work in SGs | Custom NACL with no ephemeral return |
Runbook
Side A — ENI
Describe
sg-webandsg-app. Inbound should reference SG ids, not the internet, unless that is the design. SGs are allow-only — if it is too open, delete the allow.Side B — subnet
Describe
acl-app. Read rule numbers low to high. Confirm inbound and outbound both exist for the flow. Add ephemeral outbound for a server.Side C — prove
Reach 443. If SG looks right and it still dies, it is NACL, route, or the target. Do not open 0.0.0.0/0 as a diagnostic.
aws ec2 describe-security-groups --group-names sg-web sg-app \
--query 'SecurityGroups[].{GroupName:GroupName,IpPermissions:IpPermissions,IpPermissionsEgress:IpPermissionsEgress}'
aws ec2 describe-network-acls --filters Name=tag:Name,Values=acl-app \
--query 'NetworkAcls[].Entries'
# Custom NACL: inbound 443 allow is not enough.
# Outbound must allow the return ephemeral range you designed.
# Default NACL already allows all — custom starts deny-all.Four failures
1 · Custom NACL, no return
Classic. Stateless. Ephemeral outbound (or the exact return ports) must exist.
2 · Deny on a security group
There is no deny. Remove the allow, or use a NACL deny for a prefix.
3 · 0.0.0.0/0 on sg-web to “test”
It ships. Source the ALB / sg-web instead.
4 · Rule 200 allow, rule 100 deny forgotten
NACL first-match by number. The lower number already decided.
How to prove it
1) Both SGs show the intended allows only. 2) acl-app inbound and outbound match the flow. 3) You did not add 0.0.0.0/0. 4) You can say stateful vs stateless without hedging.
Traps
| Symptom | Look at | Wrong fix |
|---|---|---|
| SYN in, no SYN-ACK | NACL outbound / route / target SG | Wide SG allow |
| Works then fails after NACL “hardening” | Missing ephemeral | Disable the SG |
| “SG deny this IP” | Cannot — use NACL deny or do not allow | Invent a deny checkbox |
Knowledge check
Judgment items. One best answer. Reasons send you back to the matching section.
AWS security class series: Shared + IAM · Org / OU · SG vs NACL · Role not key · KMS + Secrets · VPC endpoint · Trail + GuardDuty · TGW hub · S3 public · Interview
Sources
- VPC User Guide — Security groups.
- VPC User Guide — Network ACLs.
- VPC User Guide — Compare security groups and network ACLs.
Related: AWS session factory · Migration series.