The ticket
GuardDuty later (lesson 7) will scream about an access key from a strange ASN. Today you still see aws_access_key_id in the AMI’s user-data. AppEC2Role already exists. Nobody attached the instance profile.
An IAM role is assumed: STS issues temporary credentials. Humans: IAM Identity Center (SSO) → permission set → role in account 111122223333. EC2: instance profile passes AppEC2Role; the instance retrieves creds from the metadata service (prefer IMDSv2). EKS: IRSA — service account annotated with a role, trust via the cluster OIDC provider. IAM user access keys are long-lived, copied, committed, and forgotten. Do not create them for people.
Account 111122223333 · OUs Security / Workloads · hub VPC 10.50.0.0/16 · app VPC 10.51.0.0/16 · SG sg-web · NACL acl-app · bucket tc-lab-app-logs · role AppEC2Role · TGW tgw-lab · VGW public 203.0.113.60 · on-prem FortiGate WAN 203.0.113.10. Dummy account ID. Not yours.
Mental model
| Principal | How it assumes | What you should see |
|---|---|---|
| Priya (human) | Identity Center permission set | assumed-role/AWSReservedSSO_…/priya |
| EC2 in 10.51.0.0/16 | Instance profile → AppEC2Role | Instance profile attached; IMDSv2; no AKIA in user-data |
| EKS pod | IRSA (service account + OIDC trust) | Role trust sts:AssumeRoleWithWebIdentity |
| IAM user + access key | Long-lived AKIA/ASIA static pair | Treat as an incident if it belongs to a person |
Same account. Two ways in. Only one belongs in production.
AppEC2Role + permission sets
How to choose
| Situation | Do | Do not |
|---|---|---|
| Person needs console + CLI | Identity Center permission set | Create IAM user + AKIA |
| EC2 must talk to S3 / KMS | Instance profile on AppEC2Role | Keys in user-data or AMI |
| Pod must talk to AWS APIs | IRSA | Node role reused by every pod |
| Third-party CI | OIDC assume-role (same idea as IRSA) | A forever key in the pipeline secret store “for now” |
Runbook
Side A — humans
Identity Center assignments for Priya. Delete IAM user access keys that belong to people. Credential report: look for
access_key_1_active.Side B — compute
Attach the instance profile to the app instance. Confirm trust policy is
ec2.amazonaws.com. For EKS, annotate the service account and check the OIDC provider.Side C — prove
From the instance:
aws sts get-caller-identityshowsAppEC2Role.curlIMDSv2, not v1. Grep user-data forAKIA— must be empty.
aws iam get-role --role-name AppEC2Role --query 'Role.AssumeRolePolicyDocument' aws iam list-instance-profiles-for-role --role-name AppEC2Role aws iam generate-credential-report aws iam get-credential-report --query 'Content' --output text | base64 --decode | head # on the instance (dummy) TOKEN=$(curl -s -X PUT http://169.254.169.254/latest/api/token -H "X-aws-ec2-metadata-token-ttl-seconds: 21600") curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/iam/security-credentials/AppEC2Role aws sts get-caller-identity # Arn: arn:aws:sts::111122223333:assumed-role/AppEC2Role/i-labapp01
Four failures
1 · Keys in user-data
Anyone with ec2:DescribeInstanceAttribute can read them. Next lesson is secrets. This lesson is: do not put them there.
2 · Role exists, profile not attached
The instance still has no creds. Attach the instance profile, do not mint a user key as a shortcut.
3 · Node instance role shared by every pod
That is not IRSA. One compromised pod is every AWS permission on the node.
4 · “Break-glass IAM user” with AKIA in LastPass, used daily
That is just a human key. Identity Center + a rare root/break-glass runbook.
How to prove it
1) Humans assume Identity Center roles. 2) The instance caller is AppEC2Role. 3) Credential report shows no people-keys. 4) User-data has no secrets.
Traps
| You hear | You say |
|---|---|
| We need an access key for EC2 | You need an instance profile |
| IRSA is just the node role | IRSA is a per-service-account role via OIDC |
| ASIA keys in metadata are long-lived | They are temporary session creds from STS |
Knowledge check
Judgment items. One best answer. Reasons send you back to the matching section.
AWS security class series: Shared + IAM · Org / OU · SG vs NACL · Role not key · KMS + Secrets · VPC endpoint · Trail + GuardDuty · TGW hub · S3 public · Interview
Sources
Related: AWS session factory · Migration series.