audit-labs/audit-tools

A collection of scripts, queries, and other goodies you can use in an audit. audit automation compliance evidence scripts

Commit f12763587f

f12763587f1c5c680a7a1c80054470cab9ad8360

parent: e6f5e670bf

Unsigned

cmc <hello@cleberg.net> · 2026-07-29 17:10 UTC 

feat: add AWS account, network, and logging security checks

Deepen the AWS audit with four new collectors:

- account_security: IAM account summary (root MFA, root access keys, resource
  counts).
- security_groups: security-group ingress rules open to 0.0.0.0/0 or ::/0,
  scanned across all enabled regions (one row per open rule).
- cloudtrail: trail logging status, multi-region, log-file validation.
- config_recorders: AWS Config recording status per region, flagging gaps.

Add an enabled_regions() helper for the region-scoped checks (each region is
skipped independently on error). Wire the checks into aws_runner and audit.py,
add mocked-boto3 unit tests, and update the README scope/permission notes.

Layout: unified · split

README.org +1 −1
@@ -17,7 +17,7 @@ connection details, choose which checks to run, and watch live progress.
17 17
18| Directory | Description | 18| Directory | Description |
19|----------------------+------------------------------------------------------------------------------| 19|----------------------+------------------------------------------------------------------------------|
20| =applications/aws/= | AWS IAM users, password policy, and S3 bucket analysis | 20| =applications/aws/= | AWS IAM users, account/root security, password policy, S3 public access, open security groups, CloudTrail, Config, SSO |
21| =applications/github/= | GitHub admin enumeration, org security settings, webhooks, deploy keys, secret-scanning/Dependabot alerts, audit log, branch protections, commits | 21| =applications/github/= | GitHub admin enumeration, org security settings, webhooks, deploy keys, secret-scanning/Dependabot alerts, audit log, branch protections, commits |
22| =applications/gitlab/= | GitLab group/project members, branch protections, approvals, pipelines, audit events | 22| =applications/gitlab/= | GitLab group/project members, branch protections, approvals, pipelines, audit events |
23| =databases/mongo/= | MongoDB admin enumeration | 23| =databases/mongo/= | MongoDB admin enumeration |
applications/aws/README.md +11 −2
@@ -1,9 +1,11 @@
1> **NOTE**: Authentication uses the standard AWS credential chain (environment 1> **NOTE**: Authentication uses the standard AWS credential chain (environment
2> variables, shared config/credentials, SSO profiles, instance roles). This tool 2> variables, shared config/credentials, SSO profiles, instance roles). This tool
3> never handles access keys directly. Read-only permissions are enough — IAM 3> never handles access keys directly. Read-only permissions are enough — IAM
4> `Get*`/`List*`, S3 `s3:GetBucket*` + `s3:ListAllMyBuckets`, and for the SSO 4> `Get*`/`List*`, S3 `s3:GetBucket*` + `s3:ListAllMyBuckets`, EC2
5> `ec2:DescribeRegions`/`DescribeSecurityGroups`, `cloudtrail:DescribeTrails` +
6> `GetTrailStatus`, `config:DescribeConfigurationRecorders*`, and for the SSO
5> check `sso:List*`/`sso:Describe*`, `identitystore:Describe*`, and 7> check `sso:List*`/`sso:Describe*`, `identitystore:Describe*`, and
6> `organizations:ListAccounts`. 8> `organizations:ListAccounts`. The SecurityAudit managed policy covers these.
7 9
8--- 10---
9 11
@@ -42,10 +44,17 @@ Creates a directory: `<out>/aws_audit_<profile>_<YYYY-MM-DD>/`
42|---|---| 44|---|---|
43| `iam_users.csv` | IAM users with MFA status, access-key count/age, console password, last use | 45| `iam_users.csv` | IAM users with MFA status, access-key count/age, console password, last use |
44| `password_policy.csv` | Account IAM password policy (length, complexity, rotation, reuse) | 46| `password_policy.csv` | Account IAM password policy (length, complexity, rotation, reuse) |
47| `account_security.csv` | Account summary — root MFA, root access keys, and resource counts |
45| `s3_public_access.csv` | Per-bucket Public Access Block, policy public status, and ACL public exposure | 48| `s3_public_access.csv` | Per-bucket Public Access Block, policy public status, and ACL public exposure |
49| `open_security_groups.csv` | Security-group ingress rules open to `0.0.0.0/0` or `::/0`, across all regions |
50| `cloudtrail.csv` | CloudTrail trails — logging status, multi-region, log-file validation |
51| `config_recorders.csv` | AWS Config recording status per region (gaps are flagged) |
46| `sso_assignments.csv` | IAM Identity Center permission-set assignments per account (Identity Center + Organizations) | 52| `sso_assignments.csv` | IAM Identity Center permission-set assignments per account (Identity Center + Organizations) |
47| `summary.txt` | Row counts per section | 53| `summary.txt` | Row counts per section |
48 54
55`open_security_groups.csv` and `config_recorders.csv` scan every enabled region,
56so they take longer on accounts with many regions.
57
49Checks that aren't available (no password policy, no Identity Center instance, 58Checks that aren't available (no password policy, no Identity Center instance,
50missing permissions) are skipped with a warning; the rest still run. 59missing permissions) are skipped with a warning; the rest still run.
51 60
applications/aws/audit.py +9 −1
@@ -31,7 +31,7 @@ import sys
31from datetime import date 31from datetime import date
32 32
33import config 33import config
34from collectors import iam, s3, sso 34from collectors import iam, monitoring, s3, security_groups, sso
35from reporters import csv_reporter 35from reporters import csv_reporter
36 36
37 37
@@ -86,7 +86,15 @@ def run():
86 86
87 collect("IAM users", iam.iam_users, "iam_users.csv") 87 collect("IAM users", iam.iam_users, "iam_users.csv")
88 collect("Password policy", iam.password_policy, "password_policy.csv") 88 collect("Password policy", iam.password_policy, "password_policy.csv")
89 collect("Account security", iam.account_security, "account_security.csv")
89 collect("S3 public access", s3.s3_public_access, "s3_public_access.csv") 90 collect("S3 public access", s3.s3_public_access, "s3_public_access.csv")
91 collect(
92 "Open security groups",
93 security_groups.security_groups,
94 "open_security_groups.csv",
95 )
96 collect("CloudTrail", monitoring.cloudtrail, "cloudtrail.csv")
97 collect("AWS Config recorders", monitoring.config_recorders, "config_recorders.csv")
90 collect("SSO assignments", sso.sso_assignments, "sso_assignments.csv") 98 collect("SSO assignments", sso.sso_assignments, "sso_assignments.csv")
91 99
92 print() 100 print()
applications/aws/collectors/api.py +6
@@ -35,3 +35,9 @@ def account_id(cfg):
35 return cfg["session"].client("sts").get_caller_identity()["Account"] 35 return cfg["session"].client("sts").get_caller_identity()["Account"]
36 except Exception: 36 except Exception:
37 return "" 37 return ""
38
39
40def enabled_regions(cfg):
41 """Return the region names enabled for the account (for region-scoped checks)."""
42 ec2 = cfg["session"].client("ec2", region_name=cfg.get("region") or "us-east-1")
43 return [r["RegionName"] for r in ec2.describe_regions().get("Regions", [])]
applications/aws/collectors/iam.py +25 −1
@@ -1,5 +1,6 @@
1""" 1"""
2Collect IAM user hygiene and the account password policy. 2Collect IAM user hygiene, the account password policy, and account-level
3security summary (root MFA, root access keys).
3""" 4"""
4 5
5import sys 6import sys
@@ -8,6 +9,29 @@ from datetime import datetime, timezone
8from botocore.exceptions import ClientError 9from botocore.exceptions import ClientError
9 10
10 11
12def account_security(cfg):
13 """
14 One row of account-level security signals from the IAM account summary:
15 whether the root user has MFA and access keys, plus resource counts.
16 """
17 iam = cfg["session"].client("iam")
18 s = iam.get_account_summary()["SummaryMap"]
19 return [
20 {
21 "root_mfa_enabled": bool(s.get("AccountMFAEnabled", 0)),
22 "root_access_keys_present": bool(s.get("AccountAccessKeysPresent", 0)),
23 "root_signing_certs_present": bool(
24 s.get("AccountSigningCertificatesPresent", 0)
25 ),
26 "mfa_devices": s.get("MFADevices", 0),
27 "users": s.get("Users", 0),
28 "groups": s.get("Groups", 0),
29 "roles": s.get("Roles", 0),
30 "policies": s.get("Policies", 0),
31 }
32 ]
33
34
11def iam_users(cfg): 35def iam_users(cfg):
12 """ 36 """
13 One row per IAM user: MFA status, access-key count and oldest key age, 37 One row per IAM user: MFA status, access-key count and oldest key age,
applications/aws/collectors/monitoring.py added +81
@@ -0,0 +1,81 @@
1"""
2Collect audit-logging posture: CloudTrail trails and AWS Config recorders.
3"""
4
5import sys
6
7from botocore.exceptions import ClientError
8
9from .api import enabled_regions
10
11
12def cloudtrail(cfg):
13 """
14 One row per CloudTrail trail: whether it is logging, multi-region, and has
15 log-file validation. An empty result means no trails are configured.
16 """
17 region = cfg.get("region") or "us-east-1"
18 ct = cfg["session"].client("cloudtrail", region_name=region)
19 rows = []
20 for trail in ct.describe_trails(includeShadowTrails=False).get("trailList", []):
21 try:
22 status = ct.get_trail_status(Name=trail["TrailARN"])
23 except ClientError:
24 status = {}
25 rows.append(
26 {
27 "name": trail.get("Name", ""),
28 "home_region": trail.get("HomeRegion", ""),
29 "multi_region": trail.get("IsMultiRegionTrail"),
30 "log_file_validation": trail.get("LogFileValidationEnabled"),
31 "is_logging": status.get("IsLogging"),
32 "s3_bucket": trail.get("S3BucketName", ""),
33 }
34 )
35 return rows
36
37
38def config_recorders(cfg):
39 """
40 One row per region: whether AWS Config is recording. Regions with no
41 recorder are reported so gaps are visible.
42 """
43 session = cfg["session"]
44 rows = []
45 for region in enabled_regions(cfg):
46 try:
47 cc = session.client("config", region_name=region)
48 recorders = cc.describe_configuration_recorders().get(
49 "ConfigurationRecorders", []
50 )
51 statuses = {
52 s["name"]: s
53 for s in cc.describe_configuration_recorder_status().get(
54 "ConfigurationRecordersStatus", []
55 )
56 }
57 except ClientError as e:
58 print(f" Skipping {region}: config returned {e}", file=sys.stderr)
59 continue
60
61 if not recorders:
62 rows.append(
63 {
64 "region": region,
65 "recorder": "(none)",
66 "recording": False,
67 "last_status": "",
68 }
69 )
70 continue
71 for r in recorders:
72 st = statuses.get(r["name"], {})
73 rows.append(
74 {
75 "region": region,
76 "recorder": r["name"],
77 "recording": st.get("recording"),
78 "last_status": st.get("lastStatus", ""),
79 }
80 )
81 return rows
applications/aws/collectors/security_groups.py added +63
@@ -0,0 +1,63 @@
1"""
2Collect security-group ingress rules open to the internet, across all regions.
3
4Only rules allowing 0.0.0.0/0 or ::/0 are reported — one row per open rule.
5"""
6
7import sys
8
9from botocore.exceptions import ClientError
10
11from .api import enabled_regions
12
13OPEN_V4 = "0.0.0.0/0"
14OPEN_V6 = "::/0"
15
16
17def security_groups(cfg):
18 session = cfg["session"]
19 rows = []
20 for region in enabled_regions(cfg):
21 try:
22 ec2 = session.client("ec2", region_name=region)
23 groups = _all_groups(ec2)
24 except ClientError as e:
25 print(f" Skipping {region}: ec2 returned {e}", file=sys.stderr)
26 continue
27 for sg in groups:
28 rows.extend(_open_rules(region, sg))
29 return rows
30
31
32def _all_groups(ec2):
33 groups = []
34 for page in ec2.get_paginator("describe_security_groups").paginate():
35 groups.extend(page.get("SecurityGroups", []))
36 return groups
37
38
39def _open_rules(region, sg):
40 rows = []
41 for perm in sg.get("IpPermissions", []):
42 open_to = [
43 r["CidrIp"] for r in perm.get("IpRanges", []) if r.get("CidrIp") == OPEN_V4
44 ]
45 open_to += [
46 r["CidrIpv6"]
47 for r in perm.get("Ipv6Ranges", [])
48 if r.get("CidrIpv6") == OPEN_V6
49 ]
50 if not open_to:
51 continue
52 rows.append(
53 {
54 "region": region,
55 "group_id": sg.get("GroupId", ""),
56 "group_name": sg.get("GroupName", ""),
57 "protocol": perm.get("IpProtocol", ""),
58 "from_port": perm.get("FromPort", "all"),
59 "to_port": perm.get("ToPort", "all"),
60 "open_to": ", ".join(open_to),
61 }
62 )
63 return rows
tui/aws_runner.py +29 −1
@@ -21,7 +21,14 @@ _REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
21if _REPO_ROOT not in sys.path: 21if _REPO_ROOT not in sys.path:
22 sys.path.insert(0, _REPO_ROOT) 22 sys.path.insert(0, _REPO_ROOT)
23 23
24from applications.aws.collectors import api, iam, s3, sso 24from applications.aws.collectors import (
25 api,
26 iam,
27 monitoring,
28 s3,
29 security_groups,
30 sso,
31)
25from applications.aws.reporters import csv_reporter 32from applications.aws.reporters import csv_reporter
26 33
27# --- Check registry --------------------------------------------------------- 34# --- Check registry ---------------------------------------------------------
@@ -34,12 +41,33 @@ CHECKS: list[Check] = [
34 iam.password_policy, 41 iam.password_policy,
35 "password_policy.csv", 42 "password_policy.csv",
36 ), 43 ),
44 Check(
45 "account_security",
46 "Account security (root MFA)",
47 iam.account_security,
48 "account_security.csv",
49 ),
37 Check( 50 Check(
38 "s3_public_access", 51 "s3_public_access",
39 "S3 public access", 52 "S3 public access",
40 s3.s3_public_access, 53 s3.s3_public_access,
41 "s3_public_access.csv", 54 "s3_public_access.csv",
42 ), 55 ),
56 Check(
57 "security_groups",
58 "Open security groups",
59 security_groups.security_groups,
60 "open_security_groups.csv",
61 note="scans all regions",
62 ),
63 Check("cloudtrail", "CloudTrail", monitoring.cloudtrail, "cloudtrail.csv"),
64 Check(
65 "config_recorders",
66 "AWS Config recorders",
67 monitoring.config_recorders,
68 "config_recorders.csv",
69 note="scans all regions",
70 ),
43 Check( 71 Check(
44 "sso_assignments", 72 "sso_assignments",
45 "SSO assignments", 73 "SSO assignments",
tui/tests/test_aws_collectors.py added +166
@@ -0,0 +1,166 @@
1"""Unit tests for the new AWS security collectors, mocking boto3 clients."""
2
3from unittest.mock import MagicMock
4
5from applications.aws.collectors import iam as aws_iam
6from applications.aws.collectors import monitoring
7from applications.aws.collectors import security_groups as sg
8
9
10class FakeSession:
11 """Dispatch .client(service, region_name=...) to preconfigured mocks."""
12
13 def __init__(self, clients):
14 self._clients = clients
15
16 def client(self, service, region_name=None):
17 return self._clients[service]
18
19
20def _cfg(clients):
21 return {"session": FakeSession(clients), "region": "us-east-1"}
22
23
24# --- account_security -------------------------------------------------------
25
26
27def test_account_security():
28 iam = MagicMock()
29 iam.get_account_summary.return_value = {
30 "SummaryMap": {
31 "AccountMFAEnabled": 1,
32 "AccountAccessKeysPresent": 0,
33 "Users": 5,
34 "Roles": 12,
35 }
36 }
37 rows = aws_iam.account_security(_cfg({"iam": iam}))
38 assert rows[0]["root_mfa_enabled"] is True
39 assert rows[0]["root_access_keys_present"] is False
40 assert rows[0]["users"] == 5
41 assert rows[0]["roles"] == 12
42
43
44# --- cloudtrail -------------------------------------------------------------
45
46
47def test_cloudtrail():
48 ct = MagicMock()
49 ct.describe_trails.return_value = {
50 "trailList": [
51 {
52 "Name": "org-trail",
53 "TrailARN": "arn:aws:cloudtrail:...:trail/org-trail",
54 "HomeRegion": "us-east-1",
55 "IsMultiRegionTrail": True,
56 "LogFileValidationEnabled": True,
57 "S3BucketName": "logs",
58 }
59 ]
60 }
61 ct.get_trail_status.return_value = {"IsLogging": True}
62 rows = monitoring.cloudtrail(_cfg({"cloudtrail": ct}))
63 assert rows[0]["is_logging"] is True
64 assert rows[0]["multi_region"] is True
65 assert rows[0]["s3_bucket"] == "logs"
66
67
68# --- config_recorders -------------------------------------------------------
69
70
71def _ec2_one_region():
72 ec2 = MagicMock()
73 ec2.describe_regions.return_value = {"Regions": [{"RegionName": "us-east-1"}]}
74 return ec2
75
76
77def test_config_recorders_recording():
78 config = MagicMock()
79 config.describe_configuration_recorders.return_value = {
80 "ConfigurationRecorders": [{"name": "default"}]
81 }
82 config.describe_configuration_recorder_status.return_value = {
83 "ConfigurationRecordersStatus": [
84 {"name": "default", "recording": True, "lastStatus": "SUCCESS"}
85 ]
86 }
87 rows = monitoring.config_recorders(
88 _cfg({"ec2": _ec2_one_region(), "config": config})
89 )
90 assert rows == [
91 {
92 "region": "us-east-1",
93 "recorder": "default",
94 "recording": True,
95 "last_status": "SUCCESS",
96 }
97 ]
98
99
100def test_config_recorders_reports_gap():
101 config = MagicMock()
102 config.describe_configuration_recorders.return_value = {
103 "ConfigurationRecorders": []
104 }
105 config.describe_configuration_recorder_status.return_value = {
106 "ConfigurationRecordersStatus": []
107 }
108 rows = monitoring.config_recorders(
109 _cfg({"ec2": _ec2_one_region(), "config": config})
110 )
111 assert rows[0]["recorder"] == "(none)"
112 assert rows[0]["recording"] is False
113
114
115# --- security_groups --------------------------------------------------------
116
117
118def _ec2_with_groups(groups):
119 ec2 = _ec2_one_region()
120 paginator = MagicMock()
121 paginator.paginate.return_value = [{"SecurityGroups": groups}]
122 ec2.get_paginator.return_value = paginator
123 return ec2
124
125
126def test_security_groups_flags_open_ingress():
127 groups = [
128 {
129 "GroupId": "sg-1",
130 "GroupName": "web",
131 "IpPermissions": [
132 {
133 "IpProtocol": "tcp",
134 "FromPort": 22,
135 "ToPort": 22,
136 "IpRanges": [{"CidrIp": "0.0.0.0/0"}],
137 "Ipv6Ranges": [],
138 }
139 ],
140 }
141 ]
142 rows = sg.security_groups(_cfg({"ec2": _ec2_with_groups(groups)}))
143 assert len(rows) == 1
144 assert rows[0]["group_id"] == "sg-1"
145 assert rows[0]["from_port"] == 22
146 assert rows[0]["open_to"] == "0.0.0.0/0"
147
148
149def test_security_groups_ignores_scoped_ingress():
150 groups = [
151 {
152 "GroupId": "sg-2",
153 "GroupName": "internal",
154 "IpPermissions": [
155 {
156 "IpProtocol": "tcp",
157 "FromPort": 5432,
158 "ToPort": 5432,
159 "IpRanges": [{"CidrIp": "10.0.0.0/8"}],
160 "Ipv6Ranges": [],
161 }
162 ],
163 }
164 ]
165 rows = sg.security_groups(_cfg({"ec2": _ec2_with_groups(groups)}))
166 assert rows == []