audit-labs/audit-tools

A collection of scripts, queries, and other goodies you can use in an audit. audit automation compliance evidence scripts

Commit a70cee9b1e

a70cee9b1e475ff61f5ca97183969f16bf931fa8

parent: 5ea48c117a

Unsigned

cmc <hello@cleberg.net> · 2026-08-05 20:44 UTC

fix: correct password-policy evaluator bugs and harden collector

evaluate_policy.py:
- Fix MaxPasswordAge scoring: it is a maximum (lower is stricter), so
  score actual <= expected. Previously scored as a minimum, which passed
  overly long expiry windows that should fail.
- Stop crashing on absent numeric fields. AWS omits MaxPasswordAge when
  expiry is off (and PasswordReusePrevention when reuse prevention is
  off); comparing the "(missing)" sentinel to an int raised TypeError.
  A required-but-absent item is now reported as FAIL.
- Fix utc_now(): it referenced datetime.datetime (the class has no such
  attribute) and only worked via a module re-import buried in __main__.
  Use datetime.now(timezone.utc) and drop the shadowing import.
- Compare booleans with == and reject cross-type matches (a bool no
  longer satisfies a numeric rule and vice versa).

gather_policy.sh:
- Build the metadata block with `jq --arg/--argjson` so hostnames or
  working directories containing quotes or backslashes cannot produce
  malformed JSON.

Layout: unified · split

applications/aws/aws_password_policy/evaluate_policy.py +29 −13
@@ -8,8 +8,12 @@ a CSV audit report.
8 8
9Features 9Features
10* Interactive prompts – press <Enter> to mark a rule as N/A. 10* Interactive prompts – press <Enter> to mark a rule as N/A.
11* Numeric items are treated as **minimums** (actual >= expected → PASS). 11* Most numeric items are treated as **minimums** (actual >= expected → PASS).
12* Maximum password age is treated as a **maximum** (actual <= expected → PASS),
13 because a lower ceiling is the stricter/more‑secure setting.
12* Boolean items are treated as **exact matches** (actual == expected → PASS). 14* Boolean items are treated as **exact matches** (actual == expected → PASS).
15* A required item that is absent from the policy is reported as FAIL rather
16 than crashing (AWS omits e.g. MaxPasswordAge when password expiry is off).
13* The CSV begins with a small metadata block (same data that the Bash script 17* The CSV begins with a small metadata block (same data that the Bash script
14 captured) so the audit trail is self‑contained. 18 captured) so the audit trail is self‑contained.
15* Usage: 19* Usage:
@@ -27,16 +31,19 @@ from typing import Any
27# Mapping of the 10 password‑policy fields we care about 31# Mapping of the 10 password‑policy fields we care about
28# (rule_no, json_key, friendly_name, datatype) 32# (rule_no, json_key, friendly_name, datatype)
29# ---------------------------------------------------------------------- 33# ----------------------------------------------------------------------
34# Numeric datatypes carry a direction:
35# "int_min" – actual must be >= expected (higher is stricter)
36# "int_max" – actual must be <= expected (lower is stricter)
30POLICY_FIELDS = [ 37POLICY_FIELDS = [
31 (1, "MinimumPasswordLength", "Minimum password length", "int"), 38 (1, "MinimumPasswordLength", "Minimum password length", "int_min"),
32 (2, "RequireSymbols", "Require symbols (!@#$…)", "bool"), 39 (2, "RequireSymbols", "Require symbols (!@#$…)", "bool"),
33 (3, "RequireNumbers", "Require numbers (0‑9)", "bool"), 40 (3, "RequireNumbers", "Require numbers (0‑9)", "bool"),
34 (4, "RequireUppercaseCharacters", "Require uppercase letters (A‑Z)", "bool"), 41 (4, "RequireUppercaseCharacters", "Require uppercase letters (A‑Z)", "bool"),
35 (5, "RequireLowercaseCharacters", "Require lowercase letters (a‑z)", "bool"), 42 (5, "RequireLowercaseCharacters", "Require lowercase letters (a‑z)", "bool"),
36 (6, "AllowUsersToChangePassword", "Allow users to change password", "bool"), 43 (6, "AllowUsersToChangePassword", "Allow users to change password", "bool"),
37 (7, "ExpirePasswords", "Expire passwords (enable aging)", "bool"), 44 (7, "ExpirePasswords", "Expire passwords (enable aging)", "bool"),
38 (8, "MaxPasswordAge", "Maximum password age (days)", "int"), 45 (8, "MaxPasswordAge", "Maximum password age (days)", "int_max"),
39 (9, "PasswordReusePrevention", "Prevent password reuse (last N)", "int"), 46 (9, "PasswordReusePrevention", "Prevent password reuse (last N)", "int_min"),
40 (10, "HardExpiry", "Hard expiry (no grace period)", "bool"), 47 (10, "HardExpiry", "Hard expiry (no grace period)", "bool"),
41] 48]
42 49
@@ -45,8 +52,8 @@ POLICY_FIELDS = [
45# Helper functions 52# Helper functions
46# ---------------------------------------------------------------------- 53# ----------------------------------------------------------------------
47def utc_now() -> datetime: 54def utc_now() -> datetime:
48 """Return a timezone‑aware UTC datetime (compatible with all Python 3.x).""" 55 """Return a timezone‑aware UTC datetime."""
49 return datetime.datetime.now(timezone.utc) 56 return datetime.now(timezone.utc)
50 57
51 58
52def prompt_expected(field_type: str, description: str) -> Any | None: 59def prompt_expected(field_type: str, description: str) -> Any | None:
@@ -62,7 +69,7 @@ def prompt_expected(field_type: str, description: str) -> Any | None:
62 ).strip() 69 ).strip()
63 if raw == "": 70 if raw == "":
64 return None # N/A 71 return None # N/A
65 if field_type == "int": 72 if field_type.startswith("int"):
66 if raw.isdigit(): 73 if raw.isdigit():
67 return int(raw) 74 return int(raw)
68 print("Please enter a whole number (or leave blank).") 75 print("Please enter a whole number (or leave blank).")
@@ -79,13 +86,24 @@ def prompt_expected(field_type: str, description: str) -> Any | None:
79 86
80 87
81def evaluate(expect: Any | None, actual: Any, field_type: str) -> str: 88def evaluate(expect: Any | None, actual: Any, field_type: str) -> str:
82 """Return PASS / FAIL / N/A.""" 89 """Return PASS / FAIL / N/A.
90
91 A required item (expectation set) that is absent or of the wrong type in
92 the policy is a FAIL, never a crash.
93 """
83 if expect is None: 94 if expect is None:
84 return "N/A" 95 return "N/A"
85 if field_type == "int":
86 return "PASS" if actual >= expect else "FAIL"
87 if field_type == "bool": 96 if field_type == "bool":
88 return "PASS" if actual is expect else "FAIL" 97 # bool is a subclass of int, so guard against ints sneaking through.
98 return "PASS" if isinstance(actual, bool) and actual == expect else "FAIL"
99 if field_type.startswith("int"):
100 # Reject non‑numbers (e.g. a missing field rendered as a string) and
101 # booleans (a subclass of int that must not satisfy a numeric rule).
102 if isinstance(actual, bool) or not isinstance(actual, (int, float)):
103 return "FAIL"
104 if field_type == "int_max":
105 return "PASS" if actual <= expect else "FAIL"
106 return "PASS" if actual >= expect else "FAIL"
89 return "FAIL" 107 return "FAIL"
90 108
91 109
@@ -173,6 +191,4 @@ def main() -> None:
173 191
174 192
175if __name__ == "__main__": 193if __name__ == "__main__":
176 import datetime # imported here to keep the top of file tidy
177
178 main() 194 main()
applications/aws/aws_password_policy/gather_policy.sh +20 −14
@@ -60,20 +60,26 @@ fi
60# * current working directory (useful for traceability) 60# * current working directory (useful for traceability)
61# * AWS profile & region (if set) 61# * AWS profile & region (if set)
62# * AWS caller identity (ARN, account id, user id) – proves *who* ran the command 62# * AWS caller identity (ARN, account id, user id) – proves *who* ran the command
63METADATA=$(cat <<EOF 63# Build with `jq --arg` so values containing quotes/backslashes (e.g. an odd
64{ 64# hostname or working directory) can never produce malformed JSON.
65 "metadata": { 65CALLER_IDENTITY=$(aws sts get-caller-identity 2>/dev/null || echo "null")
66 "report_timestamp_utc": "$(date -u +"%Y-%m-%dT%H:%M:%SZ")", 66METADATA=$(jq -n \
67 "os_user": "$(id -un)", 67 --arg ts "$(date -u +"%Y-%m-%dT%H:%M:%SZ")" \
68 "hostname": "$(hostname)", 68 --arg user "$(id -un)" \
69 "working_directory": "$(pwd)", 69 --arg host "$(hostname)" \
70 "aws_profile": "${AWS_PROFILE:-default}", 70 --arg cwd "$(pwd)" \
71 "aws_region": "${AWS_DEFAULT_REGION:-unknown}", 71 --arg profile "${AWS_PROFILE:-default}" \
72 "aws_caller_identity": $(aws sts get-caller-identity 2>/dev/null || echo "null") 72 --arg region "${AWS_DEFAULT_REGION:-unknown}" \
73 } 73 --argjson caller "$CALLER_IDENTITY" \
74} 74 '{metadata: {
75EOF 75 report_timestamp_utc: $ts,
76) 76 os_user: $user,
77 hostname: $host,
78 working_directory: $cwd,
79 aws_profile: $profile,
80 aws_region: $region,
81 aws_caller_identity: $caller
82 }}')
77 83
78# ---------- 3. Merge policy + metadata ---------- 84# ---------- 3. Merge policy + metadata ----------
79# The final JSON will have two top‑level keys: "metadata" and "PasswordPolicy" 85# The final JSON will have two top‑level keys: "metadata" and "PasswordPolicy"