Secrets & Credentials Scan — No Key Left Behind

When to Use This Skill

• Before every release — scan the full codebase for leaked credentials
• When setting up a new repository — install pre-commit hooks from day one
• After a developer reports they "accidentally committed a key"
• During periodic security audits (recommended monthly full-history scan)
• When onboarding a new third-party service that issues API keys
• Before open-sourcing an internal repository (deep history scan mandatory)
• When a CI secret is suspected to have leaked via build logs
• After a developer laptop is lost or stolen (rotate all keys that dev had access to)
• When adding environment variables to a deployment platform
• When reviewing pull requests that touch configuration or environment files
• After a GitHub / GitLab security alert for exposed secrets

---

How It Works

┌──────────────────────────────────────────────────────────────────────┐
│                     SECRETS SCAN FLOW                                │
│                                                                      │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐            │
│  │ PHASE 1  │  │ PHASE 2  │  │ PHASE 3  │  │ PHASE 4  │            │
│  │ Pattern  │─▶│ Git      │─▶│ Config & │─▶│ Pre-     │            │
│  │ Scan     │  │ History  │  │ Env Audit│  │ commit   │            │
│  └──────────┘  └──────────┘  └──────────┘  └──────────┘            │
│   Regex + entropy  Scan every    .gitignore    Install hooks         │
│   across current   commit for    .env files    configure CI          │
│   working tree     leaked keys   secret mgmt   block on detect       │
│                                                                      │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐                          │
│  │ PHASE 5  │  │ PHASE 6  │  │ PHASE 7  │                          │
│  │ Classify │─▶│ Rotate & │─▶│ Report & │                          │
│  │ & Triage │  │ Remediate│  │ Harden   │                          │
│  └──────────┘  └──────────┘  └──────────┘                          │
│   Confidence      Revoke keys    Full report                         │
│   scoring, false  rotate creds   with rotation                       │
│   positive check  update refs    status                              │
│                                                                      │
│  ┌──────────────────────────────────────────────────────────────┐    │
│  │               SEVERITY LEVELS                                │    │
│  │                                                              │    │
│  │  CRITICAL  — Production secret in public code / git history  │    │
│  │  HIGH      — Non-prod secret committed, or prod key in logs  │    │
│  │  MEDIUM    — Secret in private repo, not yet pushed          │    │
│  │  LOW       — Test/fake key that matches a real pattern       │    │
│  │  FALSE POS — High-entropy string that is not a secret        │    │
│  └──────────────────────────────────────────────────────────────┘    │
└──────────────────────────────────────────────────────────────────────┘

---

Phase 1: Secret Pattern Taxonomy & Detection

Scan the current working tree for known secret patterns using regex and entropy analysis.

Secret Type Reference

Entropy Detection

Not all secrets follow known patterns. High-entropy strings (Shannon entropy > 4.5 for hex, > 5.0 for base64) in assignment contexts are likely secrets:

High entropy detection targets:
  - Variable assignments: SECRET = "aK8jL2mN..."
  - JSON values:          "apiKey": "xR4tY7..."
  - YAML values:          api_key: dF9gH3...
  - Config files:         API_KEY=qW5eR1...

Entropy thresholds:
  - Hex strings (0-9a-f):     entropy > 3.0 per char → flag
  - Base64 strings:           entropy > 4.0 per char → flag
  - Alphanumeric (arbitrary): entropy > 4.5 per char → flag
  - Minimum length:           16 characters to trigger

Working Tree Scan Steps

□ Run gitleaks detect --source . --verbose on the working tree
□ Check gitleaks output for findings — note file, line, commit
□ Run entropy analysis on all config files (.yml, .yaml, .json, .toml, .ini, .conf, .cfg)
□ Scan Dockerfiles for ARG/ENV with embedded secrets
□ Scan CI workflow files (.github/workflows/*.yml, .gitlab-ci.yml) for inline secrets
□ Check Terraform files (*.tf) for hardcoded credentials in provider blocks
□ Scan Kubernetes manifests for cleartext secrets (kind: Secret with stringData)
□ Check package.json / Makefile / scripts for embedded tokens in commands
□ Review logging code — are secrets printed to stdout/stderr?
□ Check error messages — do stack traces include connection strings?

---

Phase 2: Git History Deep Scan

Secrets deleted from current code may still exist in git history. Every commit ever pushed is a potential leak.

┌──────────────────────────────────────────────────────────────┐
│             GIT HISTORY SCAN APPROACH                         │
│                                                              │
│  ┌────────────┐                                              │
│  │ Full Clone │  git clone --mirror → scans ALL refs         │
│  │  (Mirror)  │  including deleted branches, tags, stashes   │
│  └──────┬─────┘                                              │
│         ▼                                                    │
│  ┌────────────┐                                              │
│  │ Scan Every │  gitleaks detect --source . --log-opts=--all │
│  │  Commit    │  truffleHog git file://./repo --since-commit │
│  └──────┬─────┘                                              │
│         ▼                                                    │
│  ┌────────────┐                                              │
│  │ Check Diff │  Focus on commits that touch:                │
│  │  Hotspots  │  .env, config, secret, credential*     │
│  └──────┬─────┘  docker-compose, .tfvars, .pem, .key    │
│         ▼                                                    │
│  ┌────────────┐                                              │
│  │ Deleted    │  git log --diff-filter=D -- '*.env'          │
│  │  Files     │  Shows files that were added then deleted    │
│  └──────┬─────┘  The delete does not remove history          │
│         ▼                                                    │
│  ┌────────────┐                                              │
│  │ Squashed   │  Squash commits hide individual commits      │
│  │  PRs       │  but the original branch may still exist     │
│  └────────────┘  Check: git reflog, remote refs, forks       │
│                                                              │
└──────────────────────────────────────────────────────────────┘

Git History Commands

# Full history scan with gitleaks
gitleaks detect --source . --log-opts="--all --full-history" --verbose

# TruffleHog full scan (verified secrets only)
trufflehog git file://. --only-verified --json

# Find commits that ever touched env files
git log --all --full-history --diff-filter=A -- '.env' '.env.' '.env'

# Show contents of a deleted file at a specific commit
git show <commit-hash>:<file-path>

# Find all files that were ever named with secret-related terms
git log --all --name-only --diff-filter=A | grep -iE '\.(env|pem|key|p12|pfx|jks|keystore)$'
git log --all --name-only --diff-filter=A | grep -iE '(secret|credential|password|token)'

# List all branches (including remote-tracking and deleted)
git branch -a --list
git reflog | grep 'checkout: moving' | head -20

# Check for secrets in git stash
git stash list
git stash show -p stash@{0}   # repeat for each stash entry

History Scan Checklist

□ Clone the repo with --mirror to capture all refs
□ Run gitleaks with --log-opts="--all" for full history
□ Run truffleHog with --only-verified for high-confidence findings
□ Search for deleted .env files: git log --diff-filter=D -- '.env'
□ Search for deleted key files: git log --diff-filter=D -- '.pem' '.key' '*.p12'
□ Check squashed merge commits for secrets hidden in the squash
□ Verify no secrets exist in git tags (annotated tag messages)
□ Check git stash for secrets (git stash list, git stash show -p)
□ Scan git notes (git notes list) for attached secrets
□ Review .git/config for embedded credentials in remote URLs

---

Phase 3: Configuration & Environment Audit

Audit .gitignore, environment variable handling, and secret management infrastructure.

.gitignore Audit

Files that MUST be in .gitignore — no exceptions:

.gitignore Verification Checklist

□ .gitignore exists at repo root
□ .env and .env.* are listed (verify with: git check-ignore .env)
□ .pem, .key, .p12, .pfx are listed
□ credentials.json, service-account*.json are listed
□ *.tfvars, terraform.tfstate are listed
□ IDE configs (.idea/, .vscode/) are listed
□ .env.example exists with placeholder values (no real secrets)
□ docker-compose.override.yml is listed if used
□ No .env file was ever committed (git log --all -- '.env*')
□ Run: git ls-files | grep -iE '\.(env|pem|key|p12|pfx|jks)$' — must return empty

Secret Storage Audit

Verify secrets are stored in the correct location for each environment:

┌────────────────────────────────────────────────────────────┐
│  Environment     │ Acceptable Storage     │ NOT Acceptable │
├──────────────────┼────────────────────────┼────────────────┤
│  Local Dev       │ .env.local (gitignored)│ Hardcoded in   │
│                  │ 1Password / Bitwarden  │ source code    │
│                  │                        │                │
│  CI/CD Pipeline  │ GitHub Actions Secrets │ Inline in YAML │
│                  │ GitLab CI Variables    │ Echoed to logs │
│                  │ (masked + protected)   │                │
│                  │                        │                │
│  Staging         │ AWS Secrets Manager    │ .env on disk   │
│                  │ Vercel Env Variables   │ Docker ENV     │
│                  │ Parameter Store (SSM)  │ Config files   │
│                  │                        │                │
│  Production      │ AWS Secrets Manager    │ .env on disk   │
│                  │ HashiCorp Vault        │ Docker ENV     │
│                  │ GCP Secret Manager     │ Config files   │
│                  │ Azure Key Vault        │ Hardcoded      │
└────────────────────────────────────────────────────────────┘

---

Phase 4: Pre-commit Hook & CI Setup

Block secrets before they enter git history. This is the single highest-ROI security control.

gitleaks Pre-commit Installation

# Install gitleaks
brew install gitleaks          # macOS
# or download from https://github.com/gitleaks/gitleaks/releases

# Option A: husky (Node.js projects)
npx husky add .husky/pre-commit "gitleaks protect --staged --verbose"

# Option B: pre-commit framework (Python ecosystem)
# .pre-commit-config.yaml:
repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.18.0
    hooks:
      - id: gitleaks

# Option C: git hooks directly
cat > .git/hooks/pre-commit << 'EOF'
#!/bin/sh
gitleaks protect --staged --verbose --redact
if [ $? -ne 0 ]; then
  echo "SECRET DETECTED — commit blocked. Remove the secret and try again."
  exit 1
fi
EOF
chmod +x .git/hooks/pre-commit

gitleaks Configuration (.gitleaks.toml)

[extend]
useDefault = true

[allowlist]
description = "Project-specific allowlist"
paths = [
  '''(^|/)test[s]?/''',
  '''(^|/)__test__/''',
  '''\.test\.(ts|js|tsx|jsx)$''',
  '''\.spec\.(ts|js|tsx|jsx)$''',
  '''\.md$''',
]
regexTarget = "line"
regexes = [
  '''sk_test_FAKE_KEY_FOR_TESTING''',
  '''REPLACE_ME|CHANGEME|YOUR_.*_HERE''',
  '''postgres://postgres:postgres@localhost''',
  '''redis://localhost''',
  '''example\.com''',
]

[[rules]]
id = "jwt-secret-hardcoded"
description = "Hardcoded JWT secret"
regex = '''(?i)(jwt_secret|jwt_key|token_secret)\s[:=]\s['"][^'"]{16,}['"]'''
tags = ["secret", "jwt"]

[[rules]]
id = "database-url-with-password"
description = "Database URL with embedded password"
regex = '''(?i)(postgres|mysql|mongodb(\+srv)?):\/\/[^:]+:[^@\s]+@[^/\s]+'''
tags = ["secret", "database"]

CI Pipeline Integration

GitHub Actions

# .github/workflows/secrets-scan.yml
name: Secrets Scan
on: [push, pull_request]

jobs:
  gitleaks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0    # Full history for deep scan
      - uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

GitLab CI

secrets-scan:
  stage: test
  image: zricethezav/gitleaks:latest
  script:
    - gitleaks detect --source . --log-opts="--all" --verbose
  allow_failure: false

Pre-commit & CI Checklist

□ gitleaks installed on all developer machines
□ Pre-commit hook installed (husky, pre-commit framework, or manual)
□ .gitleaks.toml configured with project-specific allowlist
□ CI job runs gitleaks on every push and PR
□ CI job fetches full history (fetch-depth: 0) for deep scan
□ CI job blocks merge on detection (allow_failure: false)
□ Team documentation explains the hook and how to handle blocks
□ .gitleaks.toml is committed to the repo (not gitignored)
□ Test the hook: create a file with a fake AWS key, try to commit → blocked
□ Verify allowlist works: commit test file with allowed patterns → passes

---

Phase 5: Classification & Triage

Not every detection is a real secret. Classify findings before escalating.

Confidence Scoring

Triage Decision Tree

┌────────────────────────────────────┐
│  Secret detected in scan           │
└──────────────┬─────────────────────┘
               ▼
┌────────────────────────────────────┐     YES     ┌──────────────────┐
│  Is it in current code (HEAD)?     │────────────▶│  Is it a known   │
└──────────────┬─────────────────────┘             │  test/fake key?  │
               │ NO (history only)                 └────────┬─────────┘
               ▼                                        YES │    │ NO
┌────────────────────────────────────┐                  ▼   │    ▼
│  Was it ever pushed to a remote?   │          Add to       │  ROTATE
└──────────────┬─────────────────────┘          allowlist     │  IMMEDIATELY
           YES │    │ NO                                      │
               ▼    ▼                                         ▼
          ROTATE   Low risk —              ┌──────────────────────────┐
          (it was  verify it               │  Is it a production key? │
          exposed  was never               └───────────┬──────────────┘
          on       on remote)                  YES │        │ NO
          remote)                               ▼         ▼
                                          CRITICAL      HIGH
                                          INCIDENT      PRIORITY
                                          (page on-call)(rotate in
                                                        business hrs)

---

Phase 6: Secret Rotation Procedures

When a secret is confirmed leaked, rotate it immediately. Do not wait for "business hours" for production keys.

Rotation Procedures by Provider

Rotation Checklist

□ Identified ALL services/applications using the compromised credential
□ Generated new credential from the provider
□ Updated credential in secret management system (not in code)
□ Deployed updated credential to all consuming services
□ Verified all services work with the new credential
□ Revoked / deactivated the old credential
□ Confirmed the old credential no longer works (test it)
□ Documented the rotation in the incident log
□ Updated any CI/CD secrets that reference the credential
□ Notified the team that rotation is complete

---

Phase 7: Incident Response for Leaked Secrets

┌──────────────────────────────────────────────────────────────────────┐
│              LEAKED SECRET INCIDENT RESPONSE FLOW                    │
│                                                                      │
│  ┌──────────┐     0-5 min      ┌──────────────┐                    │
│  │ DETECT   │─────────────────▶│ ROTATE       │                    │
│  │          │                   │ Revoke old   │                    │
│  │ Scanner  │                   │ Deploy new   │                    │
│  │ alert,   │                   │ Verify new   │                    │
│  │ manual   │                   │ works        │                    │
│  │ report   │                   └──────┬───────┘                    │
│  └──────────┘                          │                            │
│                                        ▼ 5-30 min                   │
│                                ┌──────────────┐                     │
│                                │ BLAST RADIUS │                     │
│                                │              │                     │
│                                │ What could   │                     │
│                                │ the attacker │                     │
│                                │ access with  │                     │
│                                │ this key?    │                     │
│                                └──────┬───────┘                     │
│                                       │                             │
│                                       ▼ 30 min - 2 hr              │
│                                ┌──────────────┐                     │
│                                │ INVESTIGATE  │                     │
│                                │              │                     │
│                                │ Check logs   │                     │
│                                │ for key use  │                     │
│                                │ by unknown   │                     │
│                                │ IPs/agents   │                     │
│                                └──────┬───────┘                     │
│                                       │                             │
│                                       ▼ 2-24 hr                    │
│                                ┌──────────────┐                     │
│                                │ POST-MORTEM  │                     │
│                                │              │                     │
│                                │ How did it   │                     │
│                                │ leak? What   │                     │
│                                │ prevented    │                     │
│                                │ detection?   │                     │
│                                │ What changes │                     │
│                                │ prevent      │                     │
│                                │ recurrence?  │                     │
│                                └──────────────┘                     │
│                                                                      │
│  BLAST RADIUS ASSESSMENT BY KEY TYPE:                                │
│                                                                      │
│  ┌─────────────────────┬──────────────────────────────────────────┐  │
│  │ Compromised Key     │ Worst-Case Access                       │  │
│  ├─────────────────────┼──────────────────────────────────────────┤  │
│  │ AWS Root Key        │ Full account: EC2, S3, RDS, IAM, billing│  │
│  │ AWS IAM User Key    │ Whatever policies are attached to user  │  │
│  │ Database URL        │ Full read/write on all tables            │  │
│  │ Stripe Secret Key   │ Charge cards, read customer data, refund│  │
│  │ JWT Signing Secret  │ Forge any user session, full app access │  │
│  │ GitHub PAT          │ Read/write repos, secrets, actions      │  │
│  │ SMTP Password       │ Send email as your domain (phishing)    │  │
│  │ OAuth Client Secret │ Impersonate OAuth app, steal auth codes │  │
│  │ SSH Private Key     │ Server access for any host trusting it  │  │
│  └─────────────────────┴──────────────────────────────────────────┘  │
└──────────────────────────────────────────────────────────────────────┘

Incident Log Template

For every detected secret, create an incident record:

INCIDENT: SECRET-[YYYY-MM-DD]-[NNN]
DETECTED:   [timestamp]
ROTATED:    [timestamp]
SEVERITY:   [CRITICAL | HIGH | MEDIUM]

SECRET TYPE:    [e.g., AWS IAM Access Key]
LOCATION:       [file:line or commit:hash]
EXPOSURE:       [public repo | private repo | CI logs | local only]
EXPOSURE TIME:  [first commit date] to [rotation date]

BLAST RADIUS:
  - [service/resource the key could access]
  - [data at risk]

INVESTIGATION:
  - CloudTrail / access logs reviewed: [YES/NO]
  - Unauthorized usage found: [YES/NO]
  - If YES: [details]

ROOT CAUSE:
  - [How the secret entered git]
  - [Why pre-commit hook did not catch it]

REMEDIATION:
  - [x] Secret rotated
  - [x] Old secret revoked
  - [x] All services updated
  - [x] Pre-commit hook updated/fixed
  - [ ] Git history cleaned (optional — key already rotated)

PREVENTION:
  - [What will prevent this from happening again]

---

Full Secrets Scan Checklist

Current Codebase (Phase 1)

□ Ran gitleaks detect on working tree — zero findings or all triaged
□ Scanned all .yml/.yaml files for inline secrets
□ Scanned all .json config files for embedded credentials
□ Scanned Dockerfiles for ARG/ENV with real secrets
□ Scanned Terraform files for hardcoded provider credentials
□ Scanned Kubernetes manifests for cleartext Secret resources
□ Checked all logging statements — no secrets in log output
□ Checked all error messages — no connection strings in stack traces
□ Checked all HTTP responses — no secrets leaked to clients
□ Verified no hardcoded JWT secrets in source code

Git History (Phase 2)

□ Full-history gitleaks scan — zero findings or all triaged
□ Searched for deleted .env files in history
□ Searched for deleted .pem / .key files in history
□ Checked git stash for secrets
□ Verified remote URLs in .git/config contain no credentials
□ Checked all branches (including remote-tracking) for secrets

Environment & Config (Phase 3)

□ .gitignore includes all env file patterns
□ .gitignore includes .pem, .key, .p12, .pfx, *.jks
□ .gitignore includes terraform.tfstate, *.tfvars
□ .gitignore includes IDE config directories
□ .env.example exists with placeholder values only
□ No .env file is tracked by git (git ls-files | grep '\.env')
□ Production secrets stored in a secret manager (not env files on disk)
□ CI/CD secrets are masked in build logs
□ CI/CD secrets use environment-level protection where available

Pre-commit & CI (Phase 4)

□ gitleaks pre-commit hook installed
□ .gitleaks.toml committed with project allowlist
□ CI pipeline includes gitleaks scan job
□ CI job uses fetch-depth: 0 for full history
□ CI job blocks merge on secret detection
□ Pre-commit hook blocks commit on secret detection
□ Hook runs in under 5 seconds (developer experience)
□ Team knows how to handle a blocked commit

Rotation Readiness (Phase 5-6)

□ Every active credential has a documented rotation procedure
□ Secret manager supports rotation without code deployment
□ Rotation can happen without downtime (dual-key or grace period)
□ On-call team knows how to rotate each credential type
□ Rotation runbook is accessible during incidents
□ Post-rotation verification steps are documented

Monitoring (Ongoing)

□ GitHub Advanced Security secret scanning enabled (if available)
□ Periodic full-history scan scheduled (monthly recommended)
□ Alert channel configured for secret detection notifications
□ Scan results are logged and tracked over time
□ False positive suppressions are reviewed quarterly
□ New secret types are added to detection rules as services are onboarded

---

Tips for Best Results

• Start with pre-commit hooks — they prevent 90% of secret leaks. Everything else is cleanup.
• Run the full-history scan once on every repository, even old ones. Secrets from 2019 are still valid if never rotated.
• Keep the allowlist tight — every allowlist entry is a potential blind spot. Review quarterly.
• Use vendor-specific patterns first (AWS AKIA, Stripe sk_live_). They have near-zero false positives. Fall back to entropy for unknown types.
• Treat CI logs as public — many CI systems expose build logs to anyone with repo access. Never echo secrets in CI scripts.
• Audit dependency lockfiles — package-lock.json and yarn.lock can contain registry tokens if npm login was used carelessly.
• Check Docker layers — docker history shows ARG and ENV values from the build. Use multi-stage builds and --secret mounts.
• Rotate on schedule, not just on leak — 90-day rotation limits the window of exposure even for undetected leaks.
• Use short-lived credentials — AWS STS temporary credentials, OAuth2 tokens with short TTLs, and session-based database connections reduce the impact of any single leak.
• Test your detection — periodically commit a known fake secret to verify the pre-commit hook and CI scan actually catch it. If they do not, your pipeline has a gap.