Skip to content

Enhance verify-action-build.py with deep composite/docker verification#629

Open
potiuk wants to merge 2 commits intomainfrom
worktree-fix-verify-action-node-version
Open

Enhance verify-action-build.py with deep composite/docker verification#629
potiuk wants to merge 2 commits intomainfrom
worktree-fix-verify-action-node-version

Conversation

@potiuk
Copy link
Copy Markdown
Member

@potiuk potiuk commented Mar 30, 2026

Summary

  • Fix Node.js version detection: correctly extract version from using: field (e.g. node2020) and handle use: typos
  • Deep verification for non-JS actions: composite and docker actions now get comprehensive analysis instead of just "SKIPPED":
    • Recursive nested action inspection (all action types — composite, node, docker)
    • Dockerfile analysis (base image pinning, suspicious commands)
    • Script pattern scanning (eval, exec, pipe-to-shell, base64 obfuscation, HTTP requests)
    • Dependency pinning checks (Python requirements, package.json, lock files)
    • Action metadata analysis (shell injection risks, GITHUB_ENV/GITHUB_PATH writes, secrets references)
    • Repository metadata (license, security policy, well-known org trust signal)
  • Trusted orgs: nested actions from actions and github orgs skip deep recursive inspection (still checked for hash-pinning) and are marked as trusted in the summary table
  • Well-known orgs (informational signal in repo metadata check): actions, github, google-github-actions, aws-actions, azure, docker, hashicorp, pypa, gradle
  • Verification summary table: structured at-a-glance table of all checks with pass/warn/fail status, plus nested actions sub-table showing type, pinning, approved, and trusted status
  • Extract action refs from actions.yml: --from-pr and dependabot review now also detect action references from added entries in actions.yml (not just workflow uses: lines), e.g. PR Add lhotari/sandboxed-trivy-action v1.0.1 #582
  • Interactive improvements: open-in-browser + approve flow after verification; all prompts support q to quit cleanly

Test plan

  • Tested with composite action: pypa/gh-action-pypi-publish@ed0c539...
  • Tested with JS action: actions/checkout@11bd719... — no regressions
  • Tested --ci mode (non-interactive)
  • Tested q quit in interactive prompts
  • Tested --from-pr 582 (actions.yml format extraction)
  • Tested --from-pr 618 (workflow uses: format extraction)
  • Test with --check-dependabot-prs flow
  • Test with a docker-type action

Generated with Claude Code

…dling

The Docker container was hardcoded to node:20-slim, causing large spurious
diffs when verifying actions built with a different Node.js version (e.g.
azure/setup-helm v5.0.0 which switched to node24). Now detects the Node.js
version from the action's action.yml `using:` field before building.

Also fix the PR diff extraction regex to match `use:` in addition to `uses:`
so that PRs with this common typo (e.g. #491) can still be processed.

Generated-by: Claude
@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Mar 30, 2026

Example raport before approval:

image

@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Mar 30, 2026

Generated with literally few prompts with Claude and few manual fixes after testing and reviewing code.

@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Mar 30, 2026

Verified two proposed actions - also enhanced action detection so that `--from-pr will properly detect the action from a PR manully adding actions.yaml

For non-JS actions the script previously just printed "SKIPPED". Now it
performs comprehensive analysis:

- Recursive nested action inspection (all types, not just composite),
  with trusted org skip for actions/ and github/
- Dockerfile analysis (base image pinning, suspicious commands)
- Script pattern scanning (eval, exec, pipe-to-shell, obfuscation)
- Dependency pinning checks (Python requirements, package.json, lock files)
- Action metadata analysis (shell injection, GITHUB_ENV writes, secrets)
- Repository metadata (license, security policy, well-known org)
- Structured verification summary table with nested actions sub-table
- Interactive open-in-browser + approve flow after verification
- All prompts now support 'q' to quit cleanly
- Extract action refs from actions.yml entries in PR diffs (--from-pr)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@potiuk potiuk force-pushed the worktree-fix-verify-action-node-version branch from 612d6b8 to b5e34c1 Compare March 30, 2026 03:04
@dave2wave dave2wave removed their assignment Mar 30, 2026
Copy link
Copy Markdown
Member

@dave2wave dave2wave left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hard to tell since the current three PRs don't fall completely into the changes. I did notice better workflow.

@dave2wave
Copy link
Copy Markdown
Member

@potiuk it looks this PR has fallen behind. Close, rebase, or recreate?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants