Add actions-audit.py for auditing Apache repo security tooling

[potiuk]

Summary

• Adds utils/actions-audit.py, a script that audits all apache/ GitHub repositories for baseline Actions security configurations
• Checks for: dependabot (github-actions ecosystem with 4-day cooldown), CodeQL analysis for Actions, zizmor workflow scanning, and ASF allowlist-check
• Creates PRs to add missing configurations, with interactive yes/no/quit confirmation before each
• Uses GraphQL for efficient batch repo fetching, with PMC-based filtering (--pmc), --dry-run mode, and --max-num limit
• Before opening a PR, runs zizmor against existing workflows — if errors are found, comments out CodeQL and zizmor workflows with instructions for the PMC to follow up
• Checks for existing PRs (open, closed, or merged) to avoid duplicates
• Documents prerequisites, reasoning, and usage in README.md

Test plan

• uv run utils/actions-audit.py --help — verify CLI options
• uv run utils/actions-audit.py --dry-run --pmc accumulo --max-num 3 — verify dry-run with PMC filter
• uv run utils/actions-audit.py --dry-run --pmc airflow --max-num 5 — verify mixed results (airflow has some tooling, detects only missing items)
• uv run utils/actions-audit.py --dry-run --max-num 15 — verify broader pagination
• Test PR creation on a low-risk repo with --pmc <pmc> --max-num 1
• Verify interactive confirmation prompt (yes/no/quit)

Generated-by: Claude Opus 4.6 (1M context)

[potiuk]

It's a bit slow but we will kikely be able to optimize it more

[potiuk]

Ok. It is fast and ratehr nice now:

[potiuk]

Tht is --dry-run, if we run it without it (tommorow) PRs will contain commented-out zizmor checks, list of errors generated by zizmor and kind request to PMC to follow up, uncomment it and fix the errors.

[potiuk]

Again - I have not written a single line of code - asked Claude to create it - all of it took less than hour including speed optimisation and nicer output.

[raboof]

I think this belongs in https://github.com/apache/infrastructure-gha-workflow-scanner , not in this repo.

[raboof]

I'm also not convinced we have full consensus on all the things this tool checks/proposes.

[dave2wave]

I am not able to scan "tooling" repositories.

I get a TOML parse error on line 16: `exclude-newer = "4 days".

[dave2wave]

Fails to find the "otava" PPMC.

[dave2wave]

Fails to find the "poi" PMC.

[dave2wave]

Fails to find any "airflow" repositories.

I ran using a checkout of the pr branch using uv run utils/actions-audit.py --dry-run --pmc ${pmc}

[dave2wave]

I guess I'm missing the GH Token, but judging from the whimsy output you must be validating PMC and that leaves out checking non-pmc repositories like infrastructure-*, tooling-*, and www-site

[dave2wave]

Missing changes to pyproject.toml?

[dave2wave]

We have not set a policy for cooldown. The tooling team has our own policy of 7 days and we run Dependabot on Monday. Not all Dependabot checks have implemented a cooldown. Be wary of precheck Dependabot which was recently added but is automatically a 0-day.

A clear consensus is needed for what help and recommendations should be provided to PMCs. @raboof and the Security team should make recommendations in consultation with Infra. Once that is done then this PR can be adjusted.

[potiuk]

I'm also not convinced we have full consensus on all the things this tool checks/proposes.

Sure. This is a proposal and once we get consensus it can i mplement any of the agreed approach. This is mostly to demonstrate what can be done automtically and decide if this might be a viable process - following "do-> show -> agree -> update -> pattern"

How do we get to consensus here? Should we make a proposal/vote on it ? I am happy to start discussion on it. Who has the binding votes / has to agree on it ?

[potiuk]

I think this belongs in https://github.com/apache/infrastructure-gha-workflow-scanner , not in this repo.

I think it depends on the process here. This proposal is that we attempt to proactively propose PRs to add standards - and automate it - as soon as we see that PMCs are fine with accepting those - I am happy to do some initial trial, as soon as we get consensus on what is a good set of checks that we want all our PMCs to follow.

But yeah - I can move it there if we think it's a good idea, however I would propose to get the consensus first here -- treating it as a test playground, and then when we know what we want to check for, we can move it to the scanner.

Note that the primary goal of this one is to actuallly make a PR to the target repo to add those workflow rather than check them (and of course it requires buy-in from teh PMC to accept it).

My proposal would be - again - once we reach the consensus on what are the things we expect - is to do a small trial on a few PMCS and submit PRs - after contacting those projects on devlist and suggesting this to happen - I am happy to lead it with a number of "friendly" PMCS. Once we have it implemented there (hopefully) - we might update our justification , axplanation and start submitting those PRs in batches. And get the scanner to verify it and possibly produce a nice summary / % etc.

Note also that this is primary a locakl CLI tool, not somethign that runs only on schedule (check yes, but proposing a PR should be human-driven - to get PRs from one of the people involved in the process.

Again - this is only proposal - showing what could be done. And I am happy to process feedback and adjust accordingly.

[potiuk]

I am not able to scan "tooling" repositories.
I get a TOML parse error on line 16: `exclude-newer = "4 days".

That's because you have an older uv tool installed - only several weeks ago they added ability to specify "days" instaed of some more abstrac way. We had similar issue in airflow and adding minimum version of uv in pyproject.toml gets better error message (and asks you to upgrade your uv. Let me add it

[potiuk]

I am not able to scan "tooling" repositories.
Fails to find the "otava" PPMC.
Fails to find the "poi" PMC.
Fails to find any "airflow" repositories.

Optimized the retrieval now when --pmc is used. now it uses direct search and limits the number of data retrieved by graphql. Previously, the problem was that it took too long and rate limits could kick in (errors were not shown).

Added --verbose mode to see what's going on and dumping errors on output.

[potiuk]

OK. should be ready for next try (but yes we should definitely agree what our expectations are before we even try it in more widespread way.

[dave2wave]

Generally all my concerns have been met aside from policy..github/
Essentially this PR will help projects assure the following workflows and configurations are present:

1. .github/dependabot.yml
2. .github/zizmor.yml
3. .gtihub/workflows/zizmor.yml
4. .github/workflows/codeql-analysis.yml
5. .github/workflows/allowlistcheck.yml

[dave2wave]

Changes no longer requested

No longer needed

[dave2wave]

Reason to keep this utility in this repository include that it is more appropriate here if PMCs and others will voluntarily run this. In other words they take action.

Reason to move this to the workflow scanner is the intent is to enforce use of these workflows on every PMC.

Either way this may be better as an action in this repository with workflows in the scanner and within any project that wants to handle their own compliance on their terms.

One step further would be to configure a section in .asf.yaml for any choices that are decided on about these configurations.

[potiuk]

Reason to move this to the workflow scanner is the intent is to enforce use of these workflows on every PMC.

Yep. It might be.

I am fine whatever we (with emphasis on who is we :) ? ) decide - and I am happy to adjust the aciton to suite the best usage - we can adapt and change it and add more things to it, as long as we agree on policies and how we "talk" to the PMCs about it. This is largely just a proposal

I think at some point of time (quoting the running joke from FOSS backstage - "eiher in a few months or maybe next Tuesday" because this is how fast the threat landscape changes - we will have to enforce at least some of those recommendations. On one hand PMCs can implement them - but essentially Infra might get all the heat when security keys leak, or infrastructure VMs are compromised, or SVN access leaks.

The question is how - technically it's possible to verify and create PRs (for example - like shown in this PR), of course the difficult part is how to enforce it and important part is exactly what policies to enforce - hopefully that can be worked out.