Fix tox allowlist_externals and coverage-ci no-data failure

[Copilot]

Two bugs in tox.ini caused CI to fail across all matrix Python versions (py310–py313): external git calls were blocked by broken allowlist patterns, and coverage combine hard-failed when no coverage data existed.

Description

Bug 1 — Broken allowlist_externals patterns

Entries like /usr/bin/git * contain a literal space+asterisk. fnmatch('/usr/bin/git', '/usr/bin/git *') is always False (no executable path contains a space), so tox blocked every external command:

/usr/bin/git clone ... is not allowed, use allowlist_externals to allow it

Fix: replace /usr/bin/git *, /usr/bin/cp *, /usr/bin/sudo * with bare names (git, rm, cp) and update commands to match (dropping hardcoded /usr/bin/ prefixes for portability). Also adds missing rm, which was used but never listed.

Bug 2 — coverage-ci fails when upstream envs produce no data

When test envs fail before writing .coverage.* files, coverage combine exits 1 with "No data to combine", masking the real upstream failure.

Fix: extracted scripts/check_coverage.py — checks for .coverage.* files first, skips combine/xml/report gracefully (exit 0) if none exist, propagates real coverage failures with a diagnostic message.

files = glob.glob('.coverage.*')
if not files:
    print('No coverage data files found, skipping combine')
    return

Type of change

• Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

• Verified tox --listenvs and tox -e coverage-ci --showconfig parse correctly.
• Ran scripts/check_coverage.py locally with no .coverage.* files (exits 0, prints skip message) and with a dummy file present (runs coverage commands, propagates exit code on failure).

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works

Original prompt

Fix failing GitHub Actions job caused by tox blocking external git and downstream coverage combine failing with "No data to combine".

Context:

• Failing job id: 73815444784
• Workflow: .github/workflows/testing.yml at ref dbcba15856464cfe7a565f73fc64e160805894bc (Code Quality)
• Logs show tox env py311 runs a command that does git clone https://github.com/mitre/caldera.git --recursive '{homedir}/tmp' and fails with: /usr/bin/git ... is not allowed, use allowlist_externals to allow it.
• Then coverage-ci runs coverage combine and fails with No data to combine.

Tasks:

1. Locate tox configuration (likely tox.ini and/or pyproject.toml [tool.tox]) that defines envs py310, py311, py312, py313, and coverage-ci.
2. Update the tox env(s) that execute git clone to include allowlist_externals = git (and any other required externals such as bash/sh if used), so the clone step is permitted.
3. Improve robustness of coverage-ci so that if no .coverage* files exist it does not fail the job with "No data to combine" (e.g., conditionally run coverage combine only when coverage data exists, or tolerate the empty case). Keep behavior sensible for CI: fail on real coverage errors but not on empty combine.
4. Ensure changes work across matrix python versions defined in .github/workflows/testing.yml.
5. Update/add any tests or documentation if needed.

Deliverable:

• Open a PR against mitre/atomic with the fixes.
• Ensure CI passes.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: Please find a solution for failing job 73815444784. Use the logs, job definition, and any referenced files where the failure occurred. Keep your response focused on the solution and include code suggestions when appropriate.

Assistant: The failure in job 73815444784 is caused by tox blocking an external command:

• Log excerpt (py311):
  • git clone https://github.com/mitre/caldera.git --recursive ...
  • failed with /usr/bin/git ... is not allowed, use allowlist_externals to allow it

Because the py311 tox env fails early, no coverage data files are produced, and then the coverage-ci env fails with:

• coverage combine
• No data to combine (exit 1)

Fix (primary): allow git in the tox environment that runs the clone

Update your tox.ini (or the relevant tox config section) for the py31x test env that performs the clone to allow git as an external.

Example:

[tox]
envlist = py310,py311,py312,py313,coverage-ci

[testenv]
allowlist_externals =
    git
commands =
    git clone https://github.com/mitre/caldera.git --recursive {envtmpdir}/caldera
    pytest

Notes:

• Put git under the specific [testenv:py311] (and other py envs) if only those envs need it.
• Consider cloning into {envtmpdir} (or {toxworkdir}) rather than a hard-coded {homedir}/tmp path to avoid permission/path issues on runners.

Fix (secondary / robustness): don’t fail coverage when there’s nothing to combine

Even with the tox fix, it’s good to make coverage-ci resilient so it doesn’t hard-fail when upstream envs produce no coverage (e.g., if one Python version fails). In the coverage-ci commands, change to something like:

[testenv:coverage-ci]
commands =
    python -c "import glob,sys; sys.exit(0 if glob.glob('.coverage*') else 0)"
    coverage combine || true

Or more cleanly, guard the combine step in shell:

commands =
    sh -c 'ls -1 .coverage* >/dev/null 2>&1 && coverage combine || echo "No coverage files to combine"'

Why this resolves the job

• Allowlisting git removes the tox policy violation, so the test env can proceed past the clone step.
• Once tests run, coverage data exists; coverage combine will succeed.
• The optional guard prevents coverage-ci from masking the real failure when tests didn’t run.

User: yes fix and implement

Assistant: I’ve kicked off the automated fix (updating the tox configuration to allow the external git call and making the coverage combine step tolerant when no coverage files exist), and I’m now waiting for the run to complete so I can confirm the PR details and CI is passing.

This pull request was created from Copilot chat.