Add code coverage#961
Conversation
25adb30 to
0ab8eae
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds comprehensive code coverage tracking to the CI workflow to support OpenSSF Silver Badge requirements. It instruments both unit and integration tests to collect coverage data, merges the profiles, and generates coverage reports that are displayed in GitHub's step summary.
Changes:
- Modified integration tests to collect coverage data with
-covermode=atomic -coverpkg=./...flags - Modified unit tests to collect coverage data, excluding the
/testdirectory packages - Added new
coverage-reportjob that merges all coverage profiles and generates summary reports
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 | ||
| with: | ||
| name: coverage-unit | ||
| path: coverage | ||
|
|
||
| - name: Download integration coverage artifacts | ||
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 |
There was a problem hiding this comment.
Missing version comment for the download-artifact action. All other action uses in this workflow include version comments (e.g., # v6.0.0). Please add a version comment to maintain consistency with the established convention in this file.
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 | |
| with: | |
| name: coverage-unit | |
| path: coverage | |
| - name: Download integration coverage artifacts | |
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 | |
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # download-artifact pinned SHA | |
| with: | |
| name: coverage-unit | |
| path: coverage | |
| - name: Download integration coverage artifacts | |
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # download-artifact pinned SHA |
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 | ||
| with: | ||
| name: coverage-unit | ||
| path: coverage | ||
|
|
||
| - name: Download integration coverage artifacts | ||
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 |
There was a problem hiding this comment.
Missing version comment for the download-artifact action. All other action uses in this workflow include version comments (e.g., # v6.0.0). Please add a version comment to maintain consistency with the established convention in this file.
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 | |
| with: | |
| name: coverage-unit | |
| path: coverage | |
| - name: Download integration coverage artifacts | |
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 | |
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.1.8 | |
| with: | |
| name: coverage-unit | |
| path: coverage | |
| - name: Download integration coverage artifacts | |
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.1.8 |
| runs-on: ubuntu-22.04 | ||
| needs: | ||
| - unit | ||
| - integration |
There was a problem hiding this comment.
The coverage-report job will not run if either the unit or integration jobs fail, even though those jobs use if: always() to upload coverage artifacts on failure. This means coverage reports won't be generated when tests fail. Consider adding if: always() to the coverage-report job (or using a more specific condition like if: success() || failure()) to ensure coverage reports are generated even when tests fail, which would be valuable for understanding coverage gaps in failing test scenarios.
| - integration | |
| - integration | |
| if: ${{ always() }} |
| exit 1 | ||
| fi | ||
|
|
||
| "$(go env GOPATH)/bin/gocovmerge" coverage/unit.out ${integration_profiles} > coverage/all.out |
There was a problem hiding this comment.
The unquoted variable expansion ${integration_profiles} could potentially cause issues with shell word splitting if any filenames contain spaces or special characters. While the current suite names don't contain spaces, it's a best practice to use an array or quote the variable to make the script more robust. Consider using a bash array: integration_profiles=($(find...)) and then "${integration_profiles[@]}" in the gocovmerge command.
0ab8eae to
853ecfb
Compare
de76fc1 to
905a3e6
Compare
905a3e6 to
b94d946
Compare
|
@kartikjoshi21 CI didn't run on this because there was temporarily a conflict (commit is now reverted so no conflict). |
124edfa to
fe8704e
Compare
@cpuguy83 Updated the PR, Thanks |
693bbc9 to
2b605d0
Compare
This PR adds automated Go coverage reporting to the CI pipeline. Unit and integration tests now generate coverage profiles (one per integration matrix suite). It also collects coverage for the Dalec BuildKit frontend by building it with coverage enabled and exporting its covdata during solves. A new coverage-report job downloads all profiles, merges them, and uploads a combined report (including an index.html coverage page) as a workflow artifact, plus a summary in the Actions run. Fixes: project-dalec#889 Signed-off-by: Kartik Joshi <karikjoshi21@gmail.com>
2b605d0 to
4d0717a
Compare
Signed-off-by: Kartik Joshi <karikjoshi21@gmail.com>
4d0717a to
0f7b13d
Compare
|
Comment from GPT-5.5: I think frontend coverage is currently being converted with metadata but no counters. Evidence:
This PR writes counter files as That matches the latest CI artifacts: all 14 Suggested fix: write counter files using exactly |
Add Go coverage reporting to CI (end-to-end)
This PR adds automated Go coverage reporting to the CI pipeline.
coverage-reportjob downloads all profiles, merges them, and uploads a combined report (including anindex.htmlcoverage page) as a workflow artifact, plus a summary in the Actions run.Fixes: #889