DO NOT MERGE: Verify CI output-on-failure with reduced verbosity#5972
Draft
hjmjohnson wants to merge 3 commits intoInsightSoftwareConsortium:mainfrom
Draft
DO NOT MERGE: Verify CI output-on-failure with reduced verbosity#5972hjmjohnson wants to merge 3 commits intoInsightSoftwareConsortium:mainfrom
hjmjohnson wants to merge 3 commits intoInsightSoftwareConsortium:mainfrom
Conversation
Replace -VV (extra verbose) with -V (verbose) in all CI pipeline invocations of `ctest -S`. This change applies to all 8 Azure DevOps pipeline YAML files and the GitHub Actions arm.yml workflow. Background: The -VV flag causes CTest to print the full stdout/stderr of every test, regardless of whether it passed or failed. With ITK's 3000+ tests, this generates hundreds of megabytes of CI log output, making it difficult to find the actual failure in a failing build. Every pipeline already sets the environment variable CTEST_OUTPUT_ON_FAILURE=1, which instructs CTest to print test output only when a test fails. However, this variable has been effectively dead code because -VV unconditionally prints all output, overriding the output-on-failure behavior. With this change: - -V (verbose) prints CTest progress: test names, pass/fail status, and timing — sufficient for monitoring build progress - CTEST_OUTPUT_ON_FAILURE=1 now takes effect, printing full test output only for tests that fail - CI logs are dramatically smaller and failures are easier to find Note: The CTEST_OUTPUT_ON_FAILURE environment variable works in both direct ctest invocations and ctest -S scripted/dashboard mode. The ctest_test() CMake command used internally by itk_common.cmake does not have an OUTPUT_ON_FAILURE keyword argument, but the environment variable is read by the CTest process regardless of invocation mode. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
github-actions
bot
added
type:Infrastructure
ctest_build() captures compiler diagnostics in Build.xml for CDash submission but does not print them to the CI log. With -V (instead of -VV), build output scrolls past quickly and warnings are easy to miss entirely. Add report_build_diagnostics.py which parses the <Warning> and <Error> elements from Build.xml and prints them in a clearly delimited section. Add a "Report build warnings and errors" step to every Azure DevOps and GitHub Actions pipeline, running after the build/test step with succeededOrFailed()/always() so diagnostics are reported even when the build fails. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DO NOT MERGE — this commit intentionally introduces: 1. An unused variable in itkObject.cxx to generate a compiler warning (-Wunused-variable) that should appear in CDash warning counts. 2. A deliberate EXIT_FAILURE in itkPointGeometryTest to verify that CTEST_OUTPUT_ON_FAILURE=1 produces visible test output in CI logs when using -V instead of -VV. This is a test artifact for PR InsightSoftwareConsortium#5971 (reduce CTest verbosity). If the CI logs show the "DELIBERATE FAILURE" message in the test output section, the new verbosity scheme is working correctly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
hjmjohnson
force-pushed
the
wip-test-ci-verbosity
branch
from
b0f4064 to
34cf43b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DO NOT MERGE
This PR intentionally introduces a compiler warning and a test failure to verify that the reduced CTest verbosity scheme proposed in #5971 correctly surfaces failures in CI logs.
What this PR introduces
1. Compiler warning (
-Wunused-variable)An unused
static intvariable inModules/Core/Common/src/itkObject.cxx:This should appear in CDash's warning count for every build.
2. Deliberate test failure (
itkPointGeometryTest)The test now returns
EXIT_FAILUREand prints a diagnostic message to stderr:What to look for in CI logs
This PR includes the
-VV→-Vchange from #5971. With the new scheme:-VCTEST_OUTPUT_ON_FAILURE=1-VVdeliberately_unused_variable_for_ci_testshould appear in the build outputIf all four conditions are met, #5971 is working as intended and can be merged.
Cleanup
This PR will be closed without merging once CI results have been inspected.
🤖 Generated with Claude Code