Skip to content

ci(workflows): migrate to gavel for test execution and add timeout li…#1947

Open
moshloop wants to merge 1 commit into
mainfrom
pr/gavel-tests
Open

ci(workflows): migrate to gavel for test execution and add timeout li…#1947
moshloop wants to merge 1 commit into
mainfrom
pr/gavel-tests

Conversation

@moshloop
Copy link
Copy Markdown
Member

@moshloop moshloop commented May 1, 2026

…mits

Replace custom test execution with gavel action across all test workflows (test, e2e, e2e-blobs, migrate). Add explicit timeout-minutes to all jobs to prevent hanging workflows. Remove postgres 14 from test matrix. Drop gavel.yml workflow in favor of integrated gavel action usage. Add migration fixture for database schema testing. Simplify environment variable handling and remove conditional RLS disabling logic.

BREAKING CHANGE: Test execution now uses gavel action instead of make targets; CI behavior and reporting format changes accordingly.

…mits

Replace custom test execution with gavel action across all test workflows (test, e2e, e2e-blobs, migrate). Add explicit timeout-minutes to all jobs to prevent hanging workflows. Remove postgres 14 from test matrix. Drop gavel.yml workflow in favor of integrated gavel action usage. Add migration fixture for database schema testing. Simplify environment variable handling and remove conditional RLS disabling logic.

BREAKING CHANGE: Test execution now uses gavel action instead of make targets; CI behavior and reporting format changes accordingly.
@moshloop moshloop requested a review from yashmehrotra May 1, 2026 11:55
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 1, 2026

Warning

Rate limit exceeded

@moshloop has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 56 minutes and 19 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: bd4268aa-e43b-46e3-87bd-11bba189692e

📥 Commits

Reviewing files that changed from the base of the PR and between 7ce008e and 4f8443c.

📒 Files selected for processing (9)
  • .github/fixtures/.gitignore
  • .github/fixtures/migrate.fixture.md
  • .github/workflows/codeql.yml
  • .github/workflows/gavel.yml
  • .github/workflows/lint.yml
  • .github/workflows/release.yml
  • .github/workflows/scorecard.yml
  • .github/workflows/test.yaml
  • .github/workflows/verify-generated.yaml
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch pr/gavel-tests
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch pr/gavel-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
Review rate limit: 0/1 reviews remaining, refill in 56 minutes and 19 seconds.

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Benchstat (Other)

Base: 7ce008ee0d2d314e61d5334a17b385650e718f0c
Head: 4f8443c6e5b3eb9307c552e9e432c76bf90b64cb

📊 5 minor regression(s) (all within 5% threshold)

Benchmark Base Head Change p-value
UpdateOfConfigsWithProperties-4 7.429m 7.800m +4.99% 0.002
ResourceSelectorQueryBuild/tags-4 17.07µ 17.91µ +4.89% 0.002
ResourceSelectorConfigs/tags-4 30.29m 31.42m +3.74% 0.002
ResourceSelectorQueryBuild/name-4 43.71µ 45.16µ +3.31% 0.002
ResourceSelectorConfigs/name-4 219.5µ 223.2µ +1.67% 0.009
Full benchstat output
goos: linux
goarch: amd64
pkg: github.com/flanksource/duty/bench
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │ bench-base.txt │           bench-head.txt           │
                                                       │     sec/op     │    sec/op     vs base              │
InsertionForRowsWithAliases/external_users.aliases-4       592.1µ ±  2%   588.3µ ±  2%       ~ (p=0.485 n=6)
InsertionForRowsWithAliases/config_items.external_id-4     1.107m ± 11%   1.104m ± 12%       ~ (p=1.000 n=6)
InsertionOfConfigsWithProperties-4                         3.669m ±  1%   3.700m ±  4%       ~ (p=0.093 n=6)
UpdateOfConfigsWithProperties-4                            7.429m ±  1%   7.800m ±  2%  +4.99% (p=0.002 n=6)
ResourceSelectorConfigs/name-4                             219.5µ ±  3%   223.2µ ±  2%  +1.67% (p=0.009 n=6)
ResourceSelectorConfigs/name_and_type-4                    237.9µ ±  5%   243.5µ ±  3%       ~ (p=0.180 n=6)
ResourceSelectorConfigs/tags-4                             30.29m ±  3%   31.42m ±  3%  +3.74% (p=0.002 n=6)
ResourceSelectorQueryBuild/name-4                          43.71µ ±  3%   45.16µ ±  1%  +3.31% (p=0.002 n=6)
ResourceSelectorQueryBuild/name_and_type-4                 63.67µ ±  4%   65.67µ ±  1%       ~ (p=0.065 n=6)
ResourceSelectorQueryBuild/tags-4                          17.07µ ±  2%   17.91µ ±  1%  +4.89% (p=0.002 n=6)
geomean                                                    516.2µ         528.5µ        +2.38%

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Benchstat (RLS)

Base: 7ce008ee0d2d314e61d5334a17b385650e718f0c
Head: 4f8443c6e5b3eb9307c552e9e432c76bf90b64cb

📊 6 minor regression(s) (all within 5% threshold)

Benchmark Base Head Change p-value
RLS/Sample-15000/catalog_changes/Without_RLS-4 5.271m 5.516m +4.65% 0.002
RLS/Sample-15000/config_types/Without_RLS-4 3.940m 4.027m +2.19% 0.002
RLS/Sample-15000/config_types/With_RLS-4 125.7m 127.1m +1.10% 0.002
RLS/Sample-15000/config_summary/With_RLS-4 748.8m 754.9m +0.82% 0.009
RLS/Sample-15000/config_classes/With_RLS-4 125.8m 126.7m +0.75% 0.002
RLS/Sample-15000/config_names/Without_RLS-4 12.65m 12.72m +0.58% 0.009
✅ 8 improvement(s)
Benchmark Base Head Change p-value
RLS/Sample-15000/analysis_types/With_RLS-4 4.160m 3.904m -6.15% 0.002
RLS/Sample-15000/config_summary/Without_RLS-4 63.12m 61.12m -3.17% 0.002
RLS/Sample-15000/change_types/With_RLS-4 5.432m 5.266m -3.07% 0.004
RLS/Sample-15000/change_types/Without_RLS-4 5.373m 5.277m -1.79% 0.041
RLS/Sample-15000/config_detail/Without_RLS-4 3.985m 3.922m -1.57% 0.015
RLS/Sample-15000/analyzer_types/Without_RLS-4 3.781m 3.741m -1.05% 0.041
RLS/Sample-15000/config_classes/Without_RLS-4 3.330m 3.298m -0.95% 0.002
RLS/Sample-15000/configs/With_RLS-4 124.4m 123.7m -0.52% 0.015
Full benchstat output
goos: linux
goarch: amd64
pkg: github.com/flanksource/duty/bench
cpu: AMD EPYC 7763 64-Core Processor                
                                               │ bench-base.txt │          bench-head.txt           │
                                               │     sec/op     │   sec/op     vs base              │
RLS/Sample-15000/catalog_changes/Without_RLS-4      5.271m ± 1%   5.516m ± 2%  +4.65% (p=0.002 n=6)
RLS/Sample-15000/catalog_changes/With_RLS-4         130.4m ± 1%   130.8m ± 0%       ~ (p=0.093 n=6)
RLS/Sample-15000/config_changes/Without_RLS-4       5.387m ± 3%   5.476m ± 2%       ~ (p=0.180 n=6)
RLS/Sample-15000/config_changes/With_RLS-4          132.8m ± 2%   131.8m ± 1%       ~ (p=0.485 n=6)
RLS/Sample-15000/config_detail/Without_RLS-4        3.985m ± 1%   3.922m ± 2%  -1.57% (p=0.015 n=6)
RLS/Sample-15000/config_detail/With_RLS-4           125.8m ± 3%   128.6m ± 1%       ~ (p=0.065 n=6)
RLS/Sample-15000/config_names/Without_RLS-4         12.65m ± 0%   12.72m ± 2%  +0.58% (p=0.009 n=6)
RLS/Sample-15000/config_names/With_RLS-4            126.2m ± 1%   125.5m ± 1%       ~ (p=0.093 n=6)
RLS/Sample-15000/config_summary/Without_RLS-4       63.12m ± 3%   61.12m ± 1%  -3.17% (p=0.002 n=6)
RLS/Sample-15000/config_summary/With_RLS-4          748.8m ± 1%   754.9m ± 1%  +0.82% (p=0.009 n=6)
RLS/Sample-15000/configs/Without_RLS-4              7.141m ± 1%   7.125m ± 1%       ~ (p=0.589 n=6)
RLS/Sample-15000/configs/With_RLS-4                 124.4m ± 1%   123.7m ± 0%  -0.52% (p=0.015 n=6)
RLS/Sample-15000/analysis_types/Without_RLS-4       3.922m ± 1%   3.905m ± 1%       ~ (p=0.240 n=6)
RLS/Sample-15000/analysis_types/With_RLS-4          4.160m ± 3%   3.904m ± 3%  -6.15% (p=0.002 n=6)
RLS/Sample-15000/analyzer_types/Without_RLS-4       3.781m ± 1%   3.741m ± 2%  -1.05% (p=0.041 n=6)
RLS/Sample-15000/analyzer_types/With_RLS-4          3.779m ± 1%   3.776m ± 2%       ~ (p=0.937 n=6)
RLS/Sample-15000/change_types/Without_RLS-4         5.373m ± 2%   5.277m ± 2%  -1.79% (p=0.041 n=6)
RLS/Sample-15000/change_types/With_RLS-4            5.432m ± 2%   5.266m ± 3%  -3.07% (p=0.004 n=6)
RLS/Sample-15000/config_classes/Without_RLS-4       3.330m ± 1%   3.298m ± 1%  -0.95% (p=0.002 n=6)
RLS/Sample-15000/config_classes/With_RLS-4          125.8m ± 0%   126.7m ± 0%  +0.75% (p=0.002 n=6)
RLS/Sample-15000/config_types/Without_RLS-4         3.940m ± 0%   4.027m ± 1%  +2.19% (p=0.002 n=6)
RLS/Sample-15000/config_types/With_RLS-4            125.7m ± 1%   127.1m ± 0%  +1.10% (p=0.002 n=6)
geomean                                             19.47m        19.41m       -0.30%

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Gavel summary

Source Pass Fail Skip Duration
./tests/e2e-blobs 0 1 0 -

Totals: 0 passed · 1 failed · 0 skipped · -

Failing tests

ginkgo Execution

�[38;5;9m�[1mginkgo run�[0m �[38;5;9mfailed�[0m
  flag provided but not defined: -ginkgo.label-filter

�[1mginkgo run <FLAGS> <PACKAGES> -- <PASS-THROUGHS>�[0m
... (226 more lines truncated)

View full results

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Gavel summary

Source Pass Fail Skip Duration
./tests/e2e 0 1 0 -

Totals: 0 passed · 1 failed · 0 skipped · -

Failing tests

ginkgo Execution

�[38;5;9m�[1mginkgo run�[0m �[38;5;9mfailed�[0m
  flag provided but not defined: -ginkgo.label-filter

�[1mginkgo run <FLAGS> <PACKAGES> -- <PASS-THROUGHS>�[0m
... (226 more lines truncated)

View full results

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Gavel crashed before producing results

Exit code: 0
Error: gavel exited 0 before writing results

Last lines of gavel.log

11:59:48.628 INF Loaded 1 total fixtures in 1 files ✓ PASS apply (802.289401ms) ✓ PASS apply (802.289401ms)  

Full gavel.log, JSON stub, and HTML stub are in the workflow artifact.

View full results

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Gavel crashed before producing results

Exit code: 0
Error: gavel exited 0 before writing results

Last lines of gavel.log

11:59:54.630 INF Loaded 1 total fixtures in 1 files ✓ PASS apply (930.439813ms) ✓ PASS apply (930.439813ms)  

Full gavel.log, JSON stub, and HTML stub are in the workflow artifact.

View full results

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Gavel crashed before producing results

Exit code: 0
Error: gavel exited 0 before writing results

Last lines of gavel.log

12:02:16.668 INF Loaded 1 total fixtures in 1 files ✓ PASS apply (292.593527ms) ✓ PASS apply (292.593527ms)  

Full gavel.log, JSON stub, and HTML stub are in the workflow artifact.

View full results

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Gavel crashed before producing results

Exit code: 0
Error: gavel exited 0 before writing results

Last lines of gavel.log

12:02:22.535 INF Loaded 1 total fixtures in 1 files ✓ PASS apply (308.488668ms) ✓ PASS apply (308.488668ms)  

Full gavel.log, JSON stub, and HTML stub are in the workflow artifact.

View full results

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Gavel summary

Source Pass Fail Skip Duration
github.com/flanksource/duty/dataquery 28 0 3 -
github.com/flanksource/duty/logs/azureloganalytics 0 0 1 -
github.com/flanksource/duty/logs/bigquery 0 0 1 -
lint: betterleaks 0 0 1 -
lint: eslint 0 0 1 -
lint: golangci-lint 0 0 1 -
lint: jscpd 0 0 1 -
lint: markdownlint 0 0 1 -
lint: pyright 0 0 1 -
lint: ruff 0 0 1 -
lint: vale 0 0 1 -
tests 794 0 3 15.0s
25 more passing source(s) 679 0 0 23.6s

Totals: 1501 passed · 0 failed · 16 skipped · 38.6s

View full results

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Gavel summary

Source Pass Fail Skip Duration
github.com/flanksource/duty/dataquery 28 0 3 -
github.com/flanksource/duty/logs/azureloganalytics 0 0 1 -
github.com/flanksource/duty/logs/bigquery 0 0 1 -
lint: betterleaks 0 0 1 -
lint: eslint 0 0 1 -
lint: golangci-lint 0 0 1 -
lint: jscpd 0 0 1 -
lint: markdownlint 0 0 1 -
lint: pyright 0 0 1 -
lint: ruff 0 0 1 -
lint: vale 0 0 1 -
tests 794 0 3 15.5s
25 more passing source(s) 679 0 0 27.0s

Totals: 1501 passed · 0 failed · 16 skipped · 42.5s

View full results

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant