Skip to content

Support reporting statistics in spark datasource#8057

Open
robert3005 wants to merge 2 commits into
developfrom
rk/sparkstats
Open

Support reporting statistics in spark datasource#8057
robert3005 wants to merge 2 commits into
developfrom
rk/sparkstats

Conversation

@robert3005
Copy link
Copy Markdown
Contributor

Spark mostly focuses on sizeInBytes which we populate from file sizes with
scaling. We also report numRows since that exists in our datasource.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 22, 2026

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 1266 untouched benchmarks


Comparing rk/sparkstats (4d9b080) with develop (126c431)

Open in CodSpeed

@robert3005 robert3005 force-pushed the rk/sparkstats branch 2 times, most recently from 6da0126 to e97797d Compare May 27, 2026 15:59
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant