demo: Add benchmark-react with normalization and ref-stability scenarios#3783
demo: Add benchmark-react with normalization and ref-stability scenarios#3783
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
🦋 Changeset detectedLatest commit: feca469 The changes in this PR will be included in the next version bump. This PR includes changesets to release 11 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
ntucker
force-pushed
the
benchmark-react-normalization
branch
from
af004f6 to
a2cb25d
Compare
|
Size Change: +16 B (+0.02%) Total Size: 80.5 kB
ℹ️ View Unchanged
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3783 +/- ##
=======================================
Coverage 98.05% 98.06%
=======================================
Files 151 151
Lines 2834 2843 +9
Branches 555 556 +1
=======================================
+ Hits 2779 2788 +9
Misses 11 11
Partials 44 44 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
ntucker
force-pushed
the
benchmark-react-normalization
branch
from
d8d904f to
98de6fa
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Benchmark
Details
| Benchmark suite | Current: feca469 | Previous: a8c10ca | Ratio |
|---|---|---|---|
normalizeLong |
447 ops/sec (±0.96%) |
448 ops/sec (±0.82%) |
1.00 |
normalizeLong Values |
410 ops/sec (±0.31%) |
404 ops/sec (±0.48%) |
0.99 |
denormalizeLong |
285 ops/sec (±3.19%) |
275 ops/sec (±2.97%) |
0.96 |
denormalizeLong Values |
269 ops/sec (±1.92%) |
264 ops/sec (±2.36%) |
0.98 |
denormalizeLong donotcache |
1048 ops/sec (±0.15%) |
1043 ops/sec (±0.15%) |
1.00 |
denormalizeLong Values donotcache |
774 ops/sec (±0.17%) |
759 ops/sec (±0.20%) |
0.98 |
denormalizeShort donotcache 500x |
1592 ops/sec (±0.07%) |
1431 ops/sec (±0.16%) |
0.90 |
denormalizeShort 500x |
862 ops/sec (±1.96%) |
785 ops/sec (±2.10%) |
0.91 |
denormalizeShort 500x withCache |
6223 ops/sec (±0.09%) |
4769 ops/sec (±0.62%) |
0.77 |
queryShort 500x withCache |
2753 ops/sec (±0.06%) |
2611 ops/sec (±0.31%) |
0.95 |
buildQueryKey All |
54557 ops/sec (±0.32%) |
58214 ops/sec (±0.33%) |
1.07 |
query All withCache |
5966 ops/sec (±0.32%) |
5813 ops/sec (±0.25%) |
0.97 |
denormalizeLong with mixin Entity |
282 ops/sec (±2.37%) |
277 ops/sec (±2.02%) |
0.98 |
denormalizeLong withCache |
6892 ops/sec (±0.22%) |
6431 ops/sec (±0.09%) |
0.93 |
denormalizeLong Values withCache |
5134 ops/sec (±0.16%) |
4658 ops/sec (±0.10%) |
0.91 |
denormalizeLong All withCache |
5853 ops/sec (±0.10%) |
5468 ops/sec (±0.09%) |
0.93 |
denormalizeLong Query-sorted withCache |
6020 ops/sec (±0.15%) |
6294 ops/sec (±0.09%) |
1.05 |
denormalizeLongAndShort withEntityCacheOnly |
1681 ops/sec (±0.21%) |
1551 ops/sec (±0.25%) |
0.92 |
getResponse |
4596 ops/sec (±0.85%) |
3770 ops/sec (±0.23%) |
0.82 |
getResponse (null) |
10594084 ops/sec (±1.03%) |
9815415 ops/sec (±0.77%) |
0.93 |
getResponse (clear cache) |
267 ops/sec (±1.92%) |
272 ops/sec (±2.12%) |
1.02 |
getSmallResponse |
3468 ops/sec (±0.14%) |
3231 ops/sec (±0.39%) |
0.93 |
getSmallInferredResponse |
2522 ops/sec (±0.57%) |
2476 ops/sec (±0.21%) |
0.98 |
getResponse Collection |
4631 ops/sec (±0.44%) |
3763 ops/sec (±0.15%) |
0.81 |
get Collection |
4665 ops/sec (±0.20%) |
3539 ops/sec (±0.31%) |
0.76 |
get Query-sorted |
5262 ops/sec (±0.29%) |
5072 ops/sec (±0.11%) |
0.96 |
setLong |
453 ops/sec (±0.27%) |
452 ops/sec (±0.54%) |
1.00 |
setLongWithMerge |
255 ops/sec (±0.22%) |
258 ops/sec (±0.29%) |
1.01 |
setLongWithSimpleMerge |
272 ops/sec (±0.15%) |
273 ops/sec (±0.18%) |
1.00 |
setSmallResponse 500x |
962 ops/sec (±0.13%) |
903 ops/sec (±0.17%) |
0.94 |
This comment was automatically generated by workflow using github-action-benchmark.
ntucker
force-pushed
the
benchmark-react-normalization
branch
from
5ad1b81 to
4df9d01
Compare
ntucker
force-pushed
the
benchmark-react-normalization
branch
2 times, most recently
from
c8bc040 to
081e48d
Compare
- Browser benchmark comparing @data-client/react (Playwright, customSmallerIsBetter). - Scenarios: mount, update entity/author, ref-stability (item/author ref counts). - Hot-path (CI) vs with-network (local): simulated delay for overfetch comparison. - CI workflow runs hot-path only; reports to rhysd/github-action-benchmark. Made-with: Cursor
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| ); | ||
| const margin = 1.96 * (stdDev / Math.sqrt(trimmed.length)); | ||
| return (margin / Math.abs(mean)) * 100 <= targetMarginPct; | ||
| } |
There was a problem hiding this comment.
Convergence check uses mean, not median as documented
Low Severity
The isConverged docstring and the RunProfile.targetMarginPct JSDoc both state convergence is checked against the median, but the implementation computes margin / Math.abs(mean). Since computeStats reports the median as the final value, convergence is assessed against a different central tendency than the one reported. For skewed distributions (common with benchmark outliers), mean and median can diverge significantly, causing premature or delayed convergence relative to what the median-based report actually needs.
Additional Locations (1)
…as HTML Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
There are 4 total unresolved issues (including 2 from previous reviews).
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
1 participant
Motivation
We need a browser-based benchmark for
@data-client/reactthat highlights the performance advantages of normalization and referential equality: when one entity is updated, only components that use that entity get new references; others keep the same object reference so React can skip rerenders. This enables comparison against other React data libraries (TanStack Query, SWR) and surfaces regressions in CI.Solution
examples/benchmark-react: Playwright-driven benchmark that runs a React app modeling GitHub Issues (with shared Users and Labels), measures mount/update duration viaperformance.measure(), and reports incustomSmallerIsBetterformat forrhysd/github-action-benchmark.data-clienttrack regressions; competitor libraries (TanStack Query, SWR, baseline) are benchmarked locally for comparison only.yarn bench:run:compilerbuilds withbabel-plugin-react-compiler(via@anansi/babel-preset'sreactCompileroption) and labels results[compiler]for side-by-side comparison.Collection.moveWith()API: Analogous toaddWith(), constructs a custom move schema. Newunshiftmerge export. Tests, docs, and changeset included.concurrency/permissionsto existing workflows (Node benchmark, bundle size, CodeQL, site preview/release) to reduce overlapping runs.Data model
GitHub-shaped:
Issueentities with nestedUser(assignee) andLabel[]. Multiple issues share the sameUser, so a single user update propagates through normalization — the core advantage data-client demonstrates over non-normalized libraries.Scenarios
Hot path (CI, data-client only)
getlist-100/getlist-500update-single-entityupdate-shared-user-500-mounted/update-shared-user-10000-mountedref-stability-issue-changed/ref-stability-user-changedsorted-view-mount-500/sorted-view-update-entityQueryschema memoization vsuseMemosortlist-detail-switchunshift-item/delete-item/move-iteminvalidate-and-resolvecontroller.invalidate→ Suspense fallback → re-resolve (data-client only)Local comparison only
memory-mount-unmount-cycleAll hot-path scenarios run across all four libraries (data-client, TanStack Query, SWR, baseline) locally, except
invalidate-and-resolvewhich is data-client only.Key design decisions
performance.measure()(JS-driven), React ProfileractualDuration(reconciliation), Chrome trace duration (full rendering pipeline)yarn validate:run): correctness checks for each library's benchmark harnessbench/report-viewer.html): filterable table (base/react-commit/trace), time-series charting via history file loadingreact-windowfor large item countsScripts
yarn bench:runyarn bench:run:compiler[compiler]yarn build:compileryarn bench:compiler[compiler]label onlyyarn bench:dcyarn bench:small/yarn bench:largeyarn validate:runCollection.moveWith()+unshiftNew public API addition in
@data-client/endpoint:Collection.moveWith(merge)— builds a move schema with custom add behavior (parallel toaddWith())unshift— convenience merge function that prepends to arraysChangeset, tests (123 new lines in
Collection.test.ts), docs (docs/rest/api/Collection.md), blog update, and playground type updates included.Made with Cursor