Skip to content

chore: Reduce allocation of Lazy #645

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
He-Pin wants to merge 4 commits into from
+2,694 −826
Closed

chore: Reduce allocation of Lazy #645

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
77827b5
chore: Reduce allocation of Lazy
He-Pin Mar 6, 2026
6e02230
chore: reduce allocation of Num
He-Pin Mar 7, 2026
e47f47f
refactor static optimizer dispatch
He-Pin Mar 7, 2026
52ee6da
add tailrec profiling checkpoint
He-Pin Mar 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
237 changes: 237 additions & 0 deletions bench/AST_VISIT_COUNTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,237 @@
# AST Visit Counts

Generated by `sjsonnet.AstVisitCorpusRunner` on 2026-03-08.

## Corpus

Primary corpus: success-only files from the JVM/native test corpus under `sjsonnet/test/resources/{test_suite,go_test_suite,new_test_suite}`.
Expected-error files were intentionally excluded from counting so the report reflects steady-state successful evaluation paths; JS-only files and the existing `go_test_suite/builtinBase64_string_high_codepoint.jsonnet` skip were also excluded.

### Corpus summary

| Suite | Success files run | Expected-error files skipped | Other skipped |
| --- | ---: | ---: | ---: |
| `test_suite` | 73 | 110 | 0 |
| `go_test_suite` | 489 | 221 | 1 |
| `new_test_suite` | 4 | 4 | 1 |
| **Total** | **566** | **335** | **2** |

### Explicit skips

- `test_suite`: none
- `go_test_suite`: `builtinBase64_string_high_codepoint.jsonnet (excluded to match FileTests corpus)`
- `new_test_suite`: `regex-js.jsonnet (JS-only corpus file)`

### Cross-check

- Every success-only file was executed with both evaluators and required to produce identical successful output.
- Aggregate counter arrays match between evaluators: `true`.
- Old evaluator elapsed time across the corpus: `1070 ms`.
- New evaluator elapsed time across the corpus: `268 ms`.

## Old evaluator summary

- Total `visitExpr` calls: `190469`
- Top old-dispatch arm: `ValidId (73011)`
- Top Expr tag: `ValidId (73011)`
- Top invalid-node case: `$ (0)`

### Old evaluator dispatch-arm counts

| Rank | Name | Count | Share |
| ---: | --- | ---: | ---: |
| 1 | `ValidId` | 73011 | 38.33% |
| 2 | `BinaryOp` | 56856 | 29.85% |
| 3 | `Val` | 37473 | 19.67% |
| 4 | `Select` | 6689 | 3.51% |
| 5 | `Apply1` | 4107 | 2.16% |
| 6 | `Lookup` | 3422 | 1.80% |
| 7 | `IfElse` | 2095 | 1.10% |
| 8 | `ObjExtend` | 2029 | 1.07% |
| 9 | `And` | 830 | 0.44% |
| 10 | `ObjBody.MemberList` | 785 | 0.41% |
| 11 | `Function` | 734 | 0.39% |
| 12 | `ApplyBuiltin1` | 587 | 0.31% |
| 13 | `ApplyBuiltin2` | 508 | 0.27% |
| 14 | `Arr` | 468 | 0.25% |
| 15 | `LocalExpr` | 288 | 0.15% |
| 16 | `Apply2` | 131 | 0.07% |
| 17 | `ObjBody.ObjComp` | 62 | 0.03% |
| 18 | `Slice` | 57 | 0.03% |
| 19 | `Apply` | 54 | 0.03% |
| 20 | `SelectSuper` | 51 | 0.03% |
| 21 | `ApplyBuiltin3` | 46 | 0.02% |
| 22 | `ApplyBuiltin` | 34 | 0.02% |
| 23 | `Apply0` | 30 | 0.02% |
| 24 | `Comp` | 27 | 0.01% |
| 25 | `Import` | 26 | 0.01% |
| 26 | `UnaryOp` | 23 | 0.01% |
| 27 | `Apply3` | 15 | 0.01% |
| 28 | `InSuper` | 9 | 0.00% |
| 29 | `ImportStr` | 8 | 0.00% |
| 30 | `AssertExpr` | 6 | 0.00% |
| 31 | `ImportBin` | 3 | 0.00% |
| 32 | `ApplyBuiltin4` | 2 | 0.00% |
| 33 | `Or` | 2 | 0.00% |
| 34 | `LookupSuper` | 1 | 0.00% |
| 35 | `ApplyBuiltin0` | 0 | 0.00% |
| 36 | `Error` | 0 | 0.00% |
| 37 | `Invalid` | 0 | 0.00% |

### Old evaluator normal Expr tag counts

| Rank | Name | Count | Share |
| ---: | --- | ---: | ---: |
| 1 | `ValidId` | 73011 | 38.33% |
| 2 | `BinaryOp` | 56856 | 29.85% |
| 3 | `Val.Literal` | 37473 | 19.67% |
| 4 | `Select` | 6689 | 3.51% |
| 5 | `Apply1` | 4107 | 2.16% |
| 6 | `Lookup` | 3422 | 1.80% |
| 7 | `IfElse` | 2095 | 1.10% |
| 8 | `ObjExtend` | 2029 | 1.07% |
| 9 | `And` | 830 | 0.44% |
| 10 | `ObjBody.MemberList` | 785 | 0.41% |
| 11 | `Function` | 734 | 0.39% |
| 12 | `ApplyBuiltin1` | 587 | 0.31% |
| 13 | `ApplyBuiltin2` | 508 | 0.27% |
| 14 | `Arr` | 468 | 0.25% |
| 15 | `LocalExpr` | 288 | 0.15% |
| 16 | `Apply2` | 131 | 0.07% |
| 17 | `ObjBody.ObjComp` | 62 | 0.03% |
| 18 | `Slice` | 57 | 0.03% |
| 19 | `Apply` | 54 | 0.03% |
| 20 | `SelectSuper` | 51 | 0.03% |
| 21 | `ApplyBuiltin3` | 46 | 0.02% |
| 22 | `ApplyBuiltin` | 34 | 0.02% |
| 23 | `Apply0` | 30 | 0.02% |
| 24 | `Comp` | 27 | 0.01% |
| 25 | `Import` | 26 | 0.01% |
| 26 | `UnaryOp` | 23 | 0.01% |
| 27 | `Apply3` | 15 | 0.01% |
| 28 | `InSuper` | 9 | 0.00% |
| 29 | `ImportStr` | 8 | 0.00% |
| 30 | `AssertExpr` | 6 | 0.00% |
| 31 | `ImportBin` | 3 | 0.00% |
| 32 | `ApplyBuiltin4` | 2 | 0.00% |
| 33 | `Or` | 2 | 0.00% |
| 34 | `LookupSuper` | 1 | 0.00% |
| 35 | `ApplyBuiltin0` | 0 | 0.00% |
| 36 | `Error` | 0 | 0.00% |
| 37 | `Val.Func` | 0 | 0.00% |

### Old evaluator invalid-node counts

| Rank | Name | Count | Share |
| ---: | --- | ---: | ---: |
| 1 | `$` | 0 | 0.00% |
| 2 | `Id` | 0 | 0.00% |
| 3 | `Other` | 0 | 0.00% |
| 4 | `Self` | 0 | 0.00% |
| 5 | `Super` | 0 | 0.00% |

## New evaluator summary

- Total `visitExpr` calls: `190469`
- Top old-dispatch arm: `ValidId (73011)`
- Top Expr tag: `ValidId (73011)`
- Top invalid-node case: `$ (0)`

### New evaluator dispatch-arm counts

| Rank | Name | Count | Share |
| ---: | --- | ---: | ---: |
| 1 | `ValidId` | 73011 | 38.33% |
| 2 | `BinaryOp` | 56856 | 29.85% |
| 3 | `Val` | 37473 | 19.67% |
| 4 | `Select` | 6689 | 3.51% |
| 5 | `Apply1` | 4107 | 2.16% |
| 6 | `Lookup` | 3422 | 1.80% |
| 7 | `IfElse` | 2095 | 1.10% |
| 8 | `ObjExtend` | 2029 | 1.07% |
| 9 | `And` | 830 | 0.44% |
| 10 | `ObjBody.MemberList` | 785 | 0.41% |
| 11 | `Function` | 734 | 0.39% |
| 12 | `ApplyBuiltin1` | 587 | 0.31% |
| 13 | `ApplyBuiltin2` | 508 | 0.27% |
| 14 | `Arr` | 468 | 0.25% |
| 15 | `LocalExpr` | 288 | 0.15% |
| 16 | `Apply2` | 131 | 0.07% |
| 17 | `ObjBody.ObjComp` | 62 | 0.03% |
| 18 | `Slice` | 57 | 0.03% |
| 19 | `Apply` | 54 | 0.03% |
| 20 | `SelectSuper` | 51 | 0.03% |
| 21 | `ApplyBuiltin3` | 46 | 0.02% |
| 22 | `ApplyBuiltin` | 34 | 0.02% |
| 23 | `Apply0` | 30 | 0.02% |
| 24 | `Comp` | 27 | 0.01% |
| 25 | `Import` | 26 | 0.01% |
| 26 | `UnaryOp` | 23 | 0.01% |
| 27 | `Apply3` | 15 | 0.01% |
| 28 | `InSuper` | 9 | 0.00% |
| 29 | `ImportStr` | 8 | 0.00% |
| 30 | `AssertExpr` | 6 | 0.00% |
| 31 | `ImportBin` | 3 | 0.00% |
| 32 | `ApplyBuiltin4` | 2 | 0.00% |
| 33 | `Or` | 2 | 0.00% |
| 34 | `LookupSuper` | 1 | 0.00% |
| 35 | `ApplyBuiltin0` | 0 | 0.00% |
| 36 | `Error` | 0 | 0.00% |
| 37 | `Invalid` | 0 | 0.00% |

### New evaluator normal Expr tag counts

| Rank | Name | Count | Share |
| ---: | --- | ---: | ---: |
| 1 | `ValidId` | 73011 | 38.33% |
| 2 | `BinaryOp` | 56856 | 29.85% |
| 3 | `Val.Literal` | 37473 | 19.67% |
| 4 | `Select` | 6689 | 3.51% |
| 5 | `Apply1` | 4107 | 2.16% |
| 6 | `Lookup` | 3422 | 1.80% |
| 7 | `IfElse` | 2095 | 1.10% |
| 8 | `ObjExtend` | 2029 | 1.07% |
| 9 | `And` | 830 | 0.44% |
| 10 | `ObjBody.MemberList` | 785 | 0.41% |
| 11 | `Function` | 734 | 0.39% |
| 12 | `ApplyBuiltin1` | 587 | 0.31% |
| 13 | `ApplyBuiltin2` | 508 | 0.27% |
| 14 | `Arr` | 468 | 0.25% |
| 15 | `LocalExpr` | 288 | 0.15% |
| 16 | `Apply2` | 131 | 0.07% |
| 17 | `ObjBody.ObjComp` | 62 | 0.03% |
| 18 | `Slice` | 57 | 0.03% |
| 19 | `Apply` | 54 | 0.03% |
| 20 | `SelectSuper` | 51 | 0.03% |
| 21 | `ApplyBuiltin3` | 46 | 0.02% |
| 22 | `ApplyBuiltin` | 34 | 0.02% |
| 23 | `Apply0` | 30 | 0.02% |
| 24 | `Comp` | 27 | 0.01% |
| 25 | `Import` | 26 | 0.01% |
| 26 | `UnaryOp` | 23 | 0.01% |
| 27 | `Apply3` | 15 | 0.01% |
| 28 | `InSuper` | 9 | 0.00% |
| 29 | `ImportStr` | 8 | 0.00% |
| 30 | `AssertExpr` | 6 | 0.00% |
| 31 | `ImportBin` | 3 | 0.00% |
| 32 | `ApplyBuiltin4` | 2 | 0.00% |
| 33 | `Or` | 2 | 0.00% |
| 34 | `LookupSuper` | 1 | 0.00% |
| 35 | `ApplyBuiltin0` | 0 | 0.00% |
| 36 | `Error` | 0 | 0.00% |
| 37 | `Val.Func` | 0 | 0.00% |

### New evaluator invalid-node counts

| Rank | Name | Count | Share |
| ---: | --- | ---: | ---: |
| 1 | `$` | 0 | 0.00% |
| 2 | `Id` | 0 | 0.00% |
| 3 | `Other` | 0 | 0.00% |
| 4 | `Self` | 0 | 0.00% |
| 5 | `Super` | 0 | 0.00% |

## Reproduction

- Command: `./mill sjsonnet.jvm[3.3.7].runMain sjsonnet.AstVisitCorpusRunner /Users/hepin/IdeaProjects/sjsonnet/bench/AST_VISIT_COUNTS.md`
- Notes: this todo only adds instrumentation, corpus execution, and markdown output. It intentionally does **not** reorder evaluator dispatch or renumber `Expr.tag` values yet.
36 changes: 36 additions & 0 deletions bench/OPTIMIZATION_LOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Optimization Log

## Wave 1: direct self-tailrec detection
- Scope: mark direct self-tail-calls in `StaticOptimizer` without requiring explicit `tailstrict`.
- Outcome: kept.
- Validation:
- `./mill 'sjsonnet.jvm[3.3.7]'.test.testOnly sjsonnet.TailCallOptimizationTests sjsonnet.EvaluatorTests sjsonnet.AstVisitProfilerTests`
- `./mill 'sjsonnet.jvm[3.3.7]'.test`
- `./mill bench.runJmh -i 1 -wi 1 -f 1 'sjsonnet.bench.OptimizerBenchmark.main'`
- Notes:
- semantic feature is correct and verified
- optimizer benchmark stayed effectively flat (`~0.552-0.564 ms/op` across short JMH runs)

## Wave 2: AST visit profiler + corpus runner
- Scope: instrument both evaluators, run the success-path corpus, and persist measured counts.
- Outcome: kept.
- Artifacts:
- `bench/AST_VISIT_COUNTS.md`
- `sjsonnet/src/sjsonnet/AstVisitProfiler.scala`
- `sjsonnet/src-jvm-native/sjsonnet/AstVisitCorpusRunner.scala`
- Key corpus findings:
- hottest old-dispatch arms: `ValidId`, `BinaryOp`, `Val`, `Select`
- hottest normal tags: `ValidId`, `BinaryOp`, `Val.Literal`, `Select`

## Wave 3: data-driven dispatch/tag reordering
- Scope: reorder old `Evaluator.visitExpr` pattern matches and renumber `Expr.tag` values from the measured corpus.
- Outcome: reverted.
- Reason:
- benchmark results regressed despite the frequency-guided order
- `Select`/`Self`/`$`/`Super` share low-tag fast paths in `StaticOptimizer`, which constrains safe tag reshuffling
- the naive reorder appears to fight Scala/JVM codegen rather than help it
- Rejected measurements:
- `MainBenchmark.main`: `2.788 ms/op -> 3.222 ms/op`
- `OptimizerBenchmark.main`: `0.555 ms/op -> 0.569 ms/op`
- corpus runner new evaluator time: `235 ms -> 261 ms`
- Resolution: restore the original dispatch/tag order and keep the profiler data for later, more targeted optimizations.
2 changes: 1 addition & 1 deletion bench/resources/refresh_golden_outputs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ for suite in bench/resources/*_suite; do
echo "Refreshing golden outputs for suite: $suite_name"
for f in "$suite"/*.jsonnet; do
echo " Processing file: $f"
java -Xss100m -Xmx2g -jar "$SJSONNET" -J "$suite" "$f" > "$f.golden"
java -Xss100m -Xmx2g -jar "$SJSONNET" --max-stack 100000 -J "$suite" "$f" > "$f.golden"
done
done

Expand Down
Loading
Loading