From d1da69ac2d1b5e5281dc5230ee3e601cb598be16 Mon Sep 17 00:00:00 2001 From: Stan Lo Date: Thu, 12 Mar 2026 20:03:44 +0000 Subject: [PATCH 1/7] Add autoresearch setup for test duration optimization --- autoresearch.md | 67 +++++++++++++++++++++++++++++++++++++++++++++++++ autoresearch.sh | 25 ++++++++++++++++++ 2 files changed, 92 insertions(+) create mode 100644 autoresearch.md create mode 100755 autoresearch.sh diff --git a/autoresearch.md b/autoresearch.md new file mode 100644 index 000000000..5aa531ec2 --- /dev/null +++ b/autoresearch.md @@ -0,0 +1,67 @@ +# Autoresearch: Ruby LSP Server Test Duration + +## Objective +Reduce the wall-clock time of `bundle exec rake test` (the server-level test suite, ~18,329 tests). +Baseline: **~490 seconds** on macOS with Ruby 3.4.7. + +The test suite uses Minitest. The Rakefile's `test` task runs `test/**/*_test.rb` excluding `test/fixtures/prism/**/*`. + +## Metrics +- **Primary**: `test_duration` (seconds, lower is better) — wall-clock time reported by Minitest's "Finished in X seconds" +- **Secondary**: `test_count` (must stay >= 18329), `failures` (must stay <= 2, the 2 pre-existing nix-path failures), `errors` (must stay 0) + +## How to Run +```bash +source /opt/homebrew/share/chruby/chruby.sh && chruby ruby-3.4.7 && ./autoresearch.sh +``` +Outputs `METRIC name=number` lines parsed by the experiment tooling. + +## Profiling Baseline (Top Slow Classes) + +| Time (s) | Class | Tests | Avg (s) | Root Cause | +|----------|-------|-------|---------|------------| +| 149.84 | SetupBundlerTest | 41 | 3.7 | `bundle install` subprocess per test | +| 104.51 | FormattingExpectationsTest | 1176 | 0.09 | GlobalState + RuboCop init per test | +| 103.18 | DiagnosticsExpectationsTest | 1176 | 0.09 | GlobalState + RuboCop init per test | +| 43.56 | IntegrationTest | 18 | 2.4 | Full process spawn + bundle install | +| 18.50 | DefinitionExpectationsTest | 1221 | 0.015 | GlobalState per test | +| 17.10 | ServerTest | 59 | 0.29 | Server + threads per test | +| 8.84 | HoverExpectationsTest | 1210 | 0.007 | GlobalState per test | +| 5.64 | CommonTest | 1 | 5.64 | Entry kind iteration | + +## Files in Scope + +### Test infrastructure (primary targets) +- `test/test_helper.rb` — test helper loaded by all tests +- `lib/ruby_lsp/test_helper.rb` — `with_server` helper, creates Server per call +- `test/requests/support/expectations_test_runner.rb` — generates expectation tests, creates GlobalState per test + +### Slow test files +- `test/setup_bundler_test.rb` — 150s, runs `bundle install` in subprocess +- `test/integration_test.rb` — 44s, spawns full LSP processes +- `test/server_test.rb` — 17s, creates Server per test + +### Core objects created per-test +- `lib/ruby_lsp/global_state.rb` — GlobalState with Index, TypeInferrer, etc. +- `lib/ruby_lsp/server.rb` — Server with threads, store, global state +- `lib/ruby_indexer/lib/ruby_indexer/index.rb` — Index with prefix trees + +## Off Limits +- **Do NOT change test semantics** — tests must still verify the same behavior +- **Do NOT delete or skip tests** — test_count must remain >= 18329 +- **Do NOT modify production (non-test) code** to optimize tests (except if adding test-only hooks) +- **Do NOT add new gem dependencies** +- `test/fixtures/` — fixture files must not be modified +- `test/expectations/` — expectation JSON files must not be modified +- `lib/ruby_lsp/test_helper.rb` — public API used by addon authors, changes must be backward-compatible +- Integration tests and SetupBundlerTest — these intentionally test subprocess behavior + +## Constraints +- All 18,329+ tests must pass (excluding 2 pre-existing nix-path failures) +- No new test failures or errors +- Changes must be safe for CI (no machine-specific hacks) +- Ruby version: 3.4.7 via chruby +- Activation: `source /opt/homebrew/share/chruby/chruby.sh && chruby ruby-3.4.7` + +## What's Been Tried +(Nothing yet — this is the initial baseline.) diff --git a/autoresearch.sh b/autoresearch.sh new file mode 100755 index 000000000..f0df7bb9c --- /dev/null +++ b/autoresearch.sh @@ -0,0 +1,25 @@ +#!/bin/bash +set -euo pipefail + +# Quick pre-check: ensure test files exist +if [ ! -f "test/test_helper.rb" ]; then + echo "ERROR: Not in ruby-lsp root directory" + exit 1 +fi + +# Run the server test suite (same as `rake test`) +output=$(bundle exec rake test 2>&1) || true + +# Extract metrics from minitest output +# Format: "Finished in 491.580254s, 37.2859 runs/s, 124.6978 assertions/s." +duration=$(echo "$output" | grep -oP 'Finished in \K[0-9.]+(?=s)') +# Format: "18329 runs, 61299 assertions, 2 failures, 0 errors, 14 skips" +stats_line=$(echo "$output" | grep -E '^[0-9]+ runs,') +test_count=$(echo "$stats_line" | grep -oP '^[0-9]+') +failures=$(echo "$stats_line" | grep -oP '[0-9]+(?= failures)') +errors=$(echo "$stats_line" | grep -oP '[0-9]+(?= errors)') + +echo "METRIC test_duration=${duration}" +echo "METRIC test_count=${test_count}" +echo "METRIC failures=${failures}" +echo "METRIC errors=${errors}" From e7a7b8913f0e90015cc92e1ed4050dcdf480d5e8 Mon Sep 17 00:00:00 2001 From: Stan Lo Date: Thu, 12 Mar 2026 20:26:36 +0000 Subject: [PATCH 2/7] Cache GlobalState and RuboCopFormatter in expectation tests Reduces test_duration from ~454s to ~301s (33% improvement) --- test/requests/diagnostics_expectations_test.rb | 8 +++++++- test/requests/formatting_expectations_test.rb | 8 +++++++- test/requests/support/expectations_test_runner.rb | 6 +++++- 3 files changed, 19 insertions(+), 3 deletions(-) diff --git a/test/requests/diagnostics_expectations_test.rb b/test/requests/diagnostics_expectations_test.rb index b8c4663dd..fd049728c 100644 --- a/test/requests/diagnostics_expectations_test.rb +++ b/test/requests/diagnostics_expectations_test.rb @@ -14,7 +14,7 @@ def run_expectations(source) }) @global_state.register_formatter( "rubocop_internal", - RubyLsp::Requests::Support::RuboCopFormatter.new, + self.class.cached_rubocop_formatter, ) document = RubyLsp::RubyDocument.new( @@ -81,4 +81,10 @@ def map_diagnostics(diagnostics) ) end.to_json end + + class << self + def cached_rubocop_formatter + @cached_rubocop_formatter ||= RubyLsp::Requests::Support::RuboCopFormatter.new + end + end end diff --git a/test/requests/formatting_expectations_test.rb b/test/requests/formatting_expectations_test.rb index aadab9d41..7516f6ad9 100644 --- a/test/requests/formatting_expectations_test.rb +++ b/test/requests/formatting_expectations_test.rb @@ -11,7 +11,7 @@ def run_expectations(source) @global_state.formatter = "rubocop_internal" @global_state.register_formatter( "rubocop_internal", - RubyLsp::Requests::Support::RuboCopFormatter.new, + self.class.cached_rubocop_formatter, ) document = RubyLsp::RubyDocument.new( source: source, @@ -37,4 +37,10 @@ def assert_expectations(source, expected) def initialize_params(_expected) end + + class << self + def cached_rubocop_formatter + @cached_rubocop_formatter ||= RubyLsp::Requests::Support::RuboCopFormatter.new + end + end end diff --git a/test/requests/support/expectations_test_runner.rb b/test/requests/support/expectations_test_runner.rb index 1c64c7275..e63fb3180 100644 --- a/test/requests/support/expectations_test_runner.rb +++ b/test/requests/support/expectations_test_runner.rb @@ -8,10 +8,14 @@ class ExpectationsTestRunner < Minitest::Test TEST_PRISM_FIXTURES = File.join(TEST_FIXTURES_DIR, "prism/test/prism/fixtures/**", "*.txt") def setup - @global_state = RubyLsp::GlobalState.new + @global_state = self.class.shared_global_state end class << self + def shared_global_state + @shared_global_state ||= RubyLsp::GlobalState.new + end + def expectations_tests(handler_class, expectation_suffix) class_eval(<<~RB, __FILE__, __LINE__ + 1) module ExpectationsRunnerMethods From 0e60e2fbaf86866199a3b4389b191fdd22a79085 Mon Sep 17 00:00:00 2001 From: Stan Lo Date: Thu, 12 Mar 2026 20:38:33 +0000 Subject: [PATCH 3/7] Skip addon loading in Definition and Hover expectation tests Addon loading (especially RuboCop addon) costs ~103ms per test. Generated expectation tests don't need addons, only the hand-written tests that explicitly test addon behavior do. Reduces test_duration from ~301s to ~217s (28% improvement). --- test/requests/definition_expectations_test.rb | 2 +- test/requests/hover_expectations_test.rb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/test/requests/definition_expectations_test.rb b/test/requests/definition_expectations_test.rb index e3f005f08..285253fc4 100644 --- a/test/requests/definition_expectations_test.rb +++ b/test/requests/definition_expectations_test.rb @@ -9,7 +9,7 @@ class DefinitionExpectationsTest < ExpectationsTestRunner def run_expectations(source) # We need to pretend that Sorbet is not a dependency or else we can't properly test - with_server(source, stub_no_typechecker: true) do |server, uri| + with_server(source, stub_no_typechecker: true, load_addons: false) do |server, uri| position = @__params&.first || { character: 0, line: 0 } index = server.global_state.index diff --git a/test/requests/hover_expectations_test.rb b/test/requests/hover_expectations_test.rb index 5b9144c69..cb9881940 100644 --- a/test/requests/hover_expectations_test.rb +++ b/test/requests/hover_expectations_test.rb @@ -10,7 +10,7 @@ class HoverExpectationsTest < ExpectationsTestRunner def run_expectations(source) position = @__params&.first || { character: 0, line: 0 } - with_server(source, stub_no_typechecker: true) do |server, uri| + with_server(source, stub_no_typechecker: true, load_addons: false) do |server, uri| # We need to pretend that Sorbet is not a dependency or else we can't properly test server.process_message( id: 1, From 291bd94e6d7cdfeec30cc367ec48de321049b6dd Mon Sep 17 00:00:00 2001 From: Stan Lo Date: Thu, 12 Mar 2026 21:15:35 +0000 Subject: [PATCH 4/7] Default load_addons: false for Definition and Hover test classes Most hand-written tests in these classes don't need addons. Only test_definition_addons and test_hover_addons explicitly enable addon loading. Saves ~3s from avoiding ~79 addon activation cycles. --- autoresearch.md | 34 ++++++++++++++++++- test/requests/definition_expectations_test.rb | 14 ++++++-- test/requests/hover_expectations_test.rb | 14 ++++++-- 3 files changed, 57 insertions(+), 5 deletions(-) diff --git a/autoresearch.md b/autoresearch.md index 5aa531ec2..cbac600d5 100644 --- a/autoresearch.md +++ b/autoresearch.md @@ -64,4 +64,36 @@ Outputs `METRIC name=number` lines parsed by the experiment tooling. - Activation: `source /opt/homebrew/share/chruby/chruby.sh && chruby ruby-3.4.7` ## What's Been Tried -(Nothing yet — this is the initial baseline.) + +### Experiment 1: Cache GlobalState at class level (KEPT) +- Changed `ExpectationsTestRunner#setup` to share GlobalState across tests in same class +- **Impact alone**: negligible (~0s saved) — GlobalState.new is only ~2ms +- But enables experiment 1b below + +### Experiment 1b: Cache RuboCopFormatter at class level (KEPT) +- `FormattingExpectationsTest` and `DiagnosticsExpectationsTest` were creating new `RuboCopFormatter` (which creates 2 `RuboCopRunner` instances each) for every test +- Cached at class level since runners are designed for reuse +- **Impact**: 454s → 301s (**153s saved, 33.8%**) + +### Experiment 2: Skip addon loading in expectation tests (KEPT) +- `with_server(load_addons: true)` costs ~103ms per call vs ~2ms without +- RuboCop addon activation/deactivation per test was the main cost +- Definition and Hover expectation `run_expectations` don't need addons +- **Impact**: 301s → 217s (**84s saved, 27.9%**) + +### Micro-benchmarks (informational) +- Server.new + shutdown: 1.72ms (fast) +- GlobalState.new: 2.1ms (fast) +- with_server(load_addons: true): 104.84ms (slow due to addon loading) +- with_server(load_addons: false): 2.19ms (fast) +- RuboCopFormatter.new: ~50ms (2 RuboCopRunner creations) +- Bundler.default_gemfile: 1.66ms (not a bottleneck) + +### Current bottlenecks (post-optimization) +- SetupBundlerTest: 102s (off-limits, subprocess `bundle install`) +- IntegrationTest: 40s (off-limits, subprocess spawning) +- FormattingExpectationsTest: 19s (actual RuboCop work on 1176 fixtures) +- ServerTest: 17s (59 tests testing server message processing) +- DefinitionExpectationsTest: 15s (with_server per test, already skip addons) +- DiagnosticsExpectationsTest: 8s (actual RuboCop work) +- CommonTest: 4.5s (indexes entire workspace in 1 test) diff --git a/test/requests/definition_expectations_test.rb b/test/requests/definition_expectations_test.rb index 285253fc4..45618d4a7 100644 --- a/test/requests/definition_expectations_test.rb +++ b/test/requests/definition_expectations_test.rb @@ -7,9 +7,19 @@ class DefinitionExpectationsTest < ExpectationsTestRunner expectations_tests RubyLsp::Requests::Definition, "definition" + private + + # Skip addon loading by default — only test_definition_addons needs addons + def with_server(source = nil, uri = Kernel.URI("file:///fake.rb"), stub_no_typechecker: false, load_addons: false, + &block) + super + end + + public + def run_expectations(source) # We need to pretend that Sorbet is not a dependency or else we can't properly test - with_server(source, stub_no_typechecker: true, load_addons: false) do |server, uri| + with_server(source, stub_no_typechecker: true) do |server, uri| position = @__params&.first || { character: 0, line: 0 } index = server.global_state.index @@ -229,7 +239,7 @@ def test_definition_addons begin create_definition_addon - with_server(source, stub_no_typechecker: true) do |server, uri| + with_server(source, stub_no_typechecker: true, load_addons: true) do |server, uri| server.global_state.index.index_file( URI::Generic.from_path( load_path_entry: "#{Dir.pwd}/lib", diff --git a/test/requests/hover_expectations_test.rb b/test/requests/hover_expectations_test.rb index cb9881940..d03508297 100644 --- a/test/requests/hover_expectations_test.rb +++ b/test/requests/hover_expectations_test.rb @@ -7,10 +7,20 @@ class HoverExpectationsTest < ExpectationsTestRunner expectations_tests RubyLsp::Requests::Hover, "hover" + private + + # Skip addon loading by default — only test_hover_addons needs addons + def with_server(source = nil, uri = Kernel.URI("file:///fake.rb"), stub_no_typechecker: false, load_addons: false, + &block) + super + end + + public + def run_expectations(source) position = @__params&.first || { character: 0, line: 0 } - with_server(source, stub_no_typechecker: true, load_addons: false) do |server, uri| + with_server(source, stub_no_typechecker: true) do |server, uri| # We need to pretend that Sorbet is not a dependency or else we can't properly test server.process_message( id: 1, @@ -356,7 +366,7 @@ class Post begin create_hover_addon - with_server(source, stub_no_typechecker: true) do |server, uri| + with_server(source, stub_no_typechecker: true, load_addons: true) do |server, uri| server.process_message( id: 1, method: "textDocument/hover", From cfa5115419daa1dfc5e99359705ead626c12aa7e Mon Sep 17 00:00:00 2001 From: Stan Lo Date: Thu, 12 Mar 2026 21:15:57 +0000 Subject: [PATCH 5/7] Update autoresearch.md with latest findings --- autoresearch.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/autoresearch.md b/autoresearch.md index cbac600d5..5c8315ea0 100644 --- a/autoresearch.md +++ b/autoresearch.md @@ -89,11 +89,24 @@ Outputs `METRIC name=number` lines parsed by the experiment tooling. - RuboCopFormatter.new: ~50ms (2 RuboCopRunner creations) - Bundler.default_gemfile: 1.66ms (not a bottleneck) +### Experiment 3: Default load_addons: false for Definition and Hover classes (KEPT) +- Override `with_server` default to `load_addons: false` in these two test classes +- Saves ~3s from ~79 fewer addon activation/deactivation cycles +- Only test_definition_addons and test_hover_addons explicitly enable addon loading +- **Impact**: ~3s (hard to measure due to run-to-run variance) + ### Current bottlenecks (post-optimization) - SetupBundlerTest: 102s (off-limits, subprocess `bundle install`) - IntegrationTest: 40s (off-limits, subprocess spawning) - FormattingExpectationsTest: 19s (actual RuboCop work on 1176 fixtures) - ServerTest: 17s (59 tests testing server message processing) -- DefinitionExpectationsTest: 15s (with_server per test, already skip addons) +- DefinitionExpectationsTest: ~12s (with_server per test, now skip addons) - DiagnosticsExpectationsTest: 8s (actual RuboCop work) - CommonTest: 4.5s (indexes entire workspace in 1 test) + +### Ideas considered but not pursued +- **GC tuning (RUBY_GC_HEAP_INIT_SLOTS)**: Made things slower (297s vs 217-243s) +- **Minitest parallelize_me!**: Thread safety concerns with shared mutable state (Addon.addons, RuboCop runners) +- **Caching Bundler lookups**: Only 1.66ms per call, not a bottleneck +- **Caching indexed files per DefinitionExpectationsTest**: Would need Server cloning, too invasive +- **Changing with_server default to load_addons: false globally**: Would break public API for addon authors From 40de14cf31b5a7eb848376f0a1271c97d504dad1 Mon Sep 17 00:00:00 2001 From: Stan Lo Date: Thu, 12 Mar 2026 21:48:05 +0000 Subject: [PATCH 6/7] Remove autoresearch scaffolding files --- autoresearch.md | 112 ------------------------------------------------ autoresearch.sh | 25 ----------- 2 files changed, 137 deletions(-) delete mode 100644 autoresearch.md delete mode 100755 autoresearch.sh diff --git a/autoresearch.md b/autoresearch.md deleted file mode 100644 index 5c8315ea0..000000000 --- a/autoresearch.md +++ /dev/null @@ -1,112 +0,0 @@ -# Autoresearch: Ruby LSP Server Test Duration - -## Objective -Reduce the wall-clock time of `bundle exec rake test` (the server-level test suite, ~18,329 tests). -Baseline: **~490 seconds** on macOS with Ruby 3.4.7. - -The test suite uses Minitest. The Rakefile's `test` task runs `test/**/*_test.rb` excluding `test/fixtures/prism/**/*`. - -## Metrics -- **Primary**: `test_duration` (seconds, lower is better) — wall-clock time reported by Minitest's "Finished in X seconds" -- **Secondary**: `test_count` (must stay >= 18329), `failures` (must stay <= 2, the 2 pre-existing nix-path failures), `errors` (must stay 0) - -## How to Run -```bash -source /opt/homebrew/share/chruby/chruby.sh && chruby ruby-3.4.7 && ./autoresearch.sh -``` -Outputs `METRIC name=number` lines parsed by the experiment tooling. - -## Profiling Baseline (Top Slow Classes) - -| Time (s) | Class | Tests | Avg (s) | Root Cause | -|----------|-------|-------|---------|------------| -| 149.84 | SetupBundlerTest | 41 | 3.7 | `bundle install` subprocess per test | -| 104.51 | FormattingExpectationsTest | 1176 | 0.09 | GlobalState + RuboCop init per test | -| 103.18 | DiagnosticsExpectationsTest | 1176 | 0.09 | GlobalState + RuboCop init per test | -| 43.56 | IntegrationTest | 18 | 2.4 | Full process spawn + bundle install | -| 18.50 | DefinitionExpectationsTest | 1221 | 0.015 | GlobalState per test | -| 17.10 | ServerTest | 59 | 0.29 | Server + threads per test | -| 8.84 | HoverExpectationsTest | 1210 | 0.007 | GlobalState per test | -| 5.64 | CommonTest | 1 | 5.64 | Entry kind iteration | - -## Files in Scope - -### Test infrastructure (primary targets) -- `test/test_helper.rb` — test helper loaded by all tests -- `lib/ruby_lsp/test_helper.rb` — `with_server` helper, creates Server per call -- `test/requests/support/expectations_test_runner.rb` — generates expectation tests, creates GlobalState per test - -### Slow test files -- `test/setup_bundler_test.rb` — 150s, runs `bundle install` in subprocess -- `test/integration_test.rb` — 44s, spawns full LSP processes -- `test/server_test.rb` — 17s, creates Server per test - -### Core objects created per-test -- `lib/ruby_lsp/global_state.rb` — GlobalState with Index, TypeInferrer, etc. -- `lib/ruby_lsp/server.rb` — Server with threads, store, global state -- `lib/ruby_indexer/lib/ruby_indexer/index.rb` — Index with prefix trees - -## Off Limits -- **Do NOT change test semantics** — tests must still verify the same behavior -- **Do NOT delete or skip tests** — test_count must remain >= 18329 -- **Do NOT modify production (non-test) code** to optimize tests (except if adding test-only hooks) -- **Do NOT add new gem dependencies** -- `test/fixtures/` — fixture files must not be modified -- `test/expectations/` — expectation JSON files must not be modified -- `lib/ruby_lsp/test_helper.rb` — public API used by addon authors, changes must be backward-compatible -- Integration tests and SetupBundlerTest — these intentionally test subprocess behavior - -## Constraints -- All 18,329+ tests must pass (excluding 2 pre-existing nix-path failures) -- No new test failures or errors -- Changes must be safe for CI (no machine-specific hacks) -- Ruby version: 3.4.7 via chruby -- Activation: `source /opt/homebrew/share/chruby/chruby.sh && chruby ruby-3.4.7` - -## What's Been Tried - -### Experiment 1: Cache GlobalState at class level (KEPT) -- Changed `ExpectationsTestRunner#setup` to share GlobalState across tests in same class -- **Impact alone**: negligible (~0s saved) — GlobalState.new is only ~2ms -- But enables experiment 1b below - -### Experiment 1b: Cache RuboCopFormatter at class level (KEPT) -- `FormattingExpectationsTest` and `DiagnosticsExpectationsTest` were creating new `RuboCopFormatter` (which creates 2 `RuboCopRunner` instances each) for every test -- Cached at class level since runners are designed for reuse -- **Impact**: 454s → 301s (**153s saved, 33.8%**) - -### Experiment 2: Skip addon loading in expectation tests (KEPT) -- `with_server(load_addons: true)` costs ~103ms per call vs ~2ms without -- RuboCop addon activation/deactivation per test was the main cost -- Definition and Hover expectation `run_expectations` don't need addons -- **Impact**: 301s → 217s (**84s saved, 27.9%**) - -### Micro-benchmarks (informational) -- Server.new + shutdown: 1.72ms (fast) -- GlobalState.new: 2.1ms (fast) -- with_server(load_addons: true): 104.84ms (slow due to addon loading) -- with_server(load_addons: false): 2.19ms (fast) -- RuboCopFormatter.new: ~50ms (2 RuboCopRunner creations) -- Bundler.default_gemfile: 1.66ms (not a bottleneck) - -### Experiment 3: Default load_addons: false for Definition and Hover classes (KEPT) -- Override `with_server` default to `load_addons: false` in these two test classes -- Saves ~3s from ~79 fewer addon activation/deactivation cycles -- Only test_definition_addons and test_hover_addons explicitly enable addon loading -- **Impact**: ~3s (hard to measure due to run-to-run variance) - -### Current bottlenecks (post-optimization) -- SetupBundlerTest: 102s (off-limits, subprocess `bundle install`) -- IntegrationTest: 40s (off-limits, subprocess spawning) -- FormattingExpectationsTest: 19s (actual RuboCop work on 1176 fixtures) -- ServerTest: 17s (59 tests testing server message processing) -- DefinitionExpectationsTest: ~12s (with_server per test, now skip addons) -- DiagnosticsExpectationsTest: 8s (actual RuboCop work) -- CommonTest: 4.5s (indexes entire workspace in 1 test) - -### Ideas considered but not pursued -- **GC tuning (RUBY_GC_HEAP_INIT_SLOTS)**: Made things slower (297s vs 217-243s) -- **Minitest parallelize_me!**: Thread safety concerns with shared mutable state (Addon.addons, RuboCop runners) -- **Caching Bundler lookups**: Only 1.66ms per call, not a bottleneck -- **Caching indexed files per DefinitionExpectationsTest**: Would need Server cloning, too invasive -- **Changing with_server default to load_addons: false globally**: Would break public API for addon authors diff --git a/autoresearch.sh b/autoresearch.sh deleted file mode 100755 index f0df7bb9c..000000000 --- a/autoresearch.sh +++ /dev/null @@ -1,25 +0,0 @@ -#!/bin/bash -set -euo pipefail - -# Quick pre-check: ensure test files exist -if [ ! -f "test/test_helper.rb" ]; then - echo "ERROR: Not in ruby-lsp root directory" - exit 1 -fi - -# Run the server test suite (same as `rake test`) -output=$(bundle exec rake test 2>&1) || true - -# Extract metrics from minitest output -# Format: "Finished in 491.580254s, 37.2859 runs/s, 124.6978 assertions/s." -duration=$(echo "$output" | grep -oP 'Finished in \K[0-9.]+(?=s)') -# Format: "18329 runs, 61299 assertions, 2 failures, 0 errors, 14 skips" -stats_line=$(echo "$output" | grep -E '^[0-9]+ runs,') -test_count=$(echo "$stats_line" | grep -oP '^[0-9]+') -failures=$(echo "$stats_line" | grep -oP '[0-9]+(?= failures)') -errors=$(echo "$stats_line" | grep -oP '[0-9]+(?= errors)') - -echo "METRIC test_duration=${duration}" -echo "METRIC test_count=${test_count}" -echo "METRIC failures=${failures}" -echo "METRIC errors=${errors}" From 14541739a94b7566245c8e140fdf4d3bf2753cf0 Mon Sep 17 00:00:00 2001 From: Stan Lo Date: Thu, 12 Mar 2026 21:51:24 +0000 Subject: [PATCH 7/7] Fix cspell: use 'add-on' in comments instead of 'addon' --- test/requests/definition_expectations_test.rb | 6 +----- test/requests/hover_expectations_test.rb | 6 +----- 2 files changed, 2 insertions(+), 10 deletions(-) diff --git a/test/requests/definition_expectations_test.rb b/test/requests/definition_expectations_test.rb index 45618d4a7..5845300aa 100644 --- a/test/requests/definition_expectations_test.rb +++ b/test/requests/definition_expectations_test.rb @@ -7,16 +7,12 @@ class DefinitionExpectationsTest < ExpectationsTestRunner expectations_tests RubyLsp::Requests::Definition, "definition" - private - - # Skip addon loading by default — only test_definition_addons needs addons + # Skip add-on loading by default — only test_definition_addons needs it def with_server(source = nil, uri = Kernel.URI("file:///fake.rb"), stub_no_typechecker: false, load_addons: false, &block) super end - public - def run_expectations(source) # We need to pretend that Sorbet is not a dependency or else we can't properly test with_server(source, stub_no_typechecker: true) do |server, uri| diff --git a/test/requests/hover_expectations_test.rb b/test/requests/hover_expectations_test.rb index d03508297..ad80a6541 100644 --- a/test/requests/hover_expectations_test.rb +++ b/test/requests/hover_expectations_test.rb @@ -7,16 +7,12 @@ class HoverExpectationsTest < ExpectationsTestRunner expectations_tests RubyLsp::Requests::Hover, "hover" - private - - # Skip addon loading by default — only test_hover_addons needs addons + # Skip add-on loading by default — only test_hover_addons needs it def with_server(source = nil, uri = Kernel.URI("file:///fake.rb"), stub_no_typechecker: false, load_addons: false, &block) super end - public - def run_expectations(source) position = @__params&.first || { character: 0, line: 0 }