Skip to content

feat(profile): vernier wrapper modes + .vernier.json analyzer#622

Closed
ryanseys wants to merge 1 commit into
matz:masterfrom
ryanseys:rs-perf-infra
Closed

feat(profile): vernier wrapper modes + .vernier.json analyzer#622
ryanseys wants to merge 1 commit into
matz:masterfrom
ryanseys:rs-perf-infra

Conversation

@ryanseys
Copy link
Copy Markdown
Contributor

@ryanseys ryanseys commented May 20, 2026

deleted.

Extends the CRuby-only `spinel_codegen_profile.rb` wrapper to expose
all of Vernier's modes (wall + retained), and adds a stdlib-only
`bin/analyze_profile.rb` that flattens `.vernier.json` profiles into
a markdown report without needing `profile-viewer` or the
profiler.firefox.com UI. Three Makefile targets stitch the two
together for the common self-compile measurement loop.

Wrapper (`spinel_codegen_profile.rb`):

* `--mode wall|retained` selects sampling vs Vernier.trace_retained
* `--interval <us>`, `--alloc <n>`, `--hooks <list>` expose the
  wall-mode tuning surface (defaults: 100us / 10 / memory_usage)
* When `--profile <path>` is omitted, derives a path under
  `build/profiles/<ast-basename>.<mode>.vernier.json`
* Prints the absolute path on a single line to STDOUT so callers can
  capture it with \`out=\$(ruby spinel_codegen_profile.rb ...)\`

Analyzer (`bin/analyze_profile.rb`):

* Walks the Firefox-Profiler-format stackTable + funcTable to charge
  self time, total time, allocation count, and retained bytes per
  function
* Emits top-N tables (self/total/alloc/retained) plus a text
  sparkline for the memory_usage counter
* \`--diff baseline.json after.json\` mode produces a function-level
  delta table for A/B comparison between runs
* Pure stdlib (json + zlib); no gem dependency

Makefile:

* \`prof-self\` -- wall+alloc profile of codegen self-compile, runs
  the analyzer, writes \`build/profiles/codegen-self.report.md\`
* \`prof-self-retained\` -- same in retained-memory mode
* \`prof-self-deep\` -- runs both and concatenates the reports

Refs #100 (sibling PR that adds --stats=csv/json to spinel_codegen).

Verification: \`make prof-self\` on a clean worktree completes in
~18 s on the codegen self-compile, captures 88680 samples, and the
report identifies top self-time frames (String#split 45.8%,
push_child_ids 5.8%, parse_id_list 5.4%) plus top alloc sites
(set_ref_field 5.6%, alloc_node 4.6%) plus the 757 MiB peak RSS
curve. No changes to spinel sources -- bootstrap fixpoint,
\`make test\`, \`make optcarrot\` unaffected by construction.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@ryanseys ryanseys closed this May 20, 2026
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a profiling suite for spinel_codegen.rb, including a Makefile integration, a Vernier wrapper script, and a profile analyzer that generates markdown reports. The reviewer identified several issues, such as potential crashes in CLI argument parsing due to missing bounds checks, an unused variable in a performance-sensitive loop, and a brittle method for extracting help text from the source file.

Comment thread bin/analyze_profile.rb
Comment on lines +93 to +99
seen = {}
leaf = true
walk_stack(stack_idx, stack_prefix, stack_frame, frame_func) do |fi|
func_data[fi][:alloc] += weight if leaf
leaf = false
seen[fi] = true
end
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The seen hash is initialized and populated but never read within the jsAllocations accumulation loop. Since this loop processes every allocation sample, removing the unused hash will slightly improve performance and clarity in this hot path.

    leaf = true
    walk_stack(stack_idx, stack_prefix, stack_frame, frame_func) do |fi|
      func_data[fi][:alloc] += weight if leaf
      leaf = false
    end

Comment thread bin/analyze_profile.rb
Comment on lines +335 to +337
when "--diff"
diff = [argv[i + 1], argv[i + 2]]
i += 3
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The --diff argument parsing does not verify that two paths are actually provided. If --diff is the last argument or only followed by one path, argv[i + 1] or argv[i + 2] will be nil, causing a crash in load_profile (specifically at path.end_with?). Consider adding a check for the number of remaining arguments.

Comment thread bin/analyze_profile.rb
diff = [argv[i + 1], argv[i + 2]]
i += 3
when "-h", "--help"
puts File.read(__FILE__).lines[1..14].join.gsub(/^# ?/, "")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Extracting the help text by hardcoded line numbers (lines[1..14]) is brittle. Any changes to the file header or the addition of comments at the top of the file will break the help output. Using a heredoc is a more robust way to define the help text.

      puts <<~HELP
        Programmatic analyzer for Vernier .vernier.json[.gz] profiles
        (Firefox Profiler format). Emits a markdown report with the top
        functions by self time, total time, allocation count, and (in
        retained mode) retained bytes -- no profiler.firefox.com required.

        Usage:
          ruby bin/analyze_profile.rb path/to/profile.vernier.json [--out report.md]
          ruby bin/analyze_profile.rb --diff baseline.json after.json [--out diff.md]

        Pure stdlib (json + zlib). No external gems.
      HELP

Comment thread spinel_codegen_profile.rb
Comment on lines +47 to +52
when "--interval"
interval = Integer(ARGV[i + 1])
i += 2
when "--alloc"
allocation_interval = Integer(ARGV[i + 1])
i += 2
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The script lacks bounds checking when accessing ARGV[i + 1] for numeric options like --interval and --alloc. If these flags are provided without a following value, Integer(nil) will be called, raising a TypeError and crashing the script.

OriPekelman added a commit to OriPekelman/tep that referenced this pull request May 20, 2026
Phase 0 + Phase 1 of the design in docs/PG-BATTERY.md. lib/tep/tep_pg.c
is the libpq shim (~430 LoC, mirrors tep_sqlite.c's slot-table +
rotating-string-buffer pattern); lib/tep/pg.rb exposes the ruby-pg-
faithful surface as PG.connect / PG::Connection / PG::Result /
PG::Error.

v1 surface:

  * PG.connect(opts) -- String conninfo or Hash<String,String>.
  * Connection -- close / finish / reset / status / transaction_status /
    server_version / error_message / exec(sql) / exec_params(sql, params) /
    escape_string / escape_identifier / escape_literal / connected? /
    last_sqlstate / last_error_message / last_result_rh.
  * Result -- status / ok? / error_message / error_field / cmd_status /
    cmd_tuples / ntuples / nfields / fname / fnumber / ftype / fformat /
    fmod / getvalue / getisnull / getlength / value / fields / values /
    column_values / each_row(Array) / each(Hash) / clear / sql_state.
  * Diagnostic field constants (PG::DIAG_SQLSTATE = 67, etc.).
  * PG.libpq_version.
  * PG::Error -- single base class for v1.

Spinel-imposed v1 divergences from the design doc (both tracked):

  * matz/spinel#622 -- raise X.new(msg, ...) doesn't lower for custom
    Exception subclasses. The two-arg `raise X, msg` form works for
    top-level classes but not module-namespaced ones (#627 below).
  * matz/spinel#627 -- rescue ParentClass and is_a?(ParentClass) skip
    the class hierarchy; rescue ModuleNS::Klass even for exact match
    skips. Until #627 lands the planned PG:: leaf-class hierarchy
    (PG::UniqueViolation et al.) would be unrescuable in practice, so
    v1 ships only the base PG::Error and returns Result-with-ok? for
    error handling. AR-style `rescue PG::UniqueViolation` becomes
    v1.5 territory.

Build integration:

  * Dockerfile.dev: + libpq-dev, postgresql-client, pkg-config.
  * Makefile: TEP_PG_O / TEP_PG_CFLAGS / TEP_PG_LIBS via pkg-config
    libpq (pg_config fallback). helper target builds tep_pg.o.
  * bin/tep: @TEP_PG_O@ / @TEP_PG_CFLAGS@ placeholder substitution.
  * lib/tep.rb: require_relative "tep/pg" after tep/sqlite.
  * .gitignore: examples/pg_hello binary.

Verified end-to-end against a real Postgres 16 instance:

  GET /         libpq 17.0.10 / server_version 160014 / SELECT 1 + 'hello'
  GET /tables   list public.tables via pg_catalog
  GET /error    deliberate SELECT FROM missing -> result.ok? false,
                conn.last_sqlstate == "42P01", error message echoes
                the libpq ERROR text

Full `make test` (213 runs) is unchanged from baseline: only the
3 open #575 TestParallel errors + the TestHttp Net::ReadTimeout
flake variants remain.

docs/PG-BATTERY.md updated to record the v1 deviation from the
design (single PG::Error class) and the upstream issues that
gate the v1.5 hierarchy landing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant