BE-457: HashQL: MIR execution pipeline extensions for postgres compilation#8525
Conversation
feat: checkpoint (II) feat: checkpoint (III) feat: snapshot vec feat: add dedicated filter feat: checkpoint feat: filter implementation feat: filter implementation (mostly) done chore: environment capture note chore: always postgres bigint feat: target clone feat: simplify lookup feat: move storage up feat: eval entity path chore: checkpoint chore: checkpoint chore: find entrypoint feat: eval context feat: eval cleanup chore: cleanup feat: track index feat: wire up filter feat: add error reporting chore: checkpoint feat: add traverse, and first postgres compiler outline feat: traverse bitmap feat: move traversal out feat: projections feat: projections fix: clippy feat: subquery projection for lateral feat: checkpoint feat: test plan feat: checkpoint feat: checkpoint – failing tests ;-; feat: checkpoint – failing tests ;-; feat: checkpoint — passing tests fix: import fix: entity type feat: checkpoint feat: attribute a cost to terminator placement switches fix: import feat: checkpoint feat: checkpoint chore: lint
PR SummaryMedium Risk Overview Extends execution planning data structures for Postgres compilation: islands/graphs gain new query helpers ( Changes placement behavior by introducing a fixed backend switch overhead into terminator transition costs (via Written by Cursor Bugbot for commit 8d113b2. This will update automatically on new commits. Configure here. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
🤖 Augment PR SummarySummary: Prepares the HashQL MIR execution pipeline for Postgres compilation by returning a full island dependency graph and exposing traversal/placement APIs needed by the SQL generator.
entity_uuid_equality).
Notes: Backend switch costs are heuristics intended to be refined by a measured cost model.
🤖 Was this summary useful? React with 👍 or 👎 |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## bm/be-456-hashql-entity-type-model-enrichment-and-graph-store-query #8525 +/- ##
========================================================================================================
+ Coverage 63.39% 83.98% +20.58%
========================================================================================================
Files 763 326 -437
Lines 89217 51060 -38157
Branches 2916 1299 -1617
========================================================================================================
- Hits 56563 42882 -13681
+ Misses 32107 7763 -24344
+ Partials 547 415 -132
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Merging this PR will not alter performance
Comparing Footnotes
|
6ef6d13 to
ef9b858
Compare
5d95ba7 to
1aa0f1c
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
libs/@local/hashql/mir/src/pass/execution/placement/solve/estimate/mod.rs
Show resolved
Hide resolved
ef9b858 to
a91293c
Compare
1aa0f1c to
87024df
Compare
a91293c to
43775ba
Compare
87024df to
269a31f
Compare
db87959 to
8d113b2
Compare
43775ba to
3ba579f
Compare

🌟 What is the purpose of this PR?
Prepares the MIR execution pipeline for consumption by the postgres compiler. The execution analysis now produces a complete
IslandGraph(not just a flat island list), the placement solver and island placement accept external allocators, and the traversal system gains theTraversalPathBitMapandas_symbol()APIs that the SQL generator needs. Also adds a backend switch cost to the terminator placement so cross-backend transitions are no longer free.🔍 What does this change?
Execution analysis (
pass/execution/mod.rs):run()becomesrun_in()with an explicit allocator parameter, returningExecutionAnalysisResidual(assignment + island graph). The island graph is now constructed as part of the analysis rather than left to the caller.run_all_in()which runs the execution analysis over all graph-read bodies in aDefIdSlice.Traversal system (
traversal/mod.rs,traversal/entity.rs):TraversalPathBitMap: per-vertex-type collection ofTraversalPathBitSets with pointwise lattice operations. The postgres compiler uses this to track which paths each island accesses across all vertex types.TraversalPath::as_symbol(): returns a static symbol for each path variant, used as SQL column aliases so the interpreter can locate result columns by name.TraversalPathBitSet::vertex(): returns the vertex type for a bitset.EntityPath::as_symbol()andEntityPath::column_name(): per-path SQL identifiers.Terminator placement (
terminator_placement/mod.rs):TransMatrixgainsAddAssignfor element-wise saturating addition.backend_switch_cost()which encodes a fixed overhead for cross-backend transitions (Postgres to Interpreter: 8, Interpreter to Embedding: 4, etc.). Previously cross-backend transitions had zero inherent cost, so empty blocks were arbitrarily assigned to the interpreter even when staying on postgres was free.Placement solver (
placement/solve/):PlacementSolver::run()becomesrun_in()with allocator parameter.Island graph (
island/graph/mod.rs):IslandGraph::new_in()now takes an allocator for its output storage.Pretty printer (
pretty/text.rs):TextFormatAnnotationsgainsannotate_basic_block()andBasicBlockAnnotationassociated type, plus a blanket impl for&mut T.Builder (
builder/rvalue.rs):RValueBuilder::opaque_entity_uuid()convenience constructor for the common pattern of extracting an entity's UUID.Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
The
backend_switch_cost()values (8, 4, etc.) are hand-tuned heuristics. They correctly prevent the solver from arbitrarily switching backends for empty blocks, but a proper cost model would derive these from measured overhead.🛡 What tests cover this?
execution/tests.rs) includingentity_uuid_equality,mixed_postgres_embedding_interpreter,projection_and_apply_splitseq_opaque_entity_uuid)❓ How to test this?