ty-aware delayed AST -> HIR lowering by aerooneqq · Pull Request #153489 · rust-lang/rust

aerooneqq · 2026-03-06T11:22:56Z

This PR implements a prototype of ty-aware delayed AST -> HIR lowering. Part of #118212.

Motivation

When lowering delegation in perfect scenario we would like to access the ty-level information, in particular, queries like generics_of, type_of, fn_sig for proper HIR generation with less hacks. For example, we do not want to generate more lifetimes than needed, because without ty-level queries we do not know which delegee's lifetimes are late-bound. Next, consider recursive delegations, for their proper support without ty we would have to either duplicate generics inheritance code in AST -> HIR lowering or create stubs for parts of the HIR and materialize them later. We already use those queries when interacting with external crates, however when dealing with compilation of a local crate we should use resolver for similar purposes. Finally, access to ty-level queries is expected to help supporting delegation to inherent impls, as we can not resolve such calls during AST -> HIR stage.

Benefits

We eliminate almost all code that uses resolver in delegation lowering:

Attributes inheritance is done without copying attributes from AST at resolve stage
Fn signatures are obtained from tcx.fn_sig
Param counts are also obtained from tcx.fn_sig
is_method function now uses tcx.associated_item instead of resolver
Generics are now inherited through get_external_generics that uses tcx.generics_of. Generics for recursive delegations should also work
DelegationIds that stored paths for recursive delegations is removed, we now use only delegee_id
Structs that were used for storing delegation-related information in resolver are almost fully removed
ast_index is no more used

Next steps

Remove creating generic params through AST cloning, proper const types propagation
Inherent impls

High level design overview

Queries

We store ids of delayed items to lower in Crate struct. During first stage of lowering, owners that correspond to delayed items are filled with MaybeOwner::Phantom.

Next, we define three new queries: lower_to_hir_delayed, get_delayed_child_owner and force_delayed_hir_lowering.

The first query is used when lowering known (which is in delayed_ids) delayed owner.

The second is feeded with children that were obtained during lowering of a delayed owner (note that the result of lowering of a single LocalDefId is not a single MaybeOwner, its a list of (LocalDefId, MaybeOwner) where the first MaybeOwner corresponds to delayed LocalDefId and others to children that were obtained during lowering (i.e. generic params)). By default get_delayed_child_owner returns MaybeOwner::Phantom. As we do not want to predict the whole list of children which will be obtained after lowering of a single delayed item, we need to store those children somewhere. There are several options:

Make the return type of lower_to_hir_delayed to be FxIndexMap<LocalDefId, MaybeOwner> and search children here. Search will be either linear or we can introduce a map somewhere which will track parent-child relations between a single delayed LocalDefId and its children.
Try to make query that will lower all delayed items in a loop and return a complete map of all delayed LocalDefIds and their children. In this case there will be problems with delayed items that require information about other delayed items.

By using such queries we handle the second concern, and in case of acyclic dependencies between delayed ids it automatically works, moreover we use queries as cache for delayed MaybeOwners. The only invariant here is that queries which are invoked during delayed HIR lowering of some LocalDefId should not in any way access children of other, yet unlowered, delayed LocalDefId, they should firstly materialize it.

The third query force_delayed_hir_lowering forces lowering of all delayed items and now integrated in hir_crate_items query.

Resolver for lowering

Currently the resolver_for_lowering is stolen in lower_to_hir function, however we want to prolong its life until all delayed LocalDefIds are materialized. For this purpose we borrow resolver_for_lowering in lower_to_hir and drop it after forcing materialization of all delayed LocalDefId in rustc_interface::run_required_analyses.

We split resolver for lowering into two parts: the first part is a readonly part that came to us from resolve, the second part is a mutable part that can be used to add or overwrite values in the readonly part. Such splitted resolver is used during delayed delegation lowering, as we can't steal it.

AST index

Lowering uses an AST index. It is created in lower_to_hir function and it references parts of AST. We want to avoid reindexing AST on every delayed LocalDefId lowering, however now it is not clear how to properly do it. As delayed HIR lowering is used only for delegation unstable feature it should not affect other use-cases of the compiler. But it will be reworked sooner or later.

petrochenkov · 2026-03-06T12:01:56Z

cc @oli-obk you may be interested in this, because this likely intersects with incremental AST lowering.

petrochenkov · 2026-03-06T13:21:23Z

@bors try @rust-timer queue

TY-aware delayed HIR lowering

oli-obk · 2026-03-06T13:47:49Z

compiler/rustc_ast_lowering/src/lib.rs

+
 #[extension(trait ResolverAstLoweringExt)]
-impl ResolverAstLowering {
+impl ResolverAstLowering<'_> {


I went down a slightly different route: c98471e

That gets rid of the extension trait entirely and allows adding custom information that doesn't need to be crate global.

I've tried to eliminate ResolverAstLoweringExt trait and use splitted (into readonly and mutable part) resolver in hir_crate, unfortunately it led to perf regressions, as before this change we modified maps in-place and after we have to check two maps to find LocalDefId for NodeId. Now I've abstracted resolver through R: ResolverAstLoweringExt where R is a generic type parameter, thanks to this we can steal resolver and modify it in-place during hir_crate and during delayed lowering of delegations we can use splitted resolver, as we can't steal resolver during delayed lowering. Abstraction changes are in #153656, in perfect scenario we would like to have resolver-per-owner, as you tried to do in your PRs, then there will be no need for this abstraction through generic param, however it is out of scope of this PR, as our main goal is to give access to ty-level queries during delegation lowering.

oli-obk · 2026-03-06T14:25:59Z

compiler/rustc_middle/src/hir/mod.rs

    providers.hir_module_items = map::hir_module_items;
-    providers.local_def_id_to_hir_id = |tcx, def_id| match tcx.hir_crate(()).owners[def_id] {
+    providers.get_delayed_child_owner = |_, _| MaybeOwner::Phantom;
+    providers.local_def_id_to_hir_id = |tcx, def_id| match tcx.hir_crate(()).owner(tcx, def_id) {


Just this change adding an owner method seems sth nice to do on main

rust-bors · 2026-03-06T15:36:16Z

☀️ Try build successful (CI)
Build commit: a77c1ae (a77c1ae1722badff1c427c67e07dc1e08989756f, parent: f82485388963286763f8c8a9261ac227627f6a18)

rust-timer · 2026-03-06T16:43:10Z

Finished benchmarking commit (a77c1ae): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.7%	[0.1%, 3.5%]	36
Regressions ❌ (secondary)	0.7%	[0.0%, 2.7%]	103
Improvements ✅ (primary)	-0.6%	[-1.4%, -0.2%]	9
Improvements ✅ (secondary)	-0.5%	[-0.8%, -0.2%]	18
All ❌✅ (primary)	0.4%	[-1.4%, 3.5%]	45

Max RSS (memory usage)

Results (primary 4.3%, secondary 3.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	4.3%	[0.4%, 27.0%]	57
Regressions ❌ (secondary)	3.4%	[1.0%, 24.6%]	83
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.7%	[-4.2%, -3.2%]	2
All ❌✅ (primary)	4.3%	[0.4%, 27.0%]	57

Cycles

Results (primary 2.7%, secondary 3.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.7%	[2.1%, 3.2%]	2
Regressions ❌ (secondary)	3.9%	[1.3%, 8.5%]	11
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.2%	[-2.2%, -2.2%]	1
All ❌✅ (primary)	2.7%	[2.1%, 3.2%]	2

Binary size

Results (primary 0.0%, secondary 0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.0%	[0.0%, 0.1%]	11
Regressions ❌ (secondary)	0.1%	[0.0%, 0.2%]	16
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.0%	[0.0%, 0.1%]	11

Bootstrap: 480.442s -> 479.934s (-0.11%)
Artifact size: 397.10 MiB -> 397.38 MiB (0.07%)

@petrochenkov

…o-hir-arena, r=petrochenkov Replace Box<[TraitCandidate]> with &'hir [TraitCandidate<'hir>] This PR allocates trait candidates on HIR arena and replaces `remove` with `get` in `ResolverAstLowering`. First step for rust-lang#153489. r? @petrochenkov

@petrochenkov

…, r=petrochenkov Replace Box<[TraitCandidate]> with &'hir [TraitCandidate<'hir>] This PR allocates trait candidates on HIR arena and replaces `remove` with `get` in `ResolverAstLowering`. First step for #153489. r? @petrochenkov

@petrochenkov

…o-hir-arena, r=petrochenkov Replace Box<[TraitCandidate]> with &'hir [TraitCandidate<'hir>] This PR allocates trait candidates on HIR arena and replaces `remove` with `get` in `ResolverAstLowering`. First step for rust-lang#153489. r? @petrochenkov

@petrochenkov

Rollup merge of #153494 - aerooneqq:boxed-trait-candidates-to-hir-arena, r=petrochenkov Replace Box<[TraitCandidate]> with &'hir [TraitCandidate<'hir>] This PR allocates trait candidates on HIR arena and replaces `remove` with `get` in `ResolverAstLowering`. First step for #153489. r? @petrochenkov

petrochenkov · 2026-03-10T13:14:27Z

@bors try @rust-timer queue

rust-timer · 2026-03-10T16:03:21Z

Finished benchmarking commit (60f2f8d): comparison URL.

Overall result: ❌ regressions - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.1%, 0.5%]	101
Regressions ❌ (secondary)	0.2%	[0.1%, 0.4%]	28
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.2%	[-0.2%, -0.2%]	1
All ❌✅ (primary)	0.3%	[0.1%, 0.5%]	101

Max RSS (memory usage)

Results (primary 2.1%, secondary 1.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.1%	[2.1%, 2.1%]	1
Regressions ❌ (secondary)	3.3%	[1.7%, 6.2%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.3%	[-1.9%, -0.7%]	2
All ❌✅ (primary)	2.1%	[2.1%, 2.1%]	1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

Results (primary -0.0%, secondary -0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.0%	[-0.0%, -0.0%]	15
Improvements ✅ (secondary)	-0.1%	[-0.1%, -0.0%]	5
All ❌✅ (primary)	-0.0%	[-0.0%, -0.0%]	15

Bootstrap: 479.559s -> 482.365s (0.59%)
Artifact size: 394.94 MiB -> 394.89 MiB (-0.01%)

…spawn

…, do not update mu part on every lowering

…ace modifications

petrochenkov · 2026-03-16T09:04:53Z

@bors try @rust-timer queue

ty-aware delayed AST -> HIR lowering

rust-bors · 2026-03-16T11:16:01Z

☀️ Try build successful (CI)
Build commit: a964ec5 (a964ec51bf08657dcd415169bf84ca1519f55201, parent: 5b61449ed85a670f1dd3fca6a8c759ee0b451b66)

rust-timer · 2026-03-16T12:35:54Z

Finished benchmarking commit (a964ec5): comparison URL.

Overall result: ❌✅ regressions and improvements - no action needed

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.1%	[0.1%, 0.1%]	1
Regressions ❌ (secondary)	0.0%	[0.0%, 0.1%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.4%	[-2.8%, -0.0%]	2
All ❌✅ (primary)	0.1%	[0.1%, 0.1%]	1

Max RSS (memory usage)

Results (primary 0.7%, secondary -0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.7%	[0.6%, 1.0%]	3
Regressions ❌ (secondary)	2.2%	[2.2%, 2.2%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.3%	[-2.3%, -2.3%]	1
All ❌✅ (primary)	0.7%	[0.6%, 1.0%]	3

Cycles

Results (secondary -3.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.1%	[-3.1%, -3.1%]	1
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 481.08s -> 484.081s (0.62%)
Artifact size: 396.88 MiB -> 395.09 MiB (-0.45%)

rustbot assigned petrochenkov Mar 6, 2026

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Mar 6, 2026

This comment has been minimized.

Sign in to view

aerooneqq mentioned this pull request Mar 6, 2026

Replace Box<[TraitCandidate]> with &'hir [TraitCandidate<'hir>] #153494

Merged

This comment has been minimized.

Sign in to view

rust-bors bot pushed a commit that referenced this pull request Mar 6, 2026

Auto merge of #153489 - aerooneqq:two-phase-hir, r=<try>

a77c1ae

TY-aware delayed HIR lowering

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 6, 2026

oli-obk reviewed Mar 6, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Mar 6, 2026

This comment has been minimized.

Sign in to view

aerooneqq mentioned this pull request Mar 9, 2026

ast_lowering: fix ICE in fn_delegation with complex generics #153443

Closed

aerooneqq force-pushed the two-phase-hir branch from 13219bc to aa935ef Compare March 10, 2026 09:22

This comment has been minimized.

Sign in to view

aerooneqq mentioned this pull request Mar 10, 2026

Abstract AST lowering resolver #153656

Open

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 10, 2026

This comment has been minimized.

Sign in to view

aerooneqq added 8 commits March 13, 2026 12:59

First version of TY-aware delayed HIR lowering prototype

374387c

Use rustc_data_structures::sync::spawn instead of rustc_thread_pool::…

0ec984e

…spawn

Fix forcing lowering of delayed HIR

19ea084

Reduce size of mut part of the resolver as only two maps are modified…

aefe347

…, do not update mu part on every lowering

./x test tidy --bless

ed12351

Remove pub visibility for MutableResolverAstLowering

70115c3

If delayed_ids are empty return MaybeOwner::Phantom immediately

90673d7

Move lowerer creation out of the loop

0d60cce

aerooneqq force-pushed the two-phase-hir branch from e4d67ca to 065c1d3 Compare March 13, 2026 10:00

This comment has been minimized.

Sign in to view

aerooneqq force-pushed the two-phase-hir branch from 065c1d3 to aae6dda Compare March 13, 2026 10:12

aerooneqq added 2 commits March 13, 2026 16:18

Second version: abstract resolver, use risky_hack_borrow_mut for inpl…

3c7e56e

…ace modifications

Add get_or_insert_mut for Owners

167e829

aerooneqq force-pushed the two-phase-hir branch from aae6dda to 167e829 Compare March 13, 2026 13:29

aerooneqq added 2 commits March 14, 2026 22:32

Better transfer of resolver to delayed lowering

4f7b4c4

Cleanups

c2251e8

This comment has been minimized.

Sign in to view

rust-bors bot pushed a commit that referenced this pull request Mar 16, 2026

Auto merge of #153489 - aerooneqq:two-phase-hir, r=<try>

a964ec5

ty-aware delayed AST -> HIR lowering

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 16, 2026

This comment has been minimized.

Sign in to view

rustbot removed perf-regression Performance regression. S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Mar 16, 2026

Uh oh!

Conversation

aerooneqq commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Benefits

Next steps

High level design overview

Queries

Resolver for lowering

AST index

Uh oh!

This comment has been minimized.

petrochenkov commented Mar 6, 2026

Uh oh!

petrochenkov commented Mar 6, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

Choose a reason for hiding this comment

Uh oh!

aerooneqq Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rust-bors bot commented Mar 6, 2026

Uh oh!

This comment has been minimized.

rust-timer commented Mar 6, 2026

Overall result: ❌✅ regressions and improvements - please read the text below

Uh oh!

This comment has been minimized.

This comment has been minimized.

petrochenkov commented Mar 10, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rust-timer commented Mar 10, 2026

Overall result: ❌ regressions - please read the text below

Uh oh!

This comment has been minimized.

This comment has been minimized.

petrochenkov commented Mar 16, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors bot commented Mar 16, 2026

Uh oh!

This comment has been minimized.

rust-timer commented Mar 16, 2026

Overall result: ❌✅ regressions and improvements - no action needed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

aerooneqq commented Mar 6, 2026 •

edited

Loading

aerooneqq Mar 16, 2026 •

edited

Loading