Skip to content

Migrate windows-bindgen to use windows-metadata for winmd parsing#3915

Merged
kennykerr merged 19 commits intomasterfrom
copilot/refactor-windows-bindgen-for-migration
Mar 2, 2026
Merged

Migrate windows-bindgen to use windows-metadata for winmd parsing#3915
kennykerr merged 19 commits intomasterfrom
copilot/refactor-windows-bindgen-for-migration

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Feb 27, 2026

windows-bindgen contained ~1,650 lines of ECMA-335 parsing logic that duplicated what windows-metadata already provides. This removes all of that internal parsing code and replaces it with direct use of windows-metadata, including full unification of the Type, Value, and Signature representations.

What's this all about?

windows-bindgen now delegates all ECMA-335 parsing — type signatures, method signatures, attribute arguments, and constant values — exclusively to windows-metadata. All intermediate extension-trait wrappers that merely forwarded to windows-metadata inherent methods have been removed; call sites now call those methods directly.

Changes

  • Deleted winmd/file.rs, row.rs, blob.rs, codes.rs, attributes.rs — the internal ECMA-335 parser
  • Deleted tables/attribute.rs — the AttributeExt trait is no longer needed; call sites now call windows-metadata::Attribute::value() directly
  • Deleted tables/impl_map.rs — the ImplMapExt trait had no call sites and is no longer needed; all call sites now call windows-metadata::ImplMap methods directly
  • winmd/bindings.rs — restored as a generated file (produced by tool_bindings); winmd/mod.rs now includes it via mod bindings; pub use bindings::*;
  • winmd/mod.rs — re-exports windows-metadata types as 'static aliases (e.g. type TypeDef = windows_metadata::reader::TypeDef<'static>), preserving the existing 'static lifetime convention without pervasive lifetime changes; CorElementType and ELEMENT_TYPE_* constants are now sourced from bindings.rs; MethodCallAttributes is now re-exported directly from windows-metadata rather than redefined locally; TypeDefOrRefExt and MemberRefParentExt type_name() impls are now free of unsafe/transmute; BlobExt trait removed as all direct blob parsing has been eliminated
  • winmd/reader.rsReader wraps windows_metadata::reader::TypeIndex via Box::into_raw + unsafe 'static transmute; the old reader: *const Reader field on each File is replaced by a thread-local CURRENT_READER; with_full_name trims generic arity suffixes (`1, `2, etc.) so raw TypeDef::name() values resolve correctly against the TypeIndex map; nested-type collection is now recursive to handle deeply-nested structs (e.g. VARIANT → VARIANT_0 → _Anonymous_e__Struct)
  • tables/*.rs — replaced with extension traits (TypeDefExt, FieldExt, MethodDefExt, etc.) that layer bindgen-specific methods on top of the metadata types; no unsafe or transmute is needed in any table extension
  • tables/field.rsfield_type() now calls windows-metadata::Field::ty() + Type::from_metadata_type() instead of manual blob parsing; FieldExt is the only remaining trait wrapper
  • tables/method_def.rsmethod_signature() now delegates to windows-metadata::MethodDef::signature() for blob parsing; a new Type::from_metadata_type() converter maps windows_metadata::Typewindows_bindgen::Type; from_blob_impl is eliminated entirely
  • tables/constant.rsConstantExt::constant_value() wrapper removed; call sites now call windows-metadata::Constant::value() directly; constant_type() uses Type::from_metadata_type()
  • tables/type_ref.rsTypeRef::name() and TypeRef::namespace() are called directly; no unsafe or transmute required
  • types/mod.rs — added Type::from_metadata_type() which converts windows-metadata::Type to windows-bindgen::Type (full remap including VALUETYPE/CLASS resolution via current_reader().unwrap_full_name()); from_blob now delegates to blob.read_type_signature() + from_metadata_type; from_blob_impl deleted
  • value.rs — removed the custom Value enum; replaced with pub use windows_metadata::Value; added ValueExt trait providing the write() method; all callers updated (Value::StrValue::Utf8, Value::TypeNameValue::Utf8, Value::StringValue::Utf16)
  • types/delegate.rs, types/interface.rs, types/class.rs, types/method.rs — code generation now uses trim_tick(def.name()) for Rust identifiers so WinRT generic types (e.g. IIterable`1) produce valid tokens
  • types/class.rsrequired_interfaces() now uses with_full_name().next() instead of unwrap_full_name() so WinRT API contract types (e.g. Windows.Foundation.UniversalApiContract) that are intentionally excluded from the Reader map are gracefully skipped rather than causing a panic; has_default_constructor() now checks for actual Type::Interface factory args rather than any Value::Utf8 arg, so ActivatableAttribute(UniversalApiContract, 4) (default constructor, no factory) is correctly identified
  • types/index.rsmethod_signature() now receives the interface's actual generics slice instead of &[], fixing a panic (index out of bounds) when tool_bindings processes generic interfaces (e.g. IIterable<T>) whose method signatures contain ELEMENT_TYPE_VAR references
  • lib.rs — restored non_snake_case and non_camel_case_types lint expectations required by bindings.rs
  • crates/libs/windows/features.json — regenerated with correct trimmed names for generic WinRT types
  • Cargo.toml — added windows-metadata dependency
  • windows-metadata: ImplMap::import_name() — changed return type from &str to &'a str
  • windows-metadata: ModuleRef::name() — changed return type from &str to &'a str
  • windows-metadata: TypeDefOrRef::namespace() and TypeDefOrRef::name() — changed to return &'a str, consistent with MemberRefParent; eliminates the need for unsafe { transmute } in TypeDefOrRefExt::type_name() and MemberRefParentExt::type_name()
  • windows-metadata: flags! macro — changed the inner field visibility from pub(crate) to pub so flag types such as MethodCallAttributes can be constructed from raw values by external crates

The approach keeps the existing unsafe 'static pattern (heap-pinned TypeIndex, never moved, freed in Drop) to avoid cascading lifetime parameter changes across the entire codebase, while eliminating all duplicate parsing logic. windows-metadata is now used exclusively for all ECMA-335 blob parsing — type signatures, method signatures, attribute arguments, and constant values. No unsafe or transmute appears in any of the tables/ extension trait implementations, and no direct blob parsing remains anywhere in windows-bindgen.


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits February 27, 2026 20:32
Remove the duplicate ECMA-335 winmd parsing code (~1647 lines) from
windows-bindgen and replace it with windows-metadata's reader types.

Changes:
- Add windows-metadata = { workspace = true } to Cargo.toml
- Delete winmd/file.rs (ECMA-335 binary parser, ~851 lines)
- Delete winmd/bindings.rs (Windows PE structs, ~234 lines)
- Delete winmd/row.rs (Row types with 'static trick, ~159 lines)
- Delete winmd/blob.rs (Blob with reader() method, ~191 lines)
- Delete winmd/codes.rs (coded indices Decode trait, ~128 lines)
- Delete winmd/attributes.rs (flags macros, ~84 lines)
- Rewrite winmd/mod.rs: type aliases for windows-metadata 'static types,
  ELEMENT_TYPE constants, extension traits (BlobExt, TypeDefOrRefExt,
  MemberRefParentExt, HasReader, GuidAttributeExt)
- Rewrite winmd/reader.rs: Reader wraps windows-metadata TypeIndex via
  Box::into_raw and unsafe 'static transmute; thread-local for reader access
- Rewrite tables/*.rs as extension traits on windows-metadata types
- Rename conflicting methods: field_type(), constant_type(),
  constant_value(), method_signature() to avoid shadowing metadata's
  inherent methods
- Update all call sites throughout codebase

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: kennykerr <9845234+kennykerr@users.noreply.github.com>
Copilot AI changed the title [WIP] Refactor winmd parsing logic in windows-bindgen Migrate windows-bindgen to use windows-metadata for winmd parsing Feb 27, 2026
@kennykerr
Copy link
Copy Markdown
Collaborator

@copilot looks like a reasonable first pass - please fix the build breaks and then I'll take another look.

… extension methods

Co-authored-by: kennykerr <9845234+kennykerr@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Feb 27, 2026

@copilot looks like a reasonable first pass - please fix the build breaks and then I'll take another look.

Fixed in c18956b. The build break was in tables/method_def.rs: ImplMap::import_name() and ModuleRef::name() return &str with lifetime elided to &self due to Rust's rules, even though the data lives in the 'static TypeIndex. Fixed with unsafe { std::mem::transmute(...) } which is sound for the same reason the rest of the 'static pattern is sound — the TypeIndex is heap-pinned and lives as long as the Reader. All 14 tests pass.

@kennykerr
Copy link
Copy Markdown
Collaborator

kennykerr commented Feb 27, 2026

@copilot more build failures - keep fixing them please.

Co-authored-by: kennykerr <9845234+kennykerr@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Feb 27, 2026

@copilot more build failures - keep fixing them please.

Fixed in 5ef584f. The CI failure was a fmt check — cargo fmt was not run before the previous commit. Running it fixed formatting diffs in winmd/mod.rs and winmd/reader.rs.

@riverar
Copy link
Copy Markdown
Collaborator

riverar commented Feb 27, 2026

@copilot Additional CI failures, fix them.

Copilot AI requested a review from riverar February 27, 2026 23:20
@riverar
Copy link
Copy Markdown
Collaborator

riverar commented Feb 27, 2026

@copilot There are additional CI failures. Check them out and apply fixes as needed.

@kennykerr
Copy link
Copy Markdown
Collaborator

@copilot please fix the build failure.

…es, bindings.rs, code gen names

Co-authored-by: kennykerr <9845234+kennykerr@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Feb 28, 2026

@copilot please fix the build failure.

Fixed in d20212c. There were multiple independent build failures:

  1. with_full_name lookup failure (type not found: Windows.Foundation.IReference1): TypeIndexstores types keyed bytrim_tick(name), but TypeDef::name()returns raw names. Fixed by trimming inwith_full_name`.

  2. Nested type lookup failure (VARIANT_0 missing _Anonymous_e__Struct): The nested-type map was only built for top-level types. Deeply nested structs like VARIANT → VARIANT_0 → _Anonymous_e__Struct require recursive collection. Fixed with a recursive collect_nested function.

  3. Missing winmd/bindings.rs: Generated by tool_bindings, this file was deleted in the migration. Restored it, removed duplicate ELEMENT_TYPE constants from mod.rs, and added back non_snake_case/non_camel_case_types to lib.rs lint expectations.

  4. Invalid Rust tokens in generated code (IIterable1_Vtbl): WinRT generic type names (e.g., IIterable1) were used raw when generating identifiers in delegate.rs, interface.rs, class.rs, method.rs. Fixed with trim_tick(def.name()).

  5. Regenerated features.json to reflect the corrected trimmed names.

All 15 windows-bindgen tests pass; all gen tools (tool_bindgen, tool_bindings, tool_yml, tool_license, tool_standalone, tool_workspace) succeed with no diff.

@kennykerr
Copy link
Copy Markdown
Collaborator

@copilot please fix the remaining build failures.

…unds

Co-authored-by: kennykerr <9845234+kennykerr@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Feb 28, 2026

@copilot please fix the remaining build failures.

Fixed in d55670c. The test_bindgen::subset_namespace test was panicking with assertion left == right failed (left: 2, right: 0) from windows-metadata's Blob::drop, which asserts self.len() == 0 when a blob is dropped.

The root cause: from_blob_impl for ELEMENT_TYPE_ARRAY was not reading num_lo_bounds or the lo bound values (per ECMA-335 §II.23.2.13 ArrayShape). The old internal Blob had no drop assertion so these unread bytes were silent; windows-metadata's Blob asserts all bytes are consumed.

The 2 remaining bytes were exactly num_lo_bounds = 1 (1 byte) + lo_bounds[0] = 0 (1 byte). Fixed by reading and discarding them.

- ConstantExt no longer has constant_value(); all call sites now use
  .value() directly (the inherent windows_metadata::Constant::value())
- Removed mod impl_map; from tables/mod.rs; ImplMapExt::scope() was an
  unused alias for import_scope() and had no call sites

Co-authored-by: kennykerr <9845234+kennykerr@users.noreply.github.com>
@kennykerr
Copy link
Copy Markdown
Collaborator

@copilot please fix build break.

Comment thread crates/libs/bindgen/src/tables/field.rs
self.str(2)
impl ImplMapExt for ImplMap {
fn scope(&self) -> ModuleRef {
self.import_scope()
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot why not remove the trait and just call the method directly?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in d5e3ac7. impl_map.rs deleted — ImplMapExt had no call sites (all call sites already use import_scope() directly).

Comment thread crates/libs/bindgen/src/winmd/mod.rs Outdated

pub use windows_metadata::MethodCallAttributes;
// Extension trait for Blob: adds bindgen-specific read helpers.
pub trait BlobExt {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot I think we've removed all uses of direct blob parsing - can we remove this now?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in d5e3ac7. The entire BlobExt trait and implementation removed — all four methods (read_usize, read_str, read_integer, reader) had no call sites outside the definition.

…s_default_constructor

- required_interfaces(): use with_full_name().next() instead of unwrap_full_name()
  so contract types (like Windows.Foundation.UniversalApiContract) that are skipped
  from the Reader map are gracefully skipped rather than causing a panic
- has_default_constructor(): check for actual Type::Interface args rather than any
  Value::Utf8, so ActivatableAttribute(UniversalApiContract, 4) (no factory interface)
  correctly returns true

Co-authored-by: kennykerr <9845234+kennykerr@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 2, 2026

@copilot please fix build break.

Fixed in 58c2a2b. The panic was type not found: Windows.Foundation.UniversalApiContract in required_interfaces().

UniversalApiContract is a WinRT struct with ApiContractAttribute, which is explicitly skipped from the Reader's type map. With windows-metadata::Attribute::value(), contract type args come back as Value::Utf8 alongside factory interface args, and the old unwrap_full_name call panicked when it couldn't find the contract type.

Fixed by replacing unwrap_full_name with with_full_name().next() in both required_interfaces() and has_default_constructor(), gracefully skipping any Utf8 arg that doesn't resolve to a Type::Interface.

…MapExt

- tables/field.rs: field_type() now calls windows-metadata Field::ty() + Type::from_metadata_type() instead of manual blob parsing
- tables/impl_map.rs: deleted (file was not in mod.rs; ImplMapExt had no call sites)
- winmd/mod.rs: removed BlobExt trait and implementation (read_usize/read_str/read_integer/reader all had no call sites)

Co-authored-by: kennykerr <9845234+kennykerr@users.noreply.github.com>
@kennykerr
Copy link
Copy Markdown
Collaborator

@copilot please fix build break.

…in index.rs

Co-authored-by: kennykerr <9845234+kennykerr@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 2, 2026

@copilot please fix build break.

Fixed in 869f32e. The gen/tool_bindings job was panicking at blob.rs:167 with index out of bounds: the len is 0 but the index is 0 — the ELEMENT_TYPE_VAR case in windows-metadata's blob parser hitting an empty generics slice.

Root cause: index.rs called method.method_signature(namespace, &[]) for all interface methods, but generic WinRT interfaces (e.g. IIterable<T>) have methods whose signatures contain ELEMENT_TYPE_VAR. Fixed by passing ty.generics for Type::Interface(ty) instead of &[].

@kennykerr kennykerr marked this pull request as ready for review March 2, 2026 14:46
@kennykerr kennykerr merged commit d35ca81 into master Mar 2, 2026
32 checks passed
@kennykerr kennykerr deleted the copilot/refactor-windows-bindgen-for-migration branch March 2, 2026 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants