Honor caller-provided rowType in HiveViewExpander.expandView

[simbadzina]

Summary

When a downstream view does SELECT * FROM upstream and upstream's HMS-recorded row type disagrees with what its re-parsed body produces in field order, the downstream's positional access can land on the wrong field. The case that surfaces this most painfully is prefix-name sibling pairs:

Field index	Caller's expected rowType	Re-parsed body's rowType
0	id (INTEGER)	idHash (VARCHAR)
1	idHash (VARCHAR)	id (INTEGER)

A case-insensitive lookup of id against the body's row type can match idHash — the resolver gets confused when a row type contains both a short name and a longer name that has it as a prefix. Every consumer of the view then silently reads values from the wrong source column.

The root cause is that HiveViewExpander.expandView() was discarding its rowType argument and returning whatever the re-parsed view body produced, violating the ViewExpander contract that the returned RelRoot's row type must match the caller-provided rowType.

Fix

After convertQuery, wrap the returned RelRoot in a LogicalProject that re-aligns fields to the caller-provided rowType by name (case-insensitive):

flowchart LR
  C["Caller's rowType<br/>[id INTEGER, idHash VARCHAR]"] --> E
  B["Re-parsed body's rowType<br/>[idHash VARCHAR, id INTEGER]"] --> E[alignToRowType]
  E --> Q{Arity match<br/>and field names<br/>aligned by order?}
  Q -->|Yes| K[Return original RelRoot]
  Q -->|All names present<br/>but different order| W["Wrap in LogicalProject<br/>that re-binds by name<br/>(case-insensitive)"]
  Q -->|Arity differs<br/>or a name is missing| L[Log warn, return original]

Loading

The safe-fallback paths preserve pre-fix behavior for callers whose expanded body genuinely produces a different shape, while making such cases visible in logs.

Test plan

• Unit tests for alignToRowType in HiveViewExpanderTest covering: already-aligned (no-op fast path), case-only differences (no-op), reordered fields (wraps in Project), single prefix-sibling pair (id / idHash), multiple prefix-sibling pairs in one row type (id/idHash, email/emailVerified, country/countryCode simultaneously — mirrors the realistic failure shape at production scale), missing field (safe fallback), and different arity (safe fallback).
• Integration test testExpandViewAlignsToCallerRowType in HiveToRelConverterTest that drives the full expandView path with a deliberately reversed caller-provided rowType.
• Sanity-check: temporarily reverting alignToRowType to return root; causes exactly the tests targeting the fix to fail (testReorderedFieldsWrapInProject, testPrefixSiblingsReorderedCorrectly, testMultiplePrefixSiblingsAllReorderedCorrectly, testExpandViewAlignsToCallerRowType); the safe-fallback and no-op tests still pass. Restoring the fix turns the suite green.
• Full :coral-hive:test and :coral-trino:test suites pass with no regressions.

Manual verification

Validated end-to-end against a real production view-on-view chain (Avro-backed base table → intermediate Hive view → downstream view) using internal tooling that drives the full Coral translation path. Confirmed:

• Without the fix: every prefix-name sibling pair in the downstream view's translated SQL had its source column references swapped, in both Trino and Spark dialects. Three pairs in the same view were affected simultaneously.
• With the fix: every reference is correct; the no-op fast path is preserved for upstream views that don't trigger the bug.

Also characterized the failure surface by varying the upstream view shape: the bug fires specifically when the upstream lists the longer prefix-extension name before its shorter sibling (e.g. idHash before id). SELECT * upstreams and shorter-name-first explicit projections do not trigger the bug.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR fixes Hive view expansion so expanded view outputs can be reordered to match the caller-provided row type, preventing positional column mismatches in downstream view-on-view scenarios.

Changes:

• Adds HiveViewExpander.alignToRowType to reorder expanded view fields by case-insensitive name when needed.
• Adds unit coverage for no-op, reorder, prefix-sibling, missing-field, and arity fallback cases.
• Adds an integration fixture and test for the full expandView path.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
coral-hive/src/main/java/com/linkedin/coral/hive/hive2rel/HiveViewExpander.java	Adds row-type alignment after view query conversion.
coral-hive/src/test/java/com/linkedin/coral/hive/hive2rel/HiveViewExpanderTest.java	Adds unit tests for alignment behavior.
coral-hive/src/test/java/com/linkedin/coral/hive/hive2rel/HiveToRelConverterTest.java	Adds integration regression test for caller row-type ordering.
coral-hive/src/test/java/com/linkedin/coral/hive/hive2rel/TestUtils.java	Adds Hive test table/view fixtures used by the integration test.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[wmoustafa]

Is not the right behavior to throw an exception? There should not be a disagreement between the caller expectation and the view structure in the first place.

[simbadzina]

Do you mean for when there is a mismatch like an missing field, or even when there is an ordering issue?

For the ordering, I think we can self-heal drift with the alignment code since there is a complete info.

I'm assuming a change like a new column could show up a mismatch, missing field.
Automatically taking the root would expose the column to consumers, whereas an error would break the view. I think for select * we'd want the new column to seamless appear.

[simbadzina]

The source of the divergence I observed was in another system, outside of Coral. During view creation, the storage columns were being explicitly set in the storageDescriptor of the view in a different order than in the view text. This particular instance of divergence has been fixed in that system.

To guard against similar broken views Coral can throw an exception as you suggested, or at the least log a warning.