[FLINK-39092][SQL] Enhance EXPLAIN plan to display watermark specification#27611
Open
featzhang wants to merge 7 commits intoapache:masterfrom
Open
[FLINK-39092][SQL] Enhance EXPLAIN plan to display watermark specification#27611featzhang wants to merge 7 commits intoapache:masterfrom
featzhang wants to merge 7 commits intoapache:masterfrom
Conversation
Collaborator
9f1095b to
322493c
Compare
322493c to
0684335
Compare
featzhang
added a commit
to featzhang/flink
that referenced
this pull request
Apr 22, 2026
…atermark The previous implementation used addDataStream + addTableWithWatermark which creates a LogicalWatermarkAssigner node but doesn't add watermark specifications to the table source's resolved schema. This caused the test to fail because FlinkLogicalTableSourceScan's explainTerms only displays watermark info when the table itself has watermark specs. Solution: Use DDL with WATERMARK FOR clause to properly create a table with watermark specifications in the resolved schema, ensuring the watermark display feature is tested correctly. Addresses CI failure in PR apache#27611.
featzhang
added a commit
to featzhang/flink
that referenced
this pull request
Apr 23, 2026
…atermark The previous implementation used addDataStream + addTableWithWatermark which creates a LogicalWatermarkAssigner node but doesn't add watermark specifications to the table source's resolved schema. This caused the test to fail because FlinkLogicalTableSourceScan's explainTerms only displays watermark info when the table itself has watermark specs. Solution: Use DDL with WATERMARK FOR clause to properly create a table with watermark specifications in the resolved schema, ensuring the watermark display feature is tested correctly. Addresses CI failure in PR apache#27611.
d9f280a to
444d397
Compare
Member
Author
|
@flinkbot run azure |
Add explicit watermark information in TableScan nodes to improve plan readability. This change helps streaming users quickly understand watermark strategies from EXPLAIN output. Example output: TableScan: fields: user_id, order_time watermark: order_time - order_time - INTERVAL '5' SECOND
Update test expected results to include watermark specification in FlinkLogicalTableSourceScan output: - ProjectSnapshotTransposeRuleTest.xml - PushFilterInCalcIntoTableSourceRuleTest.xml - PushWatermarkIntoTableSourceScanRuleTest.xml
- Added testExplainWithWatermark test case to verify watermark information is included in explain plans - Addresses review comment by davidradl requesting test coverage for explainTerms watermark functionality
…atermark The previous implementation used addDataStream + addTableWithWatermark which creates a LogicalWatermarkAssigner node but doesn't add watermark specifications to the table source's resolved schema. This caused the test to fail because FlinkLogicalTableSourceScan's explainTerms only displays watermark info when the table itself has watermark specs. Solution: Use DDL with WATERMARK FOR clause to properly create a table with watermark specifications in the resolved schema, ensuring the watermark display feature is tested correctly. Addresses CI failure in PR apache#27611.
- Apply spotless formatting to multi-line DDL string in testExplainWithWatermark - Add expected plan output for testExplainWithWatermark in ExplainTest.xml to resolve DiffRepository assertion failure
This is an empty commit to trigger a fresh CI run. The previous CI failure was due to a flaky test (RestClientTest.testRestClientClosedHandling) which is unrelated to the watermark EXPLAIN feature changes in this PR.
…ile downloading crictl)
8180e66 to
01fa5eb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
This change enhances the EXPLAIN plan output for TableScan nodes by explicitly displaying watermark specifications. Currently, watermark logic is difficult to identify in EXPLAIN plans, which creates challenges for streaming users trying to understand and debug watermark strategies.
Before:
After:
This improvement makes watermark strategies immediately visible in query plans, helping streaming users:
Brief change log
FlinkLogicalTableSourceScan.explainTerms()to extract and display watermark specificationsResolvedSchemaviaTableSourceTableVerifying this change
This change can be verified by:
Compilation: The module compiles successfully without errors
Create table with watermark:
Verify EXPLAIN output:
The output should display watermark information in the TableScan node.
Test without watermark: Verify that tables without watermarks still work normally
Does this pull request potentially affect
Documentation