Skip to content

[CALCITE-7353] introduce Milvus adapter with scan and convention support#4925

Open
wg1026688210 wants to merge 1 commit into
apache:mainfrom
wg1026688210:feat/milvus-core-foundation
Open

[CALCITE-7353] introduce Milvus adapter with scan and convention support#4925
wg1026688210 wants to merge 1 commit into
apache:mainfrom
wg1026688210:feat/milvus-core-foundation

Conversation

@wg1026688210
Copy link
Copy Markdown

Add the foundational Milvus adapter module to Apache Calcite:

  • MilvusRel convention interface and MilvusTableScan operator
  • MilvusSchema / MilvusSchemaFactory for schema integration
  • MilvusTranslatableTable with scan() for collection reads
  • MilvusToEnumerableConverter and converter rule for code generation
  • MilvusQueryEnumerator with pagination via MilvusRowIterator
  • MilvusProjectExpression for projection handling
  • Test infrastructure: E2E base, JUnit 5 extension, Testcontainers

This is the minimal skeleton; Filter/Project/VectorSearch rules are intentionally omitted for subsequent PRs.

@wg1026688210 wg1026688210 force-pushed the feat/milvus-core-foundation branch 5 times, most recently from 8439d84 to 53eab9c Compare May 9, 2026 05:51
@cjj2010
Copy link
Copy Markdown
Contributor

cjj2010 commented May 9, 2026

CI error:
"Caused by: java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration"
It seems to be related to the Docker environment. Alternatively, you can check how the Redis adapter builds the Docker environment

@wg1026688210 wg1026688210 force-pushed the feat/milvus-core-foundation branch from 53eab9c to 73dbcde Compare May 26, 2026 01:38
@wg1026688210 wg1026688210 force-pushed the feat/milvus-core-foundation branch from 73dbcde to 88fd7ca Compare May 26, 2026 03:42
@sonarqubecloud
Copy link
Copy Markdown

@wg1026688210
Copy link
Copy Markdown
Author

@cjj2010 @xuzifu666 @mihaibudiu cc ,the pr is ready

Copy link
Copy Markdown
Contributor

@mihaibudiu mihaibudiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was expecting to see some kind of README for such a large project, explaining what Milvus is, how it relates to Calcite, and how people can use the adapter. Or perhaps there's some other place for the adapters documentation? But I don't see any.

@xuzifu666
Copy link
Copy Markdown
Member

I was expecting to see some kind of README for such a large project, explaining what Milvus is, how it relates to Calcite, and how people can use the adapter. Or perhaps there's some other place for the adapters documentation? But I don't see any.

Thank you for the contribution @wg1026688210 ! As @mihaibudiu mentioned, we can first add some background and value information in Jira. Currently, we support many adapters, and the universality and uniqueness of adapters are important reference indicators for introducing adapters.

@wg1026688210
Copy link
Copy Markdown
Author

I was expecting to see some kind of README for such a large project, explaining what Milvus is, how it relates to Calcite, and how people can use the adapter. Or perhaps there's some other place for the adapters documentation? But I don't see any.

Thank you for the contribution @wg1026688210 ! As @mihaibudiu mentioned, we can first add some background and value information in Jira. Currently, we support many adapters, and the universality and uniqueness of adapters are important reference indicators for introducing adapters.
Hi~ @xuzifu666 @mihaibudiu Thanks for your feedback. There have been some discussions in this Jira.

@xiedeyantu
Copy link
Copy Markdown
Member

Thank you for the contribution @wg1026688210 ! Although I don't know much about Milvus, I think this adapter is very valuable. As @zabetak stated in CALCITE-6908, the test coverage of the milvus adaptor should ideally reach 80%. I suggest including test cases along with the feature descriptions and adding the test cases mentioned in the descriptions to the code. This will make it easier for reviewers to review the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants