If the table that an iceberg query is defined on is a table source (i.e. CREATE TABLE with iceberg mutation or explicitly defined iceberg source), we should optimize the DAG assembly to query directly against the source table instead of materializing the source table into an intermediate table (which is what the DAG planner does by default) since that is wasteful.
Once we support that, we would support vertx only deployments that only run DuckDB directly against Iceberg source tables (no need for Flink or an explicit database). This requires adjustments to how we construct the deployment artifiacts (e.g. if the Flink STATEMENT SET is empty, we don't generate a flink plan) and we need to support not having flink as an enabled engine.
If the table that an iceberg query is defined on is a table source (i.e.
CREATE TABLEwith iceberg mutation or explicitly defined iceberg source), we should optimize the DAG assembly to query directly against the source table instead of materializing the source table into an intermediate table (which is what the DAG planner does by default) since that is wasteful.Once we support that, we would support vertx only deployments that only run DuckDB directly against Iceberg source tables (no need for Flink or an explicit database). This requires adjustments to how we construct the deployment artifiacts (e.g. if the Flink STATEMENT SET is empty, we don't generate a flink plan) and we need to support not having flink as an enabled engine.