Fix multi-CTE rendering under --empty and unit-test casts on string columns#65
Conversation
…olumns
Two bugs in the current adapter prevent dbt-core 1.11+ from compiling
common project shapes against SingleStore. Both are fixed with surgical
macro overrides plus matching regression tests.
1. singlestore__strip_db_limit_aliases corrupted multi-CTE models when
running `dbt run --empty`. dbt-core rewrites every {{ ref(...) }} as
`_dbt_limit_subq_<n>`, and the cleanup macro searched forward globally
for the trailing ` as `. In multi-CTE models that match was the next
CTE's `as (`, splicing across CTE boundaries and producing invalid
SQL such as `(...) as (` with the next CTE's alias swallowed. The
trailing ` as ` lookup is now bounded to the same physical SQL line.
2. singlestore__safe_cast rejected the Postgres-flavored type names that
arrive from dbt-core's unit-test fixture renderer. dbt-core calls
`safe_cast(value, column.data_type)` for every fixture value, and the
base `Column.data_type` normalises any varchar/text column to
"character varying(N)". dbt-core 1.11 additionally strips the length
when the value is a string (dbt-labs/dbt-core GH-11974, to prevent
silent truncation), producing a bare "character varying" form.
SingleStore's `!:>` rejects both forms outright, and a bare `!:> char`
silently truncates to a single character. The macro now translates
the Postgres-flavored names into SingleStore-native targets, mapping
the unbounded form to `longtext` so fixture values pass through
intact.
Tests:
- New TestSingleStoreEmptyMultiCTE in test_empty.py — a two-CTE model
that hits `dbt run --empty` and would fail on the previous macro.
- Subscribe to the upstream BaseUnitTestingVarcharFixtureNoTruncation
(regression test for dbt-labs/dbt-core#11974) in test_unit_testing.py,
with a thin override of the upstream model to use SingleStore's
native `:> varchar(5)` cast operator (the upstream fixture uses ANSI
`cast(x as varchar(5))`, which SingleStore CAST does not accept).
Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default mode and found 1 potential issue.
Reviewed by Cursor Bugbot for commit cdf76e5. Configure here.
The branch selection in singlestore__translate_safe_cast_type uses the
lowered form, but the previous replacements operated on `raw` with
hard-coded all-lower / all-upper literals. A mixed-case input such as
`Character Varying(10)` therefore reached the right branch via the
case-insensitive `startswith`, but neither `replace('character varying',
'varchar')` nor `replace('CHARACTER VARYING', 'varchar')` matched, and
the Postgres-flavored type passed through to the `!:>` cast unchanged —
which SingleStore rejects at runtime.
Replace on `lowered` instead, since SingleStore type names are
case-insensitive so emitting them in lower case is harmless. Add a
parametrised test (TestSingleStoreTranslateSafeCastType) that pins down
the case-insensitive contract end-to-end via execute_macro, covering
lower / mixed / upper / native / unrelated inputs.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
@pmishchenko-ua - can you please review? |
There was a problem hiding this comment.
Pull request overview
This PR fixes two adapter macro issues that block dbt-core 1.11+ workflows on SingleStore: (1) dbt run --empty compilation for multi-CTE models, and (2) unit-test fixture rendering when dbt-core supplies Postgres-flavored string type names to safe_cast.
Changes:
- Update
singlestore__strip_db_limit_aliasesto bound the' as 'search to the current physical SQL line, preventing cross-CTE splicing under--empty. - Update
singlestore__safe_castto translate Postgres-flavoredcharacter varying(...)/character(...)types into SingleStore-valid cast targets (including mapping unbounded forms tolongtext). - Add/extend functional tests covering both regressions, and document the fixes in the changelog.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
dbt/include/singlestore/macros/common.sql |
Fixes _dbt_limit_subq_ alias stripping so multi-CTE SQL isn’t corrupted under dbt run --empty. |
dbt/include/singlestore/macros/utils/safe_cast.sql |
Adds a helper to translate Postgres-flavored string types for SingleStore’s !:> casting, avoiding parse errors/truncation in unit tests. |
tests/functional/adapter/test_empty.py |
Adds a regression test ensuring dbt run --empty works for a multi-CTE model. |
tests/functional/adapter/test_unit_testing.py |
Subscribes to the upstream varchar fixture no-truncation test and adds a direct test for the new translation helper. |
CHANGELOG.MD |
Documents both fixes under an Unreleased section. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Hi @hibob-rom! Thanks for catching this and contributing to our connector. I’ve reviewed the changes, and everything looks good to me. I’ll merge this PR, make one additional small change, and then publish a new PyPI release. |

Problem
Two bugs in the current adapter prevent dbt-core 1.11+ from compiling common project shapes against SingleStore. Both are reproducible on
master @ 0eadd34againstdbt-coreHEAD (1.12.0a1) +dbt-adapters1.23.0, and one of them already breaks an existing test in this repo (TestSingleStoreUnitTestingTypes) — see "Reproductions" below.Bug 1 —
singlestore__strip_db_limit_aliasescorrupts multi-CTE models underdbt run --emptydbt run --emptyrewrites every{{ ref(...) }}as a_dbt_limit_subq_<n>limited subquery. The current cleanup macro searches forward globally for the next' as '. In multi-CTE models that match is the next CTE'sas (, so the rewrite splices across CTE boundaries and produces invalid SQL such as(...) as (with the next CTE's alias swallowed.Minimal repro (fails on
master, passes on this branch):```sql
-- multi_cte_model.sql
with a as (select id, name from {{ ref('upstream_a') }}),
b as (select id, score from {{ ref('upstream_b') }})
select a.id, a.name, b.score from a join b on a.id = b.id
```
```
$ dbt run --empty
Database Error in model multi_cte_model
1064: You have an error in your SQL syntax; check the manual that corresponds to
your MySQL server version for the right syntax to use near
'(\n select id, score from (select * from
dbt_test.upstream_bwhere false l'```
Bug 2 —
singlestore__safe_castrejects Postgres-flavored type names from unit-test fixture castsdbt-core's unit-test fixture renderer calls
safe_cast(value, column.data_type)for every fixture value. The baseColumn.data_typenormalises any varchar/text column to the Postgres-style"character varying(N)". dbt-core 1.11+ additionally strips the length when the value is a string (dbt-labs/dbt-core#11974, to prevent silent truncation), producing a bare"character varying"form. SingleStore's!:>rejects both forms outright, and a bare!:> charsilently truncates to a single character.Repro: this repo's existing
TestSingleStoreUnitTestingTypesagainstdbt-coreHEAD already fails onmasterwith:```
Database Error
1064: You have an error in your SQL syntax; check the manual that corresponds to
your MySQL server version for the right syntax to use near ' character varying)
as
tested_column) select tested_column from dbt'
```
Solution
Two surgical macro changes plus matching tests.
singlestore__strip_db_limit_aliases(dbt/include/singlestore/macros/common.sql)The double-alias is always emitted by dbt-core on a single physical SQL line, so the trailing
' as 'lookup MUST be bounded to that same line. Two changes:sql.find(' as ', pos, line_end)to the current physical SQL line.singlestore__safe_cast(dbt/include/singlestore/macros/utils/safe_cast.sql)A new private helper
singlestore__translate_safe_cast_typetranslates Postgres-flavored type names to SingleStore-native cast targets only in thesafe_castpath (so contract / introspection paths that legitimately expect the Postgres-canonical names are untouched):character varying(N)varchar(N)character varyinglongtext(unbounded; no truncation)character(N)char(N)characterlongtext(unbounded; no truncation)longtextis used for the unbounded form because a bare!:> varcharis a SingleStore parse error and a bare!:> charsilently truncates to a single character (verified locally:('EMP001' !:> char)→'E').Tests
tests/functional/adapter/test_empty.py— newTestSingleStoreEmptyMultiCTE: a two-CTE model that runsdbt runfollowed bydbt run --empty. Fails before the fix, passes after.tests/functional/adapter/test_unit_testing.py— subscribed to the upstreamBaseUnitTestingVarcharFixtureNoTruncation(the regression test class for [Bug] Type casting size fail in Unit Test dbt-labs/dbt-core#11974), with a thinmodelsoverride to replace the upstream fixture's ANSIcast(x as varchar(5))with SingleStore's native('short' :> varchar(5))(SingleStore'sCASTdoes not acceptvarchar(N)as a target).Backward compatibility
Both macros are pure replacements of existing public macros; no signatures change. The
safe_casttranslator only narrows behavior for inputs the previous macro could not handle anyway. No adapter Python code is touched.Local validation
Against
dbt-coreHEAD +dbt-adapters1.23.0 + SingleStore 5.7.32:masterbefore the fix.tests/functional/adapter/test_empty.py+tests/functional/adapter/test_unit_testing.py— 7/7 green after the fix.test_basic,test_dbt_show,test_ephemeral,test_relations,test_dbt_debug,test_aliases,test_query_comment, severaltest_utilsclasses) — 48/51. The 3 remaining failures (TestIncrementalSingleStore,TestSnapshotCheckColsSingleStore,TestSnapshotTimestampSingleStore) and the 3test_caching.pyfailures are all pre-existing onmaster(verified by re-running them with the diff stashed) and require the per-suitedrop_and_create_new_dbisolation that.github/workflows/run-tests.shuses in CI.Changelog
Unreleasedsection added inCHANGELOG.MDdocumenting both fixes.Made with Cursor
Note
Medium Risk
Touches core SQL-rendering macros (
singlestore__strip_db_limit_aliasesandsinglestore__safe_cast) used broadly during compilation/execution; while changes are targeted and covered by new regressions tests, mistakes could impact SQL generation across many models.Overview
Fixes two SingleStore adapter macro regressions affecting dbt-core 1.11+.
singlestore__strip_db_limit_aliasesnow bounds its trailingassearch to the same physical SQL line and recurses by line when no double-alias is present, preventingdbt run --emptyfrom corrupting multi-CTE SQL.singlestore__safe_castnow routes cast targets through a newsinglestore__translate_safe_cast_typehelper that converts Postgres-flavored string types (e.g.character varying(N)/character varying) to SingleStore-valid targets (varchar(N)/longtext) to avoid compilation failures or truncation in unit-test fixtures. Adds/extends functional tests to lock in both behaviors and documents the fixes inCHANGELOG.MD.Reviewed by Cursor Bugbot for commit e36d7e2. Bugbot is set up for automated code reviews on this repo. Configure here.