Skip to content

Fix multi-CTE rendering under --empty and unit-test casts on string columns#65

Merged
okramarenko merged 2 commits into
memsql:masterfrom
hibob-rom:rom/fix-empty-multi-cte-and-unit-test-casts
May 18, 2026
Merged

Fix multi-CTE rendering under --empty and unit-test casts on string columns#65
okramarenko merged 2 commits into
memsql:masterfrom
hibob-rom:rom/fix-empty-multi-cte-and-unit-test-casts

Conversation

@hibob-rom
Copy link
Copy Markdown
Contributor

@hibob-rom hibob-rom commented May 10, 2026

Problem

Two bugs in the current adapter prevent dbt-core 1.11+ from compiling common project shapes against SingleStore. Both are reproducible on master @ 0eadd34 against dbt-core HEAD (1.12.0a1) + dbt-adapters 1.23.0, and one of them already breaks an existing test in this repo (TestSingleStoreUnitTestingTypes) — see "Reproductions" below.

Bug 1 — singlestore__strip_db_limit_aliases corrupts multi-CTE models under dbt run --empty

dbt run --empty rewrites every {{ ref(...) }} as a _dbt_limit_subq_<n> limited subquery. The current cleanup macro searches forward globally for the next ' as '. In multi-CTE models that match is the next CTE's as (, so the rewrite splices across CTE boundaries and produces invalid SQL such as (...) as ( with the next CTE's alias swallowed.

Minimal repro (fails on master, passes on this branch):

```sql
-- multi_cte_model.sql
with a as (select id, name from {{ ref('upstream_a') }}),
b as (select id, score from {{ ref('upstream_b') }})
select a.id, a.name, b.score from a join b on a.id = b.id
```

```
$ dbt run --empty
Database Error in model multi_cte_model
1064: You have an error in your SQL syntax; check the manual that corresponds to
your MySQL server version for the right syntax to use near
'(\n select id, score from (select * from dbt_test.upstream_b where false l'
```

Bug 2 — singlestore__safe_cast rejects Postgres-flavored type names from unit-test fixture casts

dbt-core's unit-test fixture renderer calls safe_cast(value, column.data_type) for every fixture value. The base Column.data_type normalises any varchar/text column to the Postgres-style "character varying(N)". dbt-core 1.11+ additionally strips the length when the value is a string (dbt-labs/dbt-core#11974, to prevent silent truncation), producing a bare "character varying" form. SingleStore's !:> rejects both forms outright, and a bare !:> char silently truncates to a single character.

Repro: this repo's existing TestSingleStoreUnitTestingTypes against dbt-core HEAD already fails on master with:

```
Database Error
1064: You have an error in your SQL syntax; check the manual that corresponds to
your MySQL server version for the right syntax to use near ' character varying)
as tested_column
) select tested_column from dbt'
```

Solution

Two surgical macro changes plus matching tests.

singlestore__strip_db_limit_aliases (dbt/include/singlestore/macros/common.sql)

The double-alias is always emitted by dbt-core on a single physical SQL line, so the trailing ' as ' lookup MUST be bounded to that same line. Two changes:

  1. Bound sql.find(' as ', pos, line_end) to the current physical SQL line.
  2. When no double-alias is present on the current line, leave the intermediate alias intact (it is valid SingleStore SQL on its own) and recurse on the remainder past the line break, so each recursion shrinks the input by at least one line.

singlestore__safe_cast (dbt/include/singlestore/macros/utils/safe_cast.sql)

A new private helper singlestore__translate_safe_cast_type translates Postgres-flavored type names to SingleStore-native cast targets only in the safe_cast path (so contract / introspection paths that legitimately expect the Postgres-canonical names are untouched):

Input Output
character varying(N) varchar(N)
character varying longtext (unbounded; no truncation)
character(N) char(N)
character longtext (unbounded; no truncation)
anything else pass-through

longtext is used for the unbounded form because a bare !:> varchar is a SingleStore parse error and a bare !:> char silently truncates to a single character (verified locally: ('EMP001' !:> char)'E').

Tests

  • tests/functional/adapter/test_empty.py — new TestSingleStoreEmptyMultiCTE: a two-CTE model that runs dbt run followed by dbt run --empty. Fails before the fix, passes after.
  • tests/functional/adapter/test_unit_testing.py — subscribed to the upstream BaseUnitTestingVarcharFixtureNoTruncation (the regression test class for [Bug] Type casting size fail in Unit Test dbt-labs/dbt-core#11974), with a thin models override to replace the upstream fixture's ANSI cast(x as varchar(5)) with SingleStore's native ('short' :> varchar(5)) (SingleStore's CAST does not accept varchar(N) as a target).

Backward compatibility

Both macros are pure replacements of existing public macros; no signatures change. The safe_cast translator only narrows behavior for inputs the previous macro could not handle anyway. No adapter Python code is touched.

Local validation

Against dbt-core HEAD + dbt-adapters 1.23.0 + SingleStore 5.7.32:

  • Both bugs reproduced on master before the fix.
  • tests/functional/adapter/test_empty.py + tests/functional/adapter/test_unit_testing.py — 7/7 green after the fix.
  • Wider smoke (test_basic, test_dbt_show, test_ephemeral, test_relations, test_dbt_debug, test_aliases, test_query_comment, several test_utils classes) — 48/51. The 3 remaining failures (TestIncrementalSingleStore, TestSnapshotCheckColsSingleStore, TestSnapshotTimestampSingleStore) and the 3 test_caching.py failures are all pre-existing on master (verified by re-running them with the diff stashed) and require the per-suite drop_and_create_new_db isolation that .github/workflows/run-tests.sh uses in CI.

Changelog

Unreleased section added in CHANGELOG.MD documenting both fixes.

Made with Cursor


Note

Medium Risk
Touches core SQL-rendering macros (singlestore__strip_db_limit_aliases and singlestore__safe_cast) used broadly during compilation/execution; while changes are targeted and covered by new regressions tests, mistakes could impact SQL generation across many models.

Overview
Fixes two SingleStore adapter macro regressions affecting dbt-core 1.11+.

singlestore__strip_db_limit_aliases now bounds its trailing as search to the same physical SQL line and recurses by line when no double-alias is present, preventing dbt run --empty from corrupting multi-CTE SQL.

singlestore__safe_cast now routes cast targets through a new singlestore__translate_safe_cast_type helper that converts Postgres-flavored string types (e.g. character varying(N)/character varying) to SingleStore-valid targets (varchar(N)/longtext) to avoid compilation failures or truncation in unit-test fixtures. Adds/extends functional tests to lock in both behaviors and documents the fixes in CHANGELOG.MD.

Reviewed by Cursor Bugbot for commit e36d7e2. Bugbot is set up for automated code reviews on this repo. Configure here.

…olumns

Two bugs in the current adapter prevent dbt-core 1.11+ from compiling
common project shapes against SingleStore. Both are fixed with surgical
macro overrides plus matching regression tests.

1. singlestore__strip_db_limit_aliases corrupted multi-CTE models when
   running `dbt run --empty`. dbt-core rewrites every {{ ref(...) }} as
   `_dbt_limit_subq_<n>`, and the cleanup macro searched forward globally
   for the trailing ` as `. In multi-CTE models that match was the next
   CTE's `as (`, splicing across CTE boundaries and producing invalid
   SQL such as `(...) as (` with the next CTE's alias swallowed. The
   trailing ` as ` lookup is now bounded to the same physical SQL line.

2. singlestore__safe_cast rejected the Postgres-flavored type names that
   arrive from dbt-core's unit-test fixture renderer. dbt-core calls
   `safe_cast(value, column.data_type)` for every fixture value, and the
   base `Column.data_type` normalises any varchar/text column to
   "character varying(N)". dbt-core 1.11 additionally strips the length
   when the value is a string (dbt-labs/dbt-core GH-11974, to prevent
   silent truncation), producing a bare "character varying" form.
   SingleStore's `!:>` rejects both forms outright, and a bare `!:> char`
   silently truncates to a single character. The macro now translates
   the Postgres-flavored names into SingleStore-native targets, mapping
   the unbounded form to `longtext` so fixture values pass through
   intact.

Tests:
- New TestSingleStoreEmptyMultiCTE in test_empty.py — a two-CTE model
  that hits `dbt run --empty` and would fail on the previous macro.
- Subscribe to the upstream BaseUnitTestingVarcharFixtureNoTruncation
  (regression test for dbt-labs/dbt-core#11974) in test_unit_testing.py,
  with a thin override of the upstream model to use SingleStore's
  native `:> varchar(5)` cast operator (the upstream fixture uses ANSI
  `cast(x as varchar(5))`, which SingleStore CAST does not accept).

Co-authored-by: Cursor <cursoragent@cursor.com>
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default mode and found 1 potential issue.

Fix All in Cursor

Reviewed by Cursor Bugbot for commit cdf76e5. Configure here.

Comment thread dbt/include/singlestore/macros/utils/safe_cast.sql Outdated
The branch selection in singlestore__translate_safe_cast_type uses the
lowered form, but the previous replacements operated on `raw` with
hard-coded all-lower / all-upper literals. A mixed-case input such as
`Character Varying(10)` therefore reached the right branch via the
case-insensitive `startswith`, but neither `replace('character varying',
'varchar')` nor `replace('CHARACTER VARYING', 'varchar')` matched, and
the Postgres-flavored type passed through to the `!:>` cast unchanged —
which SingleStore rejects at runtime.

Replace on `lowered` instead, since SingleStore type names are
case-insensitive so emitting them in lower case is harmless. Add a
parametrised test (TestSingleStoreTranslateSafeCastType) that pins down
the case-insensitive contract end-to-end via execute_macro, covering
lower / mixed / upper / native / unrelated inputs.

Co-authored-by: Cursor <cursoragent@cursor.com>
@hibob-rom
Copy link
Copy Markdown
Contributor Author

@pmishchenko-ua - can you please review?
Thanks

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes two adapter macro issues that block dbt-core 1.11+ workflows on SingleStore: (1) dbt run --empty compilation for multi-CTE models, and (2) unit-test fixture rendering when dbt-core supplies Postgres-flavored string type names to safe_cast.

Changes:

  • Update singlestore__strip_db_limit_aliases to bound the ' as ' search to the current physical SQL line, preventing cross-CTE splicing under --empty.
  • Update singlestore__safe_cast to translate Postgres-flavored character varying(...) / character(...) types into SingleStore-valid cast targets (including mapping unbounded forms to longtext).
  • Add/extend functional tests covering both regressions, and document the fixes in the changelog.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
dbt/include/singlestore/macros/common.sql Fixes _dbt_limit_subq_ alias stripping so multi-CTE SQL isn’t corrupted under dbt run --empty.
dbt/include/singlestore/macros/utils/safe_cast.sql Adds a helper to translate Postgres-flavored string types for SingleStore’s !:> casting, avoiding parse errors/truncation in unit tests.
tests/functional/adapter/test_empty.py Adds a regression test ensuring dbt run --empty works for a multi-CTE model.
tests/functional/adapter/test_unit_testing.py Subscribes to the upstream varchar fixture no-truncation test and adds a direct test for the new translation helper.
CHANGELOG.MD Documents both fixes under an Unreleased section.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@okramarenko
Copy link
Copy Markdown
Collaborator

Hi @hibob-rom! Thanks for catching this and contributing to our connector.

I’ve reviewed the changes, and everything looks good to me. I’ll merge this PR, make one additional small change, and then publish a new PyPI release.

@okramarenko okramarenko merged commit 2ac4416 into memsql:master May 18, 2026
11 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants