feat: add parameterized query output support#34
Merged
Conversation
Add a fully isolated, non-breaking parameterized query pipeline that returns SQL with bind placeholders (@p1/$1) plus parameter values instead of inlined literals. All existing functions remain untouched; new parallel *Param methods are created throughout the stack. - internal/params: ParamCollector, PlaceholderStyle, ValidatePlaceholderRefs - operators: ToSQLParam/valueToSQLParam for all operator types - parser: ParseParameterized pipeline with ParamExpressionParser callback - parameterized.go: public API (6 Transpiler methods + 6 package functions) - errors: E350 ErrUnreferencedPlaceholder for dropped-arg detection Made-with: Cursor
Unit and integration tests covering all parameterized methods across operators, parser, params package, and the public API. Includes tests for all 5 dialects, schema coercion, custom operators, nested expressions, error parity, and E350 unreferenced placeholder detection. Made-with: Cursor
- New docs/parameterized-queries.md with full guide (placeholder styles, schema coercion, database driver examples, custom operator contract) - Update README.md with parameterized query feature and example - Update api-reference.md with new methods, QueryParam type - Update getting-started.md, examples.md, dialects.md with param sections - Update error-handling.md with E350 entry Made-with: Cursor
Made-with: Cursor
REPL: - Add :params toggle command to switch between inline and parameterized output - printParams helper formats bind parameters for display - Both main input loop and :file command respect params mode WASM: - Add transpileParameterized, transpileConditionParameterized, quickTranspileParameterized JS bridge functions - Add Parameterized checkbox to playground UI with bind params display - Multi-dialect comparison also shows params when enabled Made-with: Cursor
- TestPrintParams: verifies formatted output for empty, single, multiple params - TestParamsModeToggle: verifies toggle state flip - TestTranspileParameterized_BigQuery: 6 cases (equality, numeric, AND, IN, null, bool) - TestTranspileParameterized_PostgreSQL: positional $1 placeholders - TestTranspileParameterized_CustomOperator: custom operators in parameterized mode - TestTranspileParameterized_Error: invalid JSON and unknown operator paths Made-with: Cursor
- README: add :params example to Interactive REPL section - docs/repl.md: add :params command and Parameterized Output section - docs/wasm-playground.md: add parameterized JS API examples and playground feature entry - Rebuild REPL and WASM binaries Made-with: Cursor
…gers - REPL LIKE operators (startsWith, endsWith, contains) now use CONCAT for bind placeholders instead of embedding them inside SQL string literals, which rendered placeholders non-functional - Large integer strings exceeding int64 range are preserved as strings in parameterized numeric path instead of being rounded via float64 Made-with: Cursor
…ecision - REPL parameterized LIKE tests for all 6 operators (startsWith, !startsWith, endsWith, !endsWith, contains, !contains) - Non-parameterized LIKE regression tests to ensure backward compat - Unit tests for large int64 overflow preservation in numeric operator - Integration test for large integer parameterized transpilation - Updated parseContainsArgs tests for raw SQL pattern return value Made-with: Cursor
- NewParser: update comment to reflect nil config fallback behavior - PlaceholderQuestion: document per-parameter validation limitation Made-with: Cursor
Made-with: Cursor
Document why LIKE patterns must concatenate bind placeholders (e.g., CONCAT(@p1, '%')) instead of embedding placeholder text in string literals, and add REPL-based revalidation notes. Made-with: Cursor
Add explicit guards for parameterized LIKE placeholder handling to ensure bind tokens are not quoted in pattern literals across dialects, and add a large-integer preservation case for numeric parameterized output. Made-with: Cursor
Document the newly added test coverage and validation commands so the runtime findings remain traceable in local PR review artifacts. Made-with: Cursor
Drop parameterized-query-output-pr-review.md from the branch so local review notes are not included in repository commits or PR diff. Made-with: Cursor
…n bugs - Defer leftSQL placeholder generation in handleInParam so the placeholder is created only after coercion decisions are made, preventing orphaned parameters that triggered false E350 errors - Use the Go type of the original left value (string vs non-string) instead of SQL quoting to distinguish string containment from array membership, fixing misclassification when placeholders replaced quoted literals Made-with: Cursor
- Update handleInParam unit tests for refactored signature - Add string-containment-without-schema and schema-coercion unit tests - Add integration tests for string containment across dialects - Add integration tests for numeric-to-string coercion with schema Made-with: Cursor
Made-with: Cursor
…pe LIKE wildcards in parameterized mode Route literal string/number right-hand sides in the 'in' operator through strposFunc instead of hardcoded POSITION(), ensuring dialect-correct SQL (STRPOS for BigQuery/Spanner/DuckDB, POSITION for PostgreSQL, position() for ClickHouse). Wrap LIKE placeholders in SQL REPLACE calls to escape %, _, and \ metacharacters at query time, matching non-parameterized escaping semantics. Made-with: Cursor
…aping Update in-operator test expectations to reflect dialect-aware function names. Add dialect compliance test for literal-on-right string containment across all 5 dialects. Update REPL LIKE test expectations to include REPLACE-based wildcard escaping in parameterized mode. Made-with: Cursor
Made-with: Cursor
…rameterized mode
Replace innerHTML with textContent + DOM element creation in the WASM
playground's parameterized output path to prevent script injection from
crafted JSONLogic string literals.
Add placeholder-aware argument detection (isPlaceholder/isBoundValue) to
parseContainsArgs so reversed contains/!contains arguments like
{"contains": ["foo", {"var": "name"}]} are correctly identified in
parameterized mode where literals become @p1/$1 placeholders.
Made-with: Cursor
Add parseContainsArgs test cases for reversed placeholder arguments (@p1, $1). Add integration tests verifying contains/!contains with reversed args produce correct column LIKE pattern SQL in both non-parameterized and parameterized modes. Made-with: Cursor
Made-with: Cursor
…rized mode When a numeric string exceeds int64 range, store it as *big.Int in the QueryParam value instead of a plain string. Database drivers (BigQuery, PostgreSQL, Spanner) bind *big.Int as a numeric type, avoiding runtime type errors when the parameter appears in arithmetic or comparison contexts (e.g. @p1 * @p2, field >= $1). Made-with: Cursor
Update unit and integration test expectations to verify overflow integers are stored as *big.Int. Add big.Int-aware comparison helpers (numericParamValuesEqual, assertParams via reflect.DeepEqual) for correct pointer-value equality checks. Made-with: Cursor
Made-with: Cursor
Split buildLikeSQL into three branches: SQL string literals (inline), bind placeholders (CONCAT+REPLACE), and unquoted primitives like numeric or boolean values (inline as literal). Previously, unquoted primitives fell into the placeholder branch, producing invalid SQL like REPLACE(1000, ...) where REPLACE requires text arguments. Made-with: Cursor
…ed mode Verify that contains and startsWith with numeric JSON values (1000, 404) produce valid inlined LIKE SQL rather than invalid REPLACE wrapping. Made-with: Cursor
Made-with: Cursor
…idation Add schemaProvider() helper that returns a true nil interface when *Schema is nil, avoiding Go's typed-nil interface pitfall. Without this, passing a nil *Schema pointer through config or SetSchema creates a non-nil SchemaProvider interface, causing convertVarName to skip the identifier regex whitelist and accept invalid names like "bad field". Made-with: Cursor
Made-with: Cursor
…ng storage Made-with: Cursor
Switch JSON decoding from json.Unmarshal (which coerces all numbers to float64) to json.Decoder.UseNumber, preserving numbers as json.Number throughout the pipeline. This fixes precision loss for integers exceeding 2^53 (e.g., 9223372036854775808 and 9223372036854775809 previously produced identical SQL). Add json.Number handling to all operator paths: isPrimitive, isNumber, valueToSQL, valueToSQLParam, getNumber, coerceValueForComparison, handleIn/handleInParam, and primitiveToSQLParam. Parameterized behavior: JS-safe integers (within ±2^53-1) bind as float64 for backward compatibility; overflow integers bind as string to preserve precision. Made-with: Cursor
Add tests verifying distinct large integers produce distinct SQL and correct parameterized string values. Update existing expectations for json.Number rendering: 9999999999999 (no longer scientific notation), 1e10 (original JSON form), 1.0 (preserved decimal). Made-with: Cursor
Made-with: Cursor
…andling Made-with: Cursor
In primitiveToSQLParam, format []interface{} values using fmt.Sprintf
instead of parameterizing them as bind values. This matches the
non-param primitiveToSQL behavior and ensures custom operators (e.g.
REPL contains) receive the same "[x]" string form in both modes,
allowing parseContainsArgs to detect and unwrap array patterns.
Previously, arrays were stored as []interface{} bind values (e.g.
@p1 = [x]), causing type errors when CAST/REPLACE was applied in
parameterized LIKE expressions.
Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
jsonNumberParamValue now falls back to the original numeric string instead of erroring when float64 conversion overflows (1e309) or underflows (1e-400). Also adds underflow detection in both data.jsonNumberParamValue and parser.primitiveToSQLParam to prevent silently binding non-zero values as 0. This aligns parameterized output with inline transpilation which preserves these literals verbatim. Made-with: Cursor
Covers both operator path (jsonNumberParamValue) and custom operator path (primitiveToSQLParam) with 1e309 overflow and 1e-400 underflow cases, asserting the original string is preserved as the param value. Made-with: Cursor
Made-with: Cursor
Add rows for overflow (1e309) and underflow (1e-400) floats to the QueryParam.Value type table. Update the UseNumber note to cover float range limits alongside integer precision. Made-with: Cursor
…om operators Add FindQuotedPlaceholderRef guard that detects when a custom operator wraps a placeholder inside a SQL string literal (e.g. '@p1'), which silently breaks bind semantics. The guard runs after custom operator SQL generation in parameterized mode and fails with E102 ErrCustomOperatorFailed. Valid expressions like CONCAT(@p1, '%') continue to work. Made-with: Cursor
Unit tests for FindQuotedPlaceholderRef covering named/positional placeholders, partial-match rejection, escaped quotes, and clean SQL. Integration tests for parameterized custom operators: rejection of quoted placeholders and acceptance of expression-based usage. Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
TranspileFromMap/TranspileFromInterface accepted crafted json.Number
values (e.g. json.Number("1 OR 1=1")) and inlined them verbatim,
enabling SQL injection. Add validJSONNumberLiteral regex that enforces
the JSON number grammar before emission in both data.valueToSQL and
numeric.valueToSQL, plus their parameterized counterparts. The
Transpile(string) path is not affected since json.Decoder only
produces valid numeric tokens.
Made-with: Cursor
Unit tests in data_test.go for valueToSQL and valueToSQLParam with valid and invalid json.Number inputs. Integration tests in transpiler_test.go for TranspileFromMap/TranspileFromInterface rejection. All-dialect parameterized custom operator tests in parameterized_test.go for quoted-placeholder detection. Made-with: Cursor
Made-with: Cursor
… injection path
The validator's isNumber() accepted any json.Number, allowing crafted
values like json.Number("1 OR 1=1") through TranspileFromMap to reach
custom operators unvalidated. Now validates against the JSON number
grammar before accepting, catching the issue at the earliest defense
point for all code paths (built-in and custom operators).
Made-with: Cursor
…tor path Validator tests for valid/invalid json.Number primitives and custom operator arguments. Integration test confirming TranspileFromMap with a custom operator rejects crafted json.Number literals. Made-with: Cursor
Made-with: Cursor
…param regex - Switch bytes.NewBufferString to strings.NewReader in decodeJSONLogic to avoid unnecessary byte slice allocation. - Copy arrays before schema coercion in handleIn/handleInParam to prevent in-place mutation of caller-provided slices. - Replace per-parameter regex compilation in ValidatePlaceholderRefs with boundary-aware string scanning, reusing hasPlaceholderBoundaries. Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@p1,$1) plus a separate parameter list, instead of inlined literals@p1) for BigQuery/Spanner/ClickHouse, positional ($1) for PostgreSQL/DuckDB"50000"->int64(50000)for integer fields)E350 ErrUnreferencedPlaceholdererror code catches custom operators that drop argumentsTranspilermethods + 6 package-level convenience functions mirroring existing inline API:paramstoggle command switches between inline and parameterized outputtranspileParameterized,transpileConditionParameterized,quickTranspileParameterizedJS functions + playground UI checkboxjson.Decoder.UseNumber()to preserve exact integer values. Unquoted integers like9223372036854775808now produce exact SQL instead of lossy float64 representations.What gets parameterized
NULL,TRUE,FALSEvar)Example
REPL
Architecture
All existing functions remain untouched. New parallel
*Parammethods are created throughout the stack:internal/params/—ParamCollector,PlaceholderStyle,ValidatePlaceholderRefs,FindQuotedPlaceholderRef,ValueForPlaceholderinternal/operators/*.go—ToSQLParam()/valueToSQLParam()for all operator typesinternal/parser/parser.go—ParseParameterizedpipeline withParamExpressionParsercallback + quoted-placeholder guard for custom operatorsparameterized.go— publicQueryParamtype and all 12 public functionsinternal/errors/errors.go—E350 ErrUnreferencedPlaceholdercmd/repl/main.go—:paramstoggle,printParamshelper, parameterized-aware custom operatorsdemo/wasm/main.go— 3 new JS bridge functionsdemo/wasm/index.html— Parameterized checkbox with bind params displayPost-review fixes
The following issues were identified during code review and fixed in follow-up commits:
Large integer precision (
json.Decoder.UseNumber)json.Unmarshalcoerces all JSON numbers tofloat64, losing precision for integers exceeding 2^53. For example,9223372036854775808and9223372036854775809produced identical SQL.json.Decoder.UseNumber()which preserves numbers asjson.Number(string-backed). Addedjson.Numberhandling to all operator paths (isPrimitive,isNumber,valueToSQL,valueToSQLParam,getNumber,coerceValueForComparison,handleIn/handleInParam,primitiveToSQLParam).float64for backward compatibility; overflow integers bind asstring.Out-of-range float preservation
jsonNumberParamValueerrored on float64 overflow (e.g.,1e309) and silently bound underflow values (e.g.,1e-400) as0, while inline transpilation preserved these literals verbatim.jsonNumberParamValueto fall back to the original numeric string whenParseFloatproduces infinity or underflow-to-zero. Added the same underflow guard toparser.primitiveToSQLParam(separate conversion path used by custom operators). Both paths now match inline behavior.Quoted-placeholder guard for custom operators
@p1) as plain string arguments. A naive operator could wrap them in SQL string literals (e.g.,'@p1'), which passes validation but silently breaks bind semantics at runtime.FindQuotedPlaceholderRefininternal/params/that scans SQL string literals for placeholder tokens with proper boundary checking (avoids false positives like'name@p1.example'). The parser now runs this guard after custom operator SQL generation in parameterized mode, failing withE102 ErrCustomOperatorFailedif a placeholder is found inside quotes. Valid expressions likeCONCAT(@p1, '%')continue to work.Parameterized
inoperatorProcessedValue{IsSQL:true}from custom operators (e.g.,'foo') was misrouted to array membership (IN UNNEST) instead of string containment (STRPOS). Fixed by mirroring the non-param quoting heuristic.@p1) lost type information. AddedParamCollector.ValueForPlaceholder()to look up the Go type of the stored parameter and route string values to containment.strposFuncrouting: BothhandleInandhandleInParamnow use dialect-awarestrposFuncfor literal string/number containment (e.g.,STRPOSon BigQuery,POSITION(x IN y)on PostgreSQL).handleInParamnow defers placeholder generation until after schema-based coercion is applied.REPL custom LIKE operators
startsWith,endsWith,containsand negations) were embedding placeholders inside SQL string literals ('@p1%'). Refactored withbuildLikeSQLhelper to useCONCAT(@p1, '%').%,_,\at runtime via SQLREPLACEfunctions.CAST(@p1 AS STRING)/CAST($1 AS TEXT)) beforeREPLACEto prevent type errors.containsargs: AddedisBoundValue()helper for robust column-vs-pattern detection in parameterized mode.["x"]) in parameterized mode are now formatted as strings inprimitiveToSQLParamso custom operators can detect and unwrap them consistently.BigQuery
regexpContainsr'...'prefix is only valid for inline SQL literals. In parameterized mode, the pattern is now emitted without therprefix to avoid invalid SQL and E350 errors.Security
innerHTMLwithtextContent+ DOM element creation for rendering SQL output in the WASM playground.json.NumberSQL injection via map/interface APIs:TranspileFromMap/TranspileFromInterfaceaccepted craftedjson.Numbervalues (e.g.,json.Number("1 OR 1=1")) and inlined them verbatim, enabling SQL injection. Defense-in-depth at two layers: (1)validJSONNumberLiteralregex indata.valueToSQL,numeric.valueToSQL, and their parameterized counterparts rejects invalid literals at the operator level; (2) the validator'sisNumber()now validatesjson.Numberagainst the JSON number grammar, catching invalid values at the earliest point before any operator (built-in or custom) sees them. TheTranspile(string)path was not affected sincejson.Decoderonly produces valid numeric tokens.Typed-nil schema fix
SetSchema(nil)andNewTranspilerWithConfigwith a nil*Schemapointer now correctly return a truenilinterface, ensuring no-schema identifier validation (regex whitelist) is not bypassed.Code quality and optimizations
bytes.NewBufferStringtostrings.NewReaderindecodeJSONLogicto avoid unnecessary byte slice copy.handleInandhandleInParamnow copy arrays before schema coercion to prevent mutating caller-provided slices.regexp.CompileinValidatePlaceholderRefswith boundary-awarestrings.Indexscanning.json.Marshalcalls indemo/wasm/main.gonow handle errors instead of swallowing them.Documentation
QueryParam.Valuetype documentation to cover unquoted large JSON integers, quoted string inputs, and out-of-range floats, withUseNumberexplanation.Test plan
internal/paramspackage (collector, styles, validator,ValueForPlaceholder,FindQuotedPlaceholderRef)ToSQLParammethods (data, comparison, numeric, logical, string, array)ParseParameterizedtests across BigQuery and PostgreSQL dialectsparameterized_test.gocovering all operators, all 5 dialects, schema coercion, custom operators, nested expressions1e309overflow and1e-400underflow preserve original string in both operator and custom operator paths'@p1'patterns and acceptance ofCONCAT(@p1, '%')expressionsjson.Numberinjection rejection:TranspileFromMap/TranspileFromInterfacereject crafted non-numericjson.Numbervalues in both built-in and custom operator pathsE350detection test for custom operators that drop argumentsprintParamsformatting, params mode toggle, parameterized transpile, custom LIKE operators, reversed args,regexpContainsBigQuery, array-pattern containsNewTranspilerWithConfignil pointer,SetSchema(nil)restore)instring containment and numeric boundaries across all 5 dialectsmake lint— 0 issuesmake test— all 8 packages passmake bench— benchmarks passmake build/make build/wasm— successful