Skip to content

CQL Function Handling#176

Draft
brandtnewton wants to merge 4 commits into
GoogleCloudPlatform:mainfrom
brandtnewton:proxy/function-refactor
Draft

CQL Function Handling#176
brandtnewton wants to merge 4 commits into
GoogleCloudPlatform:mainfrom
brandtnewton:proxy/function-refactor

Conversation

@brandtnewton
Copy link
Copy Markdown
Collaborator

Adds proper support for CQL function handling: minTimeuuid, maxTimeuuid, etc.

@brandtnewton brandtnewton force-pushed the proxy/function-refactor branch from 39ec6c7 to 8da6a73 Compare March 17, 2026 20:38
@brandtnewton brandtnewton marked this pull request as draft March 17, 2026 20:39
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly upgrades the Cassandra-Bigtable proxy's ability to process and translate CQL queries involving various built-in functions. The changes introduce a robust framework for handling functions like minTimeuuid, maxTimeuuid, and aggregate operations, ensuring more accurate and compliant query execution. The underlying architecture for parsing and type management has been refactored to be more extensible and maintainable, laying the groundwork for future enhancements.

Highlights

  • Enhanced CQL Function Support: Introduced comprehensive support for a variety of CQL functions, including aggregate functions (AVG, SUM, MIN, MAX, COUNT), time-related functions (NOW, TOTIMESTAMP, MINTIMEUUID, MAXTIMEUUID), and WRITETIME. This significantly expands the proxy's capability to handle more complex CQL queries.
  • Refactored Function Handling Architecture: Implemented a new CqlFunc interface and associated types (CqlFuncCode, CqlFuncParameter) to standardize how CQL functions are defined, validated, and applied. This modular approach improves extensibility and maintainability for future function additions.
  • Updated CQL Grammar and Parsing Logic: Modified the ANTLR4 grammar (CqlParser.g4) to correctly parse the new function syntax and introduced new relation types for comparisons involving functions and columns (e.g., relationColumnCompareFunction, relationFunctionCompareColumn). The parsing utilities were updated to leverage this new grammar.
  • Centralized Dynamic Value and Parameter Handling: Refactored DynamicValue interface and related types to provide a more unified way of representing various query elements (literals, parameters, functions, columns). Utility functions for extracting values from QueryParameterValues were moved and integrated as methods within the QueryParameterValues struct, improving data encapsulation.
  • Improved Type Safety and Validation: Added CqlDataType to ParameterizedValue and LiteralValue to ensure type information is consistently carried through the query processing. New validation logic for function arguments was introduced to enforce correct usage based on defined parameter types and allowed clauses.
Changelog
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/bigtable/bigtable_select.go
    • Updated column type retrieval to use the new GetType() method on SelectedColumn.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/bigtable/utils.go
    • Imported github.com/google/uuid for UUID operations.
    • Refactored addBindValueIfNeeded to use DynamicValue interface methods and handle uuid.UUID binary marshalling.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/cql_func_avg.go
    • Added new file implementing the FuncAvg CQL function.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/cql_func_count.go
    • Added new file implementing the FuncCount CQL function.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/cql_func_max.go
    • Added new file implementing the FuncMax CQL function.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/cql_func_max_timeuuid.go
    • Added new file implementing the FuncMaxTimeuuid CQL function, including logic for generating maximum UUIDv1 for a given time.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/cql_func_min.go
    • Added new file implementing the FuncMin CQL function.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/cql_func_min_timeuuid.go
    • Added new file implementing the FuncMinTimeuuid CQL function, including logic for generating minimum UUIDv1 for a given time.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/cql_func_now.go
    • Added new file implementing the FuncNow CQL function.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/cql_func_sum.go
    • Added new file implementing the FuncSum CQL function.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/cql_func_to_timestamp.go
    • Added new file implementing the FuncToTimestamp CQL function.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/cql_func_writetime.go
    • Added new file implementing the FuncWritetime CQL function.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/cql_funcs.go
    • Added new file defining the CqlFunc interface, CqlFuncCode enum, CqlFuncParameter struct, and a central registry for all CQL functions.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/cql_functions.go
    • Removed file as its functionality was refactored into cql_funcs.go and individual function files.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/cql_types.go
    • Added AllScalarTypes and AllNumericTypes slices for convenient type grouping.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/dynamic_value.go
    • Modified DynamicValue interface to include GetParameter() and GetType() methods.
    • Introduced new DynamicValue implementations: FunctionValue, ColumnValue, SelectStarValue, MapAccessValue, ListElementValue.
    • Removed TimestampNowValue and TimeuuidNowValue as their logic is now handled by FunctionValue.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/queries.go
    • Added FlipOperator function to invert comparison operators.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/query_parameters.go
    • Added IsInternal field to ParameterMetadata to distinguish user-facing parameters.
    • Introduced CountUserParameters() and OrderedUserParams() methods to QueryParameters.
    • Added GetValueInt32, GetValueInt64, GetValueTime, GetValueSlice, GetValueMap methods to QueryParameterValues for direct value extraction.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/query_parameters_builder.go
    • Added AddInternalParam for creating internal query parameters.
    • Updated newParameterMetadata calls to include the isInternal flag.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/select_query.go
    • Refactored SelectedColumn struct to use the new DynamicValue interface, simplifying its structure and constructors.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/global/types/types.go
    • Added QueryClause enum to categorize where a function can be used (SELECT, WHERE, VALUES).
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/mem_table/in_mem_select.go
    • Updated value extraction for limits and aggregates to use the new methods on QueryParameterValues.
    • Adapted aggregate function handling to the new FunctionValue structure.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/responsehandler/responsehandler_utils.go
    • Updated buildPreparedResult to use OrderedUserParams() to only expose user-defined parameters.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/testing/compliance/compliance_uuid_test.go
    • Added TestValidateMaxAndMin to verify minTimeuuid and maxTimeuuid function behavior.
    • Expanded TestMaxAndMinTimestamp with additional test cases for comparison operators with minTimeuuid and maxTimeuuid.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/third_party/cqlparser/CqlParser.g4
    • Modified grammar to support relationColumnCompareFunction and relationFunctionCompareColumn.
    • Generalized functionCall and functionArgs rules to allow various argument types including columns and STAR.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/third_party/cqlparser/cqlparser_base_listener.go
    • Updated base listener to reflect new grammar rules for function comparisons.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/third_party/cqlparser/cqlparser_listener.go
    • Updated listener interface to reflect new grammar rules for function comparisons.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/common/binding.go
    • Removed utilities import.
    • Updated value extraction to use QueryParameterValues methods.
    • Simplified BindSelectColumns to directly handle DynamicValue types.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/common/btql_functions.go
    • Removed file as function parsing logic was moved to translators/common/utils.go.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/common/rowkeys_test.go
    • Updated NewLiteralValue calls to include CqlDataType for improved type accuracy in tests.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/common/utils.go
    • Added slices import.
    • Updated ParseDecimalLiteral, ParseMarker, ParseTupleValue, ParseConstantValue to include CqlDataType when creating LiteralValue or ParameterizedValue.
    • Introduced parseColumnCompareFunction for handling function-column comparisons in WHERE clauses.
    • Updated ParseValueAny to correctly parse FunctionCall contexts.
    • Introduced ParseFunctionCall, ParseFunctionName, and parseFunctionArg to centralize function parsing logic.
    • Refactored ParseSelectIndex, ParseSelectColumn, ParseSelectFunction to align with the new DynamicValue and CqlFunc interfaces.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/common/utils_test.go
    • Updated NewParameterizedValue and NewLiteralValue calls to include CqlDataType for type consistency in tests.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/delete_translator/translator_delete_test.go
    • Updated NewLiteralValue and NewParameterizedValue calls to include CqlDataType.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/insert_translator/translator_insert.go
    • Modified parseInsertValues call to pass the table schema for context.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/insert_translator/translator_insert_test.go
    • Updated NewParameterizedValue and NewLiteralValue calls to include CqlDataType.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/insert_translator/utils.go
    • Updated parseInsertValues to accept table and pass it to common.ParseValueAny.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/select_translator/select_query_builder.go
    • Refactored createBtqlSelectClause to directly use DynamicValue for column representation.
    • Introduced toBtql function to convert DynamicValue to BTQL string representation.
    • Refactored createBtqlFunc to handle FunctionValue and generate appropriate BTQL for various functions.
    • Removed ToBtql function, integrating its logic into the new toBtql.
    • Updated createBigtableSql to reflect changes in select clause and limit value handling.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/select_translator/translator_select.go
    • Updated parseSelectClause call to pass params.
    • Updated GetValueInt32 call to use the method on QueryParameterValues.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/select_translator/translator_select_test.go
    • Added NewSelectedColumnFunction helper for creating function-based selected columns in tests.
    • Updated NewSelectedColumn, NewLiteralValue, NewParameterizedValue calls to include CqlDataType or use the new helper.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/select_translator/utils.go
    • Updated parseSelectClause to accept params and pass it to common.ParseSelectFunction.
    • Modified selectedColumnsToMetadata to use GetType() from SelectedColumn.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/update_translator/translator_update_test.go
    • Updated NewLiteralValue and NewParameterizedValue calls to include CqlDataType.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/translators/update_translator/utils.go
    • Updated common.ParseValueAny calls to pass tableConfig and QueryClauseValues for proper context.
  • cassandra-bigtable-migration-tools/cassandra-bigtable-proxy/utilities/utils.go
    • Removed reflect import.
    • Renamed GoToString to GoToQueryString to better reflect its purpose.
    • Removed GetValueInt32, GetValueInt64, GetValueSlice, GetValueMap as they were moved to global/types/query_parameters.go.
    • Updated isSupportedCollectionElementType to include Uuid and Timeuuid.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a significant and well-structured refactoring for handling CQL functions. The introduction of the CqlFunc interface and the DynamicValue hierarchy greatly improves the modularity and extensibility of function handling. The changes are extensive but follow a clear pattern, moving towards a more object-oriented design.

However, I've found several bugs in the new function implementations, particularly for minTimeuuid, maxTimeuuid, and toTimestamp, as well as a logic error in FlipOperator. I've also noted an incorrect test assertion change. These issues should be addressed to ensure correctness.

Comment on lines +59 to +70
u[8] = 0x7f

// clock_seq_low (byte 9, 8 bits)
u[9] = 0x7f

// Node ID (bytes 10-15, 48 bits) - set to all F's (max)
u[10] = 0x7f
u[11] = 0x7f
u[12] = 0x7f
u[13] = 0x7f
u[14] = 0x7f
u[15] = 0x7f
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The implementation for generating the maximum timeuuid is incorrect. To get the maximum UUIDv1 for a given time, the clock sequence and node ID fields should be set to their maximum values (0xFF), and the variant bits in u[8] must be set correctly.

Specifically:

  • u[8] should be 0xBF to represent the variant 10 and have the clock sequence high bits maxed out.
  • u[9] (clock sequence low) should be 0xFF.
  • u[10] through u[15] (node ID) should also be 0xFF.

The current implementation uses 0x7f, which is incorrect and doesn't set the variant bits properly.

Suggested change
u[8] = 0x7f
// clock_seq_low (byte 9, 8 bits)
u[9] = 0x7f
// Node ID (bytes 10-15, 48 bits) - set to all F's (max)
u[10] = 0x7f
u[11] = 0x7f
u[12] = 0x7f
u[13] = 0x7f
u[14] = 0x7f
u[15] = 0x7f
u[8] = 0xBF
// clock_seq_low (byte 9, 8 bits)
u[9] = 0xFF
// Node ID (bytes 10-15, 48 bits) - set to all F's (max)
u[10] = 0xFF
u[11] = 0xFF
u[12] = 0xFF
u[13] = 0xFF
u[14] = 0xFF
u[15] = 0xFF

Comment on lines +57 to +65
u[9] = 0x80 // Min 8 bits

// Node ID (bytes 10-15, 48 bits) - set to all 0's (min)
u[10] = 0x80
u[11] = 0x80
u[12] = 0x80
u[13] = 0x80
u[14] = 0x80
u[15] = 0x80
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The implementation for generating the minimum timeuuid is incorrect. To get the minimum UUIDv1 for a given time, the clock sequence and node ID fields should be set to their minimum values (0x00).

Specifically:

  • u[9] (clock sequence low) should be 0x00.
  • u[10] through u[15] (node ID) should also be 0x00.

The current implementation uses 0x80 for these bytes, which is not the minimum value.

Suggested change
u[9] = 0x80 // Min 8 bits
// Node ID (bytes 10-15, 48 bits) - set to all 0's (min)
u[10] = 0x80
u[11] = 0x80
u[12] = 0x80
u[13] = 0x80
u[14] = 0x80
u[15] = 0x80
u[9] = 0x00 // Min 8 bits
// Node ID (bytes 10-15, 48 bits) - set to all 0's (min)
u[10] = 0x00
u[11] = 0x00
u[12] = 0x00
u[13] = 0x00
u[14] = 0x00
u[15] = 0x00

Comment on lines +33 to +35
if dt.Code() == INT || dt.Code() == BIGINT {
return TypeFloat
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

According to Cassandra documentation, the avg function on int or bigint columns returns a varint. This implementation returns TypeFloat. While this might be a deliberate simplification, it deviates from the standard Cassandra behavior. Consider returning TypeVarint for better compatibility.

Suggested change
if dt.Code() == INT || dt.Code() == BIGINT {
return TypeFloat
}
if dt.Code() == INT || dt.Code() == BIGINT {
return TypeVarint
}

case GTE:
return LTE
case LTE:
return LTE
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

There's a logic error here. Flipping the <= (LTE) operator should result in >= (GTE), but it's currently returning LTE.

Suggested change
return LTE
return GTE

Comment on lines +289 to +291
assert.Equal(t, &t1, got.id)
assert.Equal(t, "3", got.val)
assert.Equal(t, &t2, got.parentEvent)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

These assertions are comparing pointers instead of the values they point to. This will likely not behave as expected. You should dereference the pointers before comparing with the expected UUID values.

Suggested change
assert.Equal(t, &t1, got.id)
assert.Equal(t, "3", got.val)
assert.Equal(t, &t2, got.parentEvent)
assert.Equal(t, t1, *got.id)
assert.Equal(t, "3", got.val)
assert.Equal(t, t2, *got.parentEvent)

Comment on lines +295 to +297
assert.Equal(t, &t1, got.id)
assert.Equal(t, "3", got.val)
assert.Equal(t, &t2, got.parentEvent)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the above, these assertions are comparing pointers instead of values. Please dereference got.id and got.parentEvent for a correct value comparison.

Suggested change
assert.Equal(t, &t1, got.id)
assert.Equal(t, "3", got.val)
assert.Equal(t, &t2, got.parentEvent)
assert.Equal(t, t1, *got.id)
assert.Equal(t, "3", got.val)
assert.Equal(t, t2, *got.parentEvent)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant