Skip to content

SNOW-3484790: use local list instead of fetch from snowflake#4217

Open
sfc-gh-yuwang wants to merge 1 commit intomainfrom
SNOW-3484790
Open

SNOW-3484790: use local list instead of fetch from snowflake#4217
sfc-gh-yuwang wants to merge 1 commit intomainfrom
SNOW-3484790

Conversation

@sfc-gh-yuwang
Copy link
Copy Markdown
Collaborator

@sfc-gh-yuwang sfc-gh-yuwang commented May 7, 2026

  1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

    Fixes SNOW-NNNNNNN

  2. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
      • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am adding new credentials
    • I am adding a new dependency
    • If this is a new feature/behavior, I'm adding the Local Testing parity changes.
    • I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines
    • If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support. Follow the link for more information: AST Support Guidelines
  3. Please describe how your code solves the related issue.

    Please write a short description of how your code change solves the related issue.

@sfc-gh-yuwang sfc-gh-yuwang requested review from a team as code owners May 7, 2026 18:35
@sfc-gh-yuwang sfc-gh-yuwang added the NO-CHANGELOG-UPDATES This pull request does not need to update CHANGELOG.md label May 7, 2026
@sfc-gh-yuwang
Copy link
Copy Markdown
Collaborator Author

the updated system function list is fetched with query:
SHOW FUNCTIONS ->> SELECT LISTAGG('"' || LOWER("name") || '"', ',\n') WITHIN GROUP (ORDER BY LOWER("name")) AS result FROM $1 WHERE "is_aggregate" = 'Y';

Comment on lines -5080 to -5081
for r in self.sql(
"""show functions ->> select "name" from $1 where "is_aggregate" = 'Y'"""
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that queries like this ensure the server is always the source of truth, and we don't have to make client code changes that are aligned with whatever Snowflake version is being run on the server. Hard-coding it to a local change is something of a philosophical change, and may unexpectedly break workloads in some circumstances.

It looks to only ever get run once per workload anyway, so I think the benefits of this would be pretty minimal.

Copy link
Copy Markdown
Collaborator Author

@sfc-gh-yuwang sfc-gh-yuwang May 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a discussion with Adam and Yijun this morning. One of the issue is that this is always run when there are potentially agg function in the query, which means user would still see this even if they just issued 1 query in SCOS.
This is also a pretty expensive operation that took 3~4 seconds, which is a big portion when user workflow is small.

To solve the problem that this list may diverge from server, I created a followup ticket: https://snowflakecomputing.atlassian.net/browse/SNOW-3489271

Currently my thoughts is adding a step to release pipeline to ensure that each snowpark release's system function list is up-to-date at the point it released. Thus user would only need to use latest snowpark to avoid the mis align issue.

But I think it also make sense to add parameter protection for this. Will add one

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parameter-protecting makes sense. I'm slightly worried about server divergence because the same client release might get run against different version backends (I'm not sure if we the N-1 version policy still exists, but at the very least testing vs. prod environments, and customers on different release cadences might get different versions)

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.04%. Comparing base (2051e02) to head (92b4d70).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4217      +/-   ##
==========================================
- Coverage   95.42%   95.04%   -0.39%     
==========================================
  Files         171      171              
  Lines       43837    43872      +35     
  Branches     7513     7525      +12     
==========================================
- Hits        41833    41698     -135     
- Misses       1225     1368     +143     
- Partials      779      806      +27     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

NO-CHANGELOG-UPDATES This pull request does not need to update CHANGELOG.md

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants