SNOW-2267482: Fix flakiness due to SnowAPI dependency by sfc-gh-yuwang · Pull Request #4190 · snowflakedb/snowpark-python

sfc-gh-yuwang · 2026-04-21T17:58:45Z

Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

Fixes SNOW-2267482
Fill out the following pre-review checklist:
- I am adding a new automated test(s) to verify correctness of my new code
  - If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
- I am adding new logging messages
- I am adding a new telemetry message
- I am adding new credentials
- I am adding a new dependency
- If this is a new feature/behavior, I'm adding the Local Testing parity changes.
- I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines
- If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support. Follow the link for more information: AST Support Guidelines
Please describe how your code solves the related issue.

Please write a short description of how your code change solves the related issue.

github-actions · 2026-04-21T17:58:59Z

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

sfc-gh-yuwang · 2026-04-21T18:03:40Z

https://snowpark-python-001.jenkinsdev1.us-west-2.aws-dev.app.snowflake.com/job/SnowparkConnectRegressionRunner/500/
first run to identify regression

sfc-gh-yuwang · 2026-04-21T18:04:42Z

I have read the CLA Document and I hereby sign the CLA

codecov-commenter · 2026-04-21T18:58:56Z

Codecov Report

❌ Patch coverage is 93.83378% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.02%. Comparing base (91f7657) to head (1823470).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/snowflake/snowpark/catalog.py	93.78%	20 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4190      +/-   ##
==========================================
- Coverage   95.04%   95.02%   -0.03%     
==========================================
  Files         171      171              
  Lines       43876    44157     +281     
  Branches     7525     7535      +10     
==========================================
+ Hits        41703    41961     +258     
- Misses       1368     1388      +20     
- Partials      805      808       +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

sfc-gh-saroskar · 2026-05-05T01:23:23Z

+        c = self._catalog
+        like_str = f"LIKE '{like}'" if like else ""
+        df = c._session.sql(
+            f"SHOW AS RESOURCE DATABASES {like_str} LIMIT {_SHOW_AS_RESOURCE_LIMIT}"


Why SHOW AS RESOURCE DATABASES instead of just SHOW DATABASES? For some resources SHOW AS RESOURCES can be much more expensive because they return a lot more data since SHOW AS RESOURCE was built for REST API output. For example, SHOW AS RESOURCE TABLE recurses into columns of the table and return all columns of the table with constraints, their data types etc.

I was following the pattern in the PR that was merged before: https://github.com/snowflakedb/snowpark-python/pull/3942/changes

I think the goal that this was originally used was to maintain the behavior as much as possible.

sfc-gh-saroskar · 2026-05-05T01:25:38Z

+            c._initialize_regex_udf()
+            assert c._python_regex_udf is not None  # pyright
+            df = df.filter(
+                c._python_regex_udf(lit(pattern), parse_json('"As Resource"')["name"])


Do we need regex here? Can we use simple col.like()?

This is also a existing pattern in current main, I am trying to follow it as much as possible

sfc-gh-saroskar · 2026-05-05T01:31:44Z

+        db_name = c._parse_database(database)
+        try:
+            return self.list_schemas(database=db_name, like=unquote_if_quoted(schema))[
+                0


Shouldn't we use DESCRIBE or DESCRIBE AS RESOURCE here for efficiency? We need to fetch only one record.

the reason is that Describe or Describe as resource is returning different results then Show. We use this to not introduce BCR.

sfc-gh-saroskar · 2026-05-05T01:33:55Z

+        schema_name = c._parse_schema(schema)
+        try:
+            return c.list_views(
+                database=db_name,


Use DESCRIBE or DESCRIBE AS RESOURCE instead?

same reason as above, describe is returning different result than show and is missing information

sfc-gh-saroskar · 2026-05-05T01:40:04Z

 """
        self.version = get_version()
        self._session_stage = None
+        self._use_sql_base_catalog = True


Who / where / how is this flag turned off?

this is controlled in SCOS at here: https://github.com/snowflake-eng/sas/pull/3822/changes#diff-e7ffc1935ec104660e3a0eccbd0693faa973359ed84d9429d28a28684fb55a38R821
basically user can controll it like:
spark.conf.set("snowpark.connect.catalog.use_sql_base_catalog", "false")

graphite-app · 2026-05-06T21:49:35Z

 #

+from abc import ABC, abstractmethod
+from ctypes import ArgumentError


Incorrect import: ArgumentError from ctypes is intended for ctypes-specific errors, not general argument validation. This is used on lines 494 and 524 to validate user input.

Impact: Raises semantically incorrect exception type that could confuse callers expecting standard Python exceptions.

Fix:

# Remove this import # from ctypes import ArgumentError # Then on lines 494 and 524, use: raise ValueError( "When provided procedure is a Procedure class no other arguments can be provided" )

Spotted by Graphite

Is this helpful? React 👍 or 👎 to let us know.

sfc-gh-mayliu

Snowpark change LGTM, can you paste a good SnowparkConnectRegressionRunnerV2 result before Snowpark PR merge to verify no regression to SCOS? https://snowpark-python-001.jenkinsdev1.us-west-2.aws-dev.app.snowflake.com/job/SnowparkConnectRegressionRunnerV2/

sfc-gh-yuwang · 2026-05-07T22:53:00Z

Snowpark change LGTM, can you paste a good SnowparkConnectRegressionRunnerV2 result before Snowpark PR merge to verify no regression to SCOS? https://snowpark-python-001.jenkinsdev1.us-west-2.aws-dev.app.snowflake.com/job/SnowparkConnectRegressionRunnerV2/

Will do! Will fix bugs if there is any

reapply change

3467b7b

sfc-gh-yuwang requested review from a team as code owners April 21, 2026 17:58

sfc-gh-yuwang requested review from sfc-gh-jdu, sfc-gh-jrose and sfc-gh-mayliu April 21, 2026 17:58

fix test

143f30a

sfc-gh-yuwang added 7 commits April 22, 2026 15:39

avoid regress

e45239a

dual mode of catalog

0692a09

init back end at catalog init

f610835

Merge branch 'main' into SNOW-2267482

bfcf249

push missed test fixture

674d831

fix lint

5933508

remove changelog and use notimplementederror

02ec3b8

sfc-gh-yuwang added the NO-CHANGELOG-UPDATES This pull request does not need to update CHANGELOG.md label Apr 28, 2026

move fixture back to test_catalog

c78333c

sfc-gh-yuwang requested a review from sfc-gh-aling April 28, 2026 22:00

sfc-gh-yuwang added 6 commits April 28, 2026 15:39

Merge branch 'main' into SNOW-2267482

9e49c85

Merge branch 'main' into SNOW-2267482

bc65acf

fix test

2afd738

add limit 10000 in sql base(scos only)

50d0c32

remove wrong test

9e89ca4

Merge branch 'main' into SNOW-2267482

4334c32

sfc-gh-yuwang requested a review from sfc-gh-saroskar April 30, 2026 21:39

sfc-gh-yuwang added 2 commits April 30, 2026 14:45

Merge branch 'main' into SNOW-2267482

701c08b

address comments

ef16ee8

sfc-gh-yuwang added 9 commits April 30, 2026 16:47

restore comment

d9dad80

parameter protection

68650b1

Merge branch 'main' into SNOW-2267482

1906d61

parameter protection

bb2ef8e

add test

e0097c9

fix test

1472367

rewrite parameter protection

8a9bc66

revert session change

40ace28

rename to _use_sql_base_catalog

e8cbe79

sfc-gh-saroskar reviewed May 5, 2026

View reviewed changes

add test

ff8a005

sfc-gh-aling reviewed May 6, 2026

View reviewed changes

sfc-gh-yuwang added 2 commits May 6, 2026 14:34

address comments

743620b

address comments

1e3af12

graphite-app Bot reviewed May 6, 2026

View reviewed changes

address comments

2ecc805

sfc-gh-yuwang requested review from sfc-gh-aling and sfc-gh-saroskar May 6, 2026 22:04

sfc-gh-aling approved these changes May 6, 2026

View reviewed changes

Merge branch 'main' into SNOW-2267482

0608e79

sfc-gh-mayliu approved these changes May 7, 2026

View reviewed changes

Merge branch 'main' into SNOW-2267482

1823470

Conversation

sfc-gh-yuwang commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sfc-gh-yuwang commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sfc-gh-yuwang commented Apr 21, 2026

Uh oh!

codecov-commenter commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

graphite-app Bot May 6, 2026

Choose a reason for hiding this comment

Uh oh!

sfc-gh-mayliu left a comment

Choose a reason for hiding this comment

Uh oh!

sfc-gh-yuwang commented May 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

sfc-gh-yuwang commented Apr 21, 2026 •

edited

Loading

github-actions Bot commented Apr 21, 2026 •

edited

Loading

sfc-gh-yuwang commented Apr 21, 2026 •

edited

Loading

codecov-commenter commented Apr 21, 2026 •

edited

Loading