From 67e796b81d0e1db44ca74b5e2c02c20ed173c053 Mon Sep 17 00:00:00 2001 From: Pawel Kosiec Date: Wed, 20 May 2026 17:48:02 +0200 Subject: [PATCH 1/4] Fix eval regression from PR#76: soften State Storage Rule PR#76 introduced an aggressive State Storage Rule that auto-detected Lakebase need for any app mentioning state-like terms (preferences, bookmarks, etc.), causing 16 app regressions in the May 19 nightly eval. Analytics apps like property_search_app and host_onboarding_checklist were incorrectly pushed toward Lakebase, dropping to 0.00. Changes: - Replace aggressive auto-detect with softer guidance that asks the user - Remove "preferences, bookmarks" from trigger list (too broad) - Restore user agency ("Ask the user" vs "Do not wait for the user") - Explicitly exclude analytics/dashboard apps from Lakebase push - Revert description metadata and Decision Gate skip clause - Still recommends Lakebase for genuine CRUD/state storage needs Fixes: LKB-12991 Co-authored-by: Isaac --- skills/databricks-apps/SKILL.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/skills/databricks-apps/SKILL.md b/skills/databricks-apps/SKILL.md index 27b15b4..12aaac3 100644 --- a/skills/databricks-apps/SKILL.md +++ b/skills/databricks-apps/SKILL.md @@ -1,6 +1,6 @@ --- name: databricks-apps -description: "Build apps on Databricks Apps platform. Use when asked to create dashboards, data apps, analytics tools, or visualizations. Auto-detects need for Lakebase when app stores state; evaluates data access patterns (analytics vs Lakebase synced tables) before scaffolding. Invoke BEFORE starting implementation." +description: "Build apps on Databricks Apps platform. Use when asked to create dashboards, data apps, analytics tools, or visualizations. Evaluates data access patterns (analytics vs Lakebase synced tables) before scaffolding. Invoke BEFORE starting implementation." compatibility: Requires databricks CLI (>= v0.294.0) metadata: version: "0.1.1" @@ -17,7 +17,7 @@ Build apps that deploy to Databricks Apps platform. | Phase | READ BEFORE proceeding | |-------|------------------------| -| Scaffolding | **⚠️ STOP — evaluate the State Storage Rule and Data Access Decision Gate below before scaffolding.** Parent `databricks-core` skill (auth, warehouse discovery); then run `databricks apps manifest` + `databricks apps init` with `--features` and `--set` (see AppKit section below) | +| Scaffolding | **⚠️ STOP — complete the Data Access Decision Gate below before scaffolding.** Parent `databricks-core` skill (auth, warehouse discovery); then run `databricks apps manifest` + `databricks apps init` with `--features` and `--set` (see AppKit section below) | | Writing SQL queries | [SQL Queries Guide](references/appkit/sql-queries.md) | | Writing UI components | [Frontend Guide](references/appkit/frontend.md) | | Using `useAnalyticsQuery` | [AppKit SDK](references/appkit/appkit-sdk.md) | @@ -61,15 +61,16 @@ Build apps that deploy to Databricks Apps platform. Before writing any SQL, use the parent `databricks-core` skill for data exploration — search `information_schema` by keyword, then batch `discover-schema` for the tables you need. Do NOT skip this step. -**State Storage Rule (evaluate BEFORE the Decision Gate):** +**State Storage Guidance (evaluate BEFORE the Decision Gate):** -If the user's app description implies storing or persisting data — forms, CRUD operations, user input, preferences, bookmarks, orders, todos, comments, votes, or any user-generated content — the app needs a Lakebase database. Do not wait for the user to ask for one. +If the user's app description involves storing or persisting data — forms, CRUD operations, user submissions, orders, todos, or other user-generated content — the app likely needs a Lakebase database. -1. Use the **`databricks-lakebase`** skill to create a Lakebase project (if one doesn't already exist) and obtain the branch and database resource names. -2. Scaffold with `--features lakebase` and pass `--set lakebase.postgres.branch= --set lakebase.postgres.database=`. -3. If the app **also** reads from Unity Catalog tables, proceed to the Data Access Decision Gate below to determine whether to add `--features analytics` or use Lakebase synced tables. +1. **Ask the user** whether the app needs persistent storage (Lakebase) before scaffolding. Do not silently add Lakebase. +2. If confirmed, use the **`databricks-lakebase`** skill to create a Lakebase project and obtain the branch and database resource names. +3. Scaffold with `--features lakebase` and pass `--set lakebase.postgres.branch= --set lakebase.postgres.database=`. +4. If the app **also** reads from Unity Catalog tables, proceed to the Data Access Decision Gate below to determine whether to add `--features analytics` or use Lakebase synced tables. -This rule governs **state storage** only. For how the app reads existing lakehouse data, proceed to the Decision Gate below. This is not optional — any app that writes user-generated data needs Lakebase. +Do NOT add Lakebase to analytics, dashboard, or visualization apps unless the user explicitly requests persistent write-back storage. Read-only data display, filters, and preferences do not require a database. ## Development Workflow (FOLLOW THIS ORDER) @@ -89,7 +90,7 @@ After the user chooses: - (A) Lakebase synced tables → scaffold with `--features lakebase`. See [Lakebase Guide](references/appkit/lakebase.md) for full workflow. - (B) Analytics → scaffold with `--features analytics`. - Both → scaffold with `--features analytics,lakebase` if the app needs both patterns. -- If the app does NOT read UC data (pure CRUD, Genie, Model Serving), skip this gate. For pure CRUD/state apps, the State Storage Rule above already applies — scaffold with `--features lakebase`. For Genie or Model Serving, scaffold with the corresponding `--features` flag. +- If the app does NOT read UC data (pure CRUD, Genie, Model Serving), skip this gate and scaffold with the appropriate `--features` flag. **Analytics apps** (`--features analytics`): From 415448a82d00bf8cfb5ccb5af3ef05be37200316 Mon Sep 17 00:00:00 2001 From: Pawel Kosiec Date: Wed, 20 May 2026 18:04:37 +0200 Subject: [PATCH 2/4] Restore soft State Storage Guidance mention in scaffolding table The previous commit removed the State Storage reference from the Required Reading table entirely. This creates a discoverability gap: agents jumping from the table to the Decision Gate skip the State Storage Guidance section. Restore it with softer "review" language (vs the old aggressive "evaluate the State Storage Rule") to preserve the flow for CRUD apps that need Lakebase without re-triggering the regression on analytics apps. Co-authored-by: Isaac --- skills/databricks-apps/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/skills/databricks-apps/SKILL.md b/skills/databricks-apps/SKILL.md index 12aaac3..0c28014 100644 --- a/skills/databricks-apps/SKILL.md +++ b/skills/databricks-apps/SKILL.md @@ -17,7 +17,7 @@ Build apps that deploy to Databricks Apps platform. | Phase | READ BEFORE proceeding | |-------|------------------------| -| Scaffolding | **⚠️ STOP — complete the Data Access Decision Gate below before scaffolding.** Parent `databricks-core` skill (auth, warehouse discovery); then run `databricks apps manifest` + `databricks apps init` with `--features` and `--set` (see AppKit section below) | +| Scaffolding | **⚠️ STOP — review the State Storage Guidance and complete the Data Access Decision Gate below before scaffolding.** Parent `databricks-core` skill (auth, warehouse discovery); then run `databricks apps manifest` + `databricks apps init` with `--features` and `--set` (see AppKit section below) | | Writing SQL queries | [SQL Queries Guide](references/appkit/sql-queries.md) | | Writing UI components | [Frontend Guide](references/appkit/frontend.md) | | Using `useAnalyticsQuery` | [AppKit SDK](references/appkit/appkit-sdk.md) | From 25303834b16b8eee6fd711d0ee6c3c882d68e6de Mon Sep 17 00:00:00 2001 From: Pawel Kosiec Date: Fri, 22 May 2026 11:29:31 +0200 Subject: [PATCH 3/4] Fix Decision Gate recommendation: default to Analytics, not Lakebase The comparison table and recommendation text mapped "search" too broadly to Lakebase synced tables. A "property search" app doing SQL WHERE filtering is Analytics, not full-text search. This caused the agent to recommend Lakebase synced tables for apps like property_search_app that just need filtered SQL queries. Changes: - Default recommendation is now Analytics (B) for read-only apps - Lakebase synced tables (A) reserved for explicit sub-second needs: full-text search, typeahead, autocomplete, real-time lookups - Clarify that "search" or "filter" usually means SQL WHERE (Analytics) - Update comparison table: replace broad "Search, lookups, catalogs" with specific "Full-text search, typeahead, autocomplete, real-time lookups"; add "filtered queries, browsing" to Analytics column Co-authored-by: Isaac --- skills/databricks-apps/SKILL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/skills/databricks-apps/SKILL.md b/skills/databricks-apps/SKILL.md index 0c28014..a43854b 100644 --- a/skills/databricks-apps/SKILL.md +++ b/skills/databricks-apps/SKILL.md @@ -81,10 +81,10 @@ If the app reads from Unity Catalog / lakehouse tables, you MUST show the compar | | **(A) Lakebase synced tables** | **(B) Analytics** | |--|---|---| | Speed | Sub-second responses | Takes a few seconds | -| Best for | Search, lookups, catalogs, real-time data, operational apps | Dashboards, charts, aggregations, KPIs | +| Best for | Full-text search, typeahead, autocomplete, real-time lookups, operational apps | Dashboards, charts, aggregations, KPIs, filtered queries, browsing | | How it works | Data synced from Delta into Lakebase Postgres | Queries run on SQL warehouse at read time | -After showing the table, add a brief recommendation. Default to recommending Lakebase synced tables (A) unless the use case is clearly about aggregations, charts, or dashboards where seconds of latency is acceptable. For lookups, searches, serving data to users, or any interactive use case, recommend Lakebase synced tables. Always let the user make the final call. +After showing the table, add a brief recommendation. Default to recommending Analytics (B) for most read-only apps — dashboards, charts, filtered queries, browsing, and aggregations. Recommend Lakebase synced tables (A) only when the app needs sub-second latency for full-text search, typeahead/autocomplete, real-time lookups by ID, or operational data serving. Note: "search" or "filter" in a prompt usually means SQL WHERE clauses (Analytics), not full-text search (Lakebase). Always let the user make the final call. After the user chooses: - (A) Lakebase synced tables → scaffold with `--features lakebase`. See [Lakebase Guide](references/appkit/lakebase.md) for full workflow. From fcd3091a59a8a31f23aff8644277f4edd9457a72 Mon Sep 17 00:00:00 2001 From: Pawel Kosiec Date: Tue, 26 May 2026 12:42:26 +0200 Subject: [PATCH 4/4] Use full name "Unity Catalog" instead of abbreviation "UC" Addresses keugenek's review comment on PR#83. Co-authored-by: Isaac --- skills/databricks-apps/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/skills/databricks-apps/SKILL.md b/skills/databricks-apps/SKILL.md index a43854b..f5609cf 100644 --- a/skills/databricks-apps/SKILL.md +++ b/skills/databricks-apps/SKILL.md @@ -90,7 +90,7 @@ After the user chooses: - (A) Lakebase synced tables → scaffold with `--features lakebase`. See [Lakebase Guide](references/appkit/lakebase.md) for full workflow. - (B) Analytics → scaffold with `--features analytics`. - Both → scaffold with `--features analytics,lakebase` if the app needs both patterns. -- If the app does NOT read UC data (pure CRUD, Genie, Model Serving), skip this gate and scaffold with the appropriate `--features` flag. +- If the app does NOT read Unity Catalog data (pure CRUD, Genie, Model Serving), skip this gate and scaffold with the appropriate `--features` flag. **Analytics apps** (`--features analytics`):