Akshay/lakebox staging gateway ue1

.agent/rules/auto-generated-files.md

@@ -50,49 +50,54 @@ paths:
 
 ## Identification
 
-The files matching this rule glob pattern are most likely generated artifacts. Auto-generated files generally have a comment (if the file type allows for comments) at or near the top of the file indicating that they are generated, or their file name/path may indicate they are generated. You may also consult Makefile as starting point to determine if a file is auto-generated.
+Files matching this rule's glob pattern are most likely generated artifacts. Auto-generated files generally have a comment (when the file type allows it) at or near the top indicating they are generated, or their name or path signals it. You may also consult the Makefile as a starting point to determine if a file is auto-generated.
 
 ## Rules
 
-DO NOT "MANUALLY" EDIT THESE FILES!
+**RULE: Do not manually edit auto-generated files.**
 
-If a change is needed in any matched file:
-1. Find the source logic/template/annotation that drives the file.
-2. Run the appropriate generator/update command.
-3. Commit both the source change (if any) and regenerated outputs.
+**RULE: To change a generated file, edit the source and regenerate.**
+
+1. Find the source logic, template, or annotation that drives the file.
+2. Run the appropriate generator or update command.
+3. Commit both the source change (if any) and the regenerated outputs.
 
 ### Core generation commands
 
+- Everything, in one shot:
+  - `./task generate` — aggregator that runs all generators below
 - OpenAPI SDK/CLI command stubs and related generated artifacts:
-  - `make generate`
-  - Includes generated `cmd/account/**`, `cmd/workspace/**`, `.gitattributes`, `internal/genkit/tagging.py`, and direct engine refresh.
+  - `./task generate-genkit`
+  - Includes generated `cmd/account/**`, `cmd/workspace/**`, `.gitattributes`, `internal/genkit/tagging.py`.
 - Direct engine generated YAML:
-  - `make generate-direct` (or `make generate-direct-apitypes`, `make generate-direct-resources`)
+  - `./task generate-direct` (or `./task generate-direct-apitypes`, `./task generate-direct-resources`)
 - Bundle schemas:
-  - `make schema`
-  - `make schema-for-docs`
+  - `./task generate-schema`
+  - `./task generate-schema-docs`
   - This can also refresh `bundle/internal/schema/annotations_openapi.yml` when OpenAPI annotation extraction is enabled.
 - Bundle docs:
-  - `make docs`
+  - `./task generate-docs`
 - Validation generated code:
-  - `make generate-validation`
+  - `./task generate-validation`
 - Mock files:
   - `go run github.com/vektra/mockery/v2@b9df18e0f7b94f0bc11af3f379c8a9aea1e1e8da`
 - Python bundle codegen:
-  - `make -C python codegen`
+  - `./task pydabs-codegen`
 
 ### Acceptance and test generated outputs
 
-Acceptance outputs are generated and should not be hand-edited (except rare, intentional mass replacement when explicitly justified by repo guidance).
+**RULE: Do not hand-edit acceptance outputs.** Exception: rare, intentional mass replacement when explicitly justified by repo guidance.
+
+Regeneration commands:
+
+- `./task test-update`
+- `./task test-update-templates` (templates only)
+
+Typical generated files:
 
-- Preferred regeneration:
-  - `make test-update`
-  - `make test-update-templates` (templates only)
-  - `make generate-out-test-toml` (only `out.test.toml`)
-- Typical generated files include:
-  - `acceptance/**/out*`
-  - `acceptance/**/output.txt`
-  - `acceptance/**/output.*.txt`
-  - `acceptance/**/output/**` (materialized template output trees)
+- `acceptance/**/out*`
+- `acceptance/**/output.txt`
+- `acceptance/**/output.*.txt`
+- `acceptance/**/output/**` (materialized template output trees)
 
-When touching acceptance sources (`databricks.yml`, scripts, templates, or test config), regenerate outputs instead of editing generated files directly.
+**RULE: When touching acceptance sources, regenerate outputs instead of editing generated files.** Sources include `databricks.yml`, scripts, templates, and test config.

.agent/rules/dresources.md

@@ -0,0 +1,8 @@
+---
+description: Rules for implementing resources in bundle/direct/dresources
+globs: bundle/direct/dresources/**/*.go
+paths:
+  - "bundle/direct/dresources/**/*.go"
+---
+
+**RULE: Before implementing or modifying a resource, read `bundle/direct/dresources/README.md`.** It covers constraints, field classification, update mask rules, async patterns, testing requirements, and state compatibility.

.agent/rules/style-guide-go.md

@@ -9,11 +9,9 @@ paths:
 
 ## General guidance
 
-Please make sure code that you author is consistent with the codebase and concise.
+Code you author should be consistent with the codebase and concise. The code should be self-documenting based on its function and variable names.
 
-The code should be self-documenting based on the code and function names.
-
-Functions should be documented with a doc comment as follows:
+**RULE: Document functions with a doc comment that starts with the function name and ends with a period.**
 
 ```go
 // SomeFunc does something.
@@ -22,33 +20,197 @@ func SomeFunc() {
 }
 ```
 
-Note how the comment starts with the name of the function and is followed by a period.
+**RULE: Avoid redundant and verbose comments.** Only add a comment if it complements the code rather than repeating it.
+
+**RULE: Focus on making implementations as small and elegant as possible.** Avoid unnecessary loops and allocations. If dropping or relaxing a requirement would simplify things, ask the user about the trade-off.
+
+### Modern Go (1.24+) idioms
+
+**RULE: Use `for i := range X` for integer iteration, not `for i := 0; i < X; i++`.**
+
+**RULE: Use builtin `min()` and `max()` where possible.** They work on any type and any number of values.
+
+**RULE: Do not capture the for-range variable.** Since Go 1.22 a new copy is created each iteration, so the capture workaround is no longer needed.
+
+**RULE: Use empty struct types for context keys.**
 
-Avoid redundant and verbose comments. Use terse comments and only add comments if it complements, not repeats the code.
+GOOD:
 
-Focus on making implementation as small and elegant as possible. Avoid unnecessary loops and allocations. If you see an opportunity of making things simpler by dropping or relaxing some requirements, ask user about the trade-off.
+```go
+type myKeyType struct{}
+```
+
+BAD:
+
+```go
+type myKeyType int
+```
 
-Use modern idiomatic Golang features (version 1.24+). Specifically:
- - Use for-range for integer iteration where possible. Instead of for i:=0; i < X; i++ {} you must write for i := range X{}.
- - Use builtin min() and max() where possible (works on any type and any number of values).
- - Do not capture the for-range variable, since go 1.22 a new copy of the variable is created for each loop iteration.
- - Use empty struct types for context keys: `type myKeyType struct{}` (not `int`).
- - Define magic strings as named constants at the top of the file.
- - When integrating external tools or detecting environment variables, include source reference URLs as comments so they can be traced later.
+**RULE: Define magic strings as named constants at the top of the file.** When a value is used in more than one place, define a package-level constant even if the literal is short. If a related value already has a constant in the same package, follow the existing pattern instead of introducing a parallel literal.
+
+**RULE: When integrating external tools or detecting environment variables, include source reference URLs as comments.** This lets future readers trace where the behavior came from.
+
+### Control flow
+
+**RULE: When several branches are alternatives for the same decision, prefer `switch` over `if / else if`.** Same-decision means: you're dispatching on one value or one boolean question and each branch is a different answer to it. Ordered precedence chains (try this, else this, else this) are a different shape; leave those as early-return `if`s.
+
+GOOD (alternatives for one decision):
+
+```go
+switch mode {
+case modeText:
+	renderText(out, result)
+case modeJSON:
+	renderJSON(out, result)
+default:
+	return fmt.Errorf("unknown mode %q", mode)
+}
+```
+
+Also fine (ordered precedence, early-return style):
+
+```go
+// see libs/auth/storage/mode.go#ResolveStorageMode
+if override != "" {
+	return override, nil
+}
+if envValue != "" {
+	return parseMode(envValue)
+}
+return loadFromFile()
+```
+
+**RULE: Collapse `if err != nil { return err }; return nil` to just `return err`.** The pattern is never doing anything useful in the intermediate step.
+
+GOOD:
+
+```go
+return someCall()
+```
+
+BAD:
+
+```go
+if err := someCall(); err != nil {
+	return err
+}
+return nil
+```
+
+### Determinism
+
+**RULE: When you build a slice by iterating over a map and its order affects tests, logs, update masks, or the wire format, sort it before returning.** Go maps have randomized iteration order, so the same input produces different outputs across runs. Reviewers catch these when they produce flaky test output or noisy diffs in update masks. If the slice is purely internal and nothing downstream observes its order, sorting is unnecessary — some accepted code in `bundle/direct/dresources/` and `bundle/config/mutator/resourcemutator/` returns unsorted slices precisely because order doesn't matter there.
+
+GOOD:
+
+```go
+fieldPaths := make([]string, 0, len(changes))
+for p := range changes {
+	fieldPaths = append(fieldPaths, p)
+}
+slices.Sort(fieldPaths)
+return fieldPaths
+```
+
+BAD:
+
+```go
+fieldPaths := make([]string, 0, len(changes))
+for p := range changes {
+	fieldPaths = append(fieldPaths, p)
+}
+return fieldPaths
+```
 
-### Configuration Patterns
+### Environment variables
+
+**RULE: In library and product code, use `github.com/databricks/cli/libs/env` for reading environment variables, not `os.Getenv`.** `env.Get(ctx, name)` and `env.Lookup(ctx, name)` can be overridden per-context in tests, so you don't have to mutate process-wide state to exercise a code path. `os.Getenv` is still fine in `main`, tests, and acceptance/integration harnesses where no `ctx` is available and overrides aren't needed.
+
+GOOD:
+
+```go
+import "github.com/databricks/cli/libs/env"
+
+token := env.Get(ctx, "DATABRICKS_TOKEN")
+if path, ok := env.Lookup(ctx, "DATABRICKS_CLI_PATH"); ok {
+	// use path
+}
+```
+
+BAD:
+
+```go
+token := os.Getenv("DATABRICKS_TOKEN")
+```
+
+### Lazy initialization
+
+**RULE: Use `sync.OnceValue`, `sync.OnceValues`, or `sync.OnceFunc` for one-time initialization, not `sync.Once` with package variables.** The `sync.OnceValue[T]` family (Go 1.21+) removes the boilerplate and makes the cached result a first-class return value. Use `sync.OnceFunc` when the initialization has side effects and no return value (the CLI already uses it in places like `libs/cmdio/spinner.go`).
+
+GOOD:
+
+```go
+var loadConfig = sync.OnceValues(func() (*Config, error) {
+	return parseConfigFile("config.yml")
+})
+
+func GetConfig() (*Config, error) {
+	return loadConfig()
+}
+```
+
+BAD:
+
+```go
+var (
+	config     *Config
+	configErr  error
+	configOnce sync.Once
+)
+
+func GetConfig() (*Config, error) {
+	configOnce.Do(func() {
+		config, configErr = parseConfigFile("config.yml")
+	})
+	return config, configErr
+}
+```
+
+Caveats that apply to both: results are cached forever (including errors), and a panic is rethrown on every subsequent call. Create a new instance if you need retry semantics.
+
+### Constructors
+
+When a constructor's parameter list is long enough that callers forget the order or misread positional arguments, consider grouping dependencies into a struct. This is a judgment call. The CLI has many ordinary constructors with 4+ parameters, and that's fine when the arguments are obvious at the call site. The signal for switching is readability, not parameter count.
+
+```go
+type ServiceDeps struct {
+	Client    HTTPClient
+	Logger    Logger
+	Config    Config
+	Validator Validator
+	Metrics   MetricsCollector
+}
+
+func NewService(deps ServiceDeps) *Service { ... }
+```
+
+### Configuration patterns
 
 - Bundle config uses `dyn.Value` for dynamic typing
 - Config loading supports includes, variable interpolation, and target overrides
 - Schema generation is automated from Go struct tags
 
 ## Context
 
-Always pass `context.Context` as a function argument; never store it in a struct. Storing context in a struct obscures the lifecycle and prevents callers from setting per-call deadlines, cancellation, and metadata (see https://go.dev/blog/context-and-structs). Do not use `context.Background()` outside of `main.go` files. In tests, use `t.Context()` (or `b.Context()` for benchmarks).
+**RULE: Always pass `context.Context` as a function argument; never store it in a struct.** Storing context in a struct obscures the lifecycle and prevents callers from setting per-call deadlines, cancellation, and metadata. See https://go.dev/blog/context-and-structs.
+
+**RULE: Do not use `context.Background()` outside of `main.go` files.**
+
+**RULE: In tests, use `t.Context()` (or `b.Context()` for benchmarks).**
 
 ## Logging
 
-Use the following for logging:
+**RULE: Use `github.com/databricks/cli/libs/log` for debug/info/warn/error logging.** The `ctx` variable must be passed in by the caller.
 
 ```go
 import "github.com/databricks/cli/libs/log"
@@ -59,15 +221,34 @@ log.Warnf(ctx, "...")
 log.Errorf(ctx, "...")
 ```
 
-Note that the 'ctx' variable here is something that should be passed in as
-an argument by the caller.
-
-Use cmdio.LogString to print to stdout:
+**RULE: Use `cmdio.LogString` to print to stdout.**
 
 ```go
 import "github.com/databricks/cli/libs/cmdio"
 
 cmdio.LogString(ctx, "...")
 ```
 
-Always output file path with forward slashes, even on Windows, so that acceptance test output is stable between OSes. Use filepath.ToSlash for this.
+**RULE: Always output file paths with forward slashes, even on Windows.** Use `filepath.ToSlash` so acceptance test output is stable between OSes.
+
+**RULE: Pick log levels deliberately.** Warn for things the user should know about and might act on. Debug for diagnostic signal a developer wants but a user shouldn't see by default. Error for actual failures that also surface as a returned error. A message that's warn-by-default but isn't user-actionable belongs at debug.
+
+GOOD:
+
+```go
+if err := w.Config.Authenticate(); err != nil {
+	// user can check their profile; worth warning
+	log.Warnf(ctx, "could not authenticate: %v", err)
+}
+
+if err := cleanupExpiredCacheEntries(ctx); err != nil {
+	// internal-only, user can't act on this
+	log.Debugf(ctx, "cache cleanup failed: %v", err)
+}
+```
+
+BAD:
+
+```go
+log.Warnf(ctx, "cache cleanup failed: %v", err) // noisy, not actionable
+```

.agent/rules/style-guide-py.md

@@ -9,11 +9,18 @@ paths:
 
 ## General guidance
 
-When writing Python scripts, we bias for conciseness. We think of Python in this code base as scripts.
-- use Python 3.11
-- Do not catch exceptions to make nicer messages, only catch if you can add critical information
-- use pathlib.Path in almost all cases over os.path unless it makes code longer
-- Do not add redundant comments.
-- Try to keep your code small and the number of abstractions low.
-- After done, format your code with `ruff format -n <path>`
-- Use `#!/usr/bin/env python3` shebang.
+Python in this codebase is written as scripts. Bias for conciseness.
+
+**RULE: Use Python 3.11.**
+
+**RULE: Use `#!/usr/bin/env python3` as the shebang.**
+
+**RULE: Prefer `pathlib.Path` over `os.path`.** Exception: when it would make the code longer.
+
+**RULE: Do not catch exceptions just to add a nicer message.** Only catch if you can add critical information the caller can't produce.
+
+**RULE: Avoid redundant comments.**
+
+**RULE: Keep code small and minimize abstractions.**
+
+**RULE: Format with `ruff format -n <path>` before committing.**