Cedric/better dx

[ViaDézo1er / cedric (viadezo1er)]

bt setup asks 0 and 2 questions: project and coding agent (and needs authentication)

org choice:

if --api-key is passed, use the provided api key
else if --prefer-profile, use the already registered profile (probably registered with OAuth previously)
else if the user has an api key in the env (variable BRAINTRUST_API_KEY), use it
else if there is already a registered profile, use it. If a profile is registered but the access token isn't accessible (for example storing the access token in plain text in ~/.config/bt/secrets.json but the file was deleted), just force re authentication
else same behavior as a user picking OAuth: give a url to paste in the browser

project choice:

if -p/--project is passed, use the specified project. If the project does not exist, stop the setup. (maybe we should create the project instead?)
else if BRAINTRUST_DEFAULT_PROJECT exist in the env, same as for -p/--project
else if the user has two or more projects, prompt the user to choose the project to use
else the user has exactly one project, the default 'My Project' projects. Create a project in the user's organization named '{whoami}-test' where {whoami} is the result of the whoami command and use this project

skills, mcp, local, global:

by default, choose skills and mcp and global scope
--no-skill, --no-mcp unselect skill/mcp
--local selects local instead of global
--skill --mcp --global are only here to conflict with their --no-... counterpart, they don't do anything else
--no-mcp/skills don't remove pre-existing mcp/skills.

instrument:

by default, instrument
--(no-)instrument (un)select instrumentation
--no-instrument still configures mcp and skills, it's only the agent that isn't launched

agent:

if --agent AGENT flag is passed, use the specified agent
else if only one exists in [claude code, codex, cursor, opencode] (ie only one exist in PATH), use it
otherwise, prompt the user

workflow docs:

if one or multiple --workflow WORKFLOW flags are passed, fetch these workflows
otherwise, fetch all workflows

language:

try to identify the language based on file like pyproject.toml in the project root/sub folders (in current folder and subfolder with depth 1)
if nothing is found, alter the start prompt of the agent to find which language to instrument

TUI/background:

TUI by default
if --background is passed, launch the agent in the background
--tui exist only to conflict with --background (and --no-instrument)
I'm not really sure how to handle --background without --yolo. The agent will often try to use restricted command, but can't get user approval.

yolo mode

self explanatory for claude, codex, cursor
opencode is in yolo mode by default for the directory it is launched in
Besides, --yolo or equivalent don't exist in opencode yet (don't trust your LLM about this, there are many closed/merged/open PRS about this but just check opencode -h and you'll see it's not there).

interactive mode

-i/--interactive
Prompt the user to choose the configuration instead of passing options. Does not ask questions which are already answered by options.

$ bt setup -i
✔ Install scope · global (user-wide)
Set up coding-agent skills? yes
Set up MCP server? yes
Run instrumentation agent? yes
✔ Select agent to instrument this repo · claude
✔ Select additional workflow docs to prefetch (instrument is always included) · observe, annotate, evaluate, deploy
✔ How do you want to run the agent? · Interactive (TUI)
Grant agent full permissions? (bypass permission prompts) yes

When opening the agent TUI, a wall of text (the prompt) is printed: do we want this?

Bugs

json is very buggy
project isn't asked

Untested/poorly tested:

env variables
no-color, api/app-url, env-file, no-fetch-docs, refresh-docs, workers

TODO

Better help message in the interactive wizard
better json objects, they are not consistent

[github-actions]

Latest downloadable build artifacts for this PR commit 73568b445d8e:

• Workflow run: https://github.com/braintrustdata/bt/actions/runs/24109170512
• Download all artifacts (GitHub CLI): gh run download 24109170512 --repo braintrustdata/bt
• Installers are published from main automatically. To publish one for a PR branch, run release-canary manually via workflow_dispatch.

Available artifact names

• ``artifacts-build-global
• ``artifacts-build-local-x86_64-apple-darwin
• ``artifacts-build-local-x86_64-pc-windows-msvc
• ``artifacts-build-local-x86_64-unknown-linux-musl
• ``artifacts-build-local-aarch64-apple-darwin
• ``artifacts-build-local-x86_64-unknown-linux-gnu
• ``artifacts-build-local-aarch64-unknown-linux-gnu
• ``artifacts-build-local-aarch64-unknown-linux-musl
• ``artifacts-plan-dist-manifest
• ``cargo-dist-cache

[ViaDézo1er / cedric (viadezo1er)]

Not really sure how to test this but there is definitely a lack of tests.
OAuth link
TUIs
interactive wizard

[ViaDézo1er / cedric (viadezo1er)]

bt auth profiles does nothing