Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Ken Jiang (knjiang)
force-pushed
the
04-01-native_inference_billing
branch
from
93e0bb7 to
4f147d0
Compare
| # You should not need to edit this | ||
| BRAINTRUST_APP_URL = "https://www.braintrust.dev" | ||
| METRICS_LICENSE_KEY="<YOUR_METRICS_LICENSE_KEY>" | ||
| BILLING_TELEMETRY_URL="https://api.braintrust.dev/billing/telemetry/ingest" |
There was a problem hiding this comment.
is this how u configure the proxy?
There was a problem hiding this comment.
i think these are sample variables. i've had a bit of a tough time figuring out how to get variables to "stick". I think you have to add them as secrets in github, and configure the action to export them. Otherwise, they can get overridden each time you deploy.
Ken Jiang (knjiang)
force-pushed
the
04-01-native_inference_billing
branch
from
9a9655a to
b8426af
Compare
apis/cloudflare/src/billing.ts
Outdated
| function isBrainModel(model: string): boolean { | ||
| return model.startsWith("brain-"); |
There was a problem hiding this comment.
i added this guard after testing with gpt-5-mini
There was a problem hiding this comment.
there's a more precise definition, which is that
responseJson.encrypted === true &&
this comes back in the response from the server. I don't think we propagate this through to the secret, although for all intents and purposes, if overrideModel is set, then it is native inference. Maybe the latter is a better thing to look for
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b8426af769
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| let spanId: string | undefined; | ||
| let spanExport: string | undefined; | ||
| let billingOrgId: string | undefined; | ||
| const orgName = request.headers.get(ORG_NAME_HEADER) ?? undefined; |
There was a problem hiding this comment.
Billing org lookup is keyed only by ORG_NAME_HEADER, but proxy routing can override org from the URL path (/btorg/{org}/... in packages/proxy/src/proxy.ts). In that case, billingOrgId is resolved for the key’s default/header org while the request is actually executed under the path org, so telemetry can be attributed to the wrong external_customer_id and corrupt billing for multi-org keys.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
hmmm i think this is fine? how do u make a mutli-org api key in braintrust today?
| const apiKey = | ||
| parseAuthHeader({ | ||
| authorization: request.headers.get("authorization") ?? undefined, | ||
| }) ?? undefined; |
There was a problem hiding this comment.
This API key extraction only reads Authorization, but proxyV1 also supports x-api-key authentication. For clients that authenticate via x-api-key (common in Anthropic-compatible flows), billingOrgId remains unset and buildPayloadEvent skips the telemetry because org_id is missing, so native inference usage is silently not billed.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
this is fine, we'll get warning logs if we can't build the payload without org id.
i just moved the logic out of BT_PARENT conditional.
2 participants
This starts emitting billing events for models with the prefix of
brain-*slack thread has my QA testing.
TODO: wire up embeddings