Skip to main content
Connect your Braintrust organization to let CloudThinker agents answer how an experiment scored, what a monitor shows, and how a project is set up. Braintrust authenticates with an API key, and the connection is read-only.

​
Prerequisites

  • A Braintrust account in an organization whose data plane is in the US, Braintrust’s default.
  • The ability to create an API key in your organization settings.
CloudThinker connects to Braintrust’s US endpoint, api.braintrust.dev. Braintrust serves EU data plane organizations and self-hosted deployments from different endpoints, which this connection does not use.

​
Setup

1

Open API keys

Sign in to Braintrust and go to Settings → API keys.
2

Create the key

Click + API key and enter a name such as cloudthinker. Choose an expiration, which defaults to one year and cannot be changed after creation. Click Create.
3

Copy the key

Copy the key right away. Braintrust shows it only once and cannot recover it later, so a lost key means creating a new one.
4

Add the connection in CloudThinker

Go to Connections → Braintrust, paste the key into BRAINTRUST_API_KEY, and click Connect. CloudThinker shows a Connected status.

​
Connection details

​
Required permissions

Braintrust says an API key inherits the permissions of the user who created it, and that a personal API key cannot be scoped below that user’s own permissions.
Create the key from an account that only needs to read. On the Pro and Enterprise plans, put that account in the built-in Viewers group, which Braintrust describes as read-only access to all projects and resources in the organization. On the Starter plan only the Owners group is available, so a key made there carries owner access.

​
Agent capabilities

Once connected, agents use a fixed set of read-only actions. Agents report names and counts, and show an object ID only when you ask for it. They cannot run SQL queries over logs, list automations, or create, edit, or run anything in Braintrust.

​
Verify the connection

​
Example prompts

​
Troubleshooting

Braintrust’s API returns 401 when no valid API key is provided. The key may be mistyped, deleted, or past its expiration date. Create a new key and update the connection.
Braintrust’s API returns 403 when the key lacks permission for the request. The key has only the access of the account that created it, so check that account’s access to the project or experiment.
Braintrust API keys can expire, and a new key defaults to one year. An expired key stops authenticating and cannot be renewed. Create a new key and update the connection.
The connection is read-only by design and covers only the actions in the table above. Run SQL queries, edit datasets, and create alerts or scorers in Braintrust itself.

​
Security

  • Least privilege — grant only the permissions the agents need for your use case; start read-only and widen later.
  • Read-only by default — use read-only credentials unless you want agents to make changes through this connection.
  • Rotate credentials — rotate keys and tokens on your normal schedule; CloudThinker picks up the new value when you update the connection.
  • Revoke on offboarding — remove the credential at the provider when you delete a connection or a teammate leaves.
  • Expiration — set an expiration you can track, and rotate the key before it lapses.
  • Account scope — the key acts as its creator, so create it from an account with only the access agents need.

Langfuse Connection

LLM traces, prompts, and evaluations

Datadog Connection

Log search, metrics, and monitoring