Codex

Codex is OpenAI’s command-line AI coding assistant. You can integrate Codex with Braintrust in two ways:

Trace Codex sessions: Install the trace-codex plugin to trace Codex CLI sessions to Braintrust as session, turn, model call, and tool spans.
MCP server: Configure Codex to access your Braintrust projects, experiments, and logs through the Braintrust MCP server.

Trace Codex sessions

The trace-codex plugin traces your Codex CLI sessions to Braintrust. Each session produces a hierarchy of spans:

Session spans (codex: <dir> or codex session): One root span per session, with the working directory, model, source, and permission mode.
Turn spans (turn: <turn_id>): One span per user turn, with the prompt as input and the agent’s final response as output.
LLM spans (named with the model slug): One span per model call within a turn, with conversation history as input and response output; includes prompt, completion, cached, and reasoning token counts.
Tool spans (named with the tool name): One span per tool execution, with the tool’s input and output. Tools that request escalated permissions are tagged permission-request.
Subagent spans (subagent: <agent_id>): When Codex spawns a sub-agent via the spawn_agent tool, a dedicated span is created under the spawning turn to root the sub-agent’s session. The sub-agent’s turns, LLM calls, and tool spans nest inside it.
Compaction tags: When context compaction occurs (manually or automatically), the turn that triggered it is tagged compaction.

Setup

Install Codex

If you haven’t already, install Codex.

Add the plugin marketplace

codex plugin marketplace add braintrustdata/braintrust-codex-plugin

Add the trace-codex plugin

Install the trace-codex plugin from the marketplace:

codex plugin add trace-codex@braintrust-codex-plugins

The first time you run Codex after installing, it prompts you to trust the plugin’s hooks. Accept the prompt to enable tracing.

Enable tracing

Set your API key and enable tracing before starting Codex:

export BRAINTRUST_API_KEY="your-api-key-here"
export TRACE_TO_BRAINTRUST=true
export BRAINTRUST_PROJECT=my-codex-project  # Replace with your project name
codex

Configuration

Each setting can be provided as a config.json key or as an environment variable. Environment variables always override config.json.

`config.json` key	Environment variable	Default	Description
`traceToBraintrust`	`TRACE_TO_BRAINTRUST`	`false`	Enable tracing. Tracing is disabled when this is `false` or unset.
`apiKey`	`BRAINTRUST_API_KEY`	—	Braintrust API key.
`project`	`BRAINTRUST_PROJECT`	`"codex"`	Project to log traces into.
`apiUrl`	`BRAINTRUST_API_URL`	`api.braintrust.dev`	Braintrust API URL. For EU organizations, use `https://api-eu.braintrust.dev`. For self-hosted deployments, use your data plane URL.
`additionalMetadata`	`BRAINTRUST_ADDITIONAL_METADATA`	—	JSON object of extra metadata merged into the root span. Standard keys take precedence on conflict.
`flushOnTurnEnd`	`BRAINTRUST_FLUSH_ON_TURN_END`	`false`	Block at each turn’s end until all spans are delivered to Braintrust. Recommended for CI runs.
`recordFile`	`BRAINTRUST_EVENT_SERVER_RECORD_FILE`	—	Record every event to this NDJSON file for replay and debugging.

To use a config file, copy the example from the plugin cache:

cp ~/.codex/plugins/cache/braintrust-codex-plugins/trace-codex/<version>/config.json.example \
   ~/.codex/plugins/data/trace-codex-braintrust-codex-plugins/config.json

Using in CI

To trace codex exec runs in CI, set BRAINTRUST_FLUSH_ON_TURN_END=true so spans are delivered before the job exits, and pass --dangerously-bypass-hook-trust to skip the interactive hook-trust prompt.

- name: Run a traced Codex session
  env:
    TRACE_TO_BRAINTRUST: "true"
    BRAINTRUST_API_KEY: ${{ secrets.BRAINTRUST_API_KEY }}
    BRAINTRUST_PROJECT: my-codex-project
    BRAINTRUST_FLUSH_ON_TURN_END: "true"
  run: |
    codex exec \
      --dangerously-bypass-hook-trust \
      --sandbox read-only \
      "summarize the changes in this repo"

Pin the marketplace to a specific release tag in CI for repeatable builds: codex plugin marketplace add braintrustdata/braintrust-codex-plugin@trace-codex-v0.0.2. See the braintrust-codex-plugin releases for available tags.

Resuming sessions

When you resume a Codex session, the trace continues in the same root span. The plugin snapshots session state to disk so later turns attach to the original trace even after the background server has restarted. When resuming, the original session’s tracing settings remain in effect. If the original session had TRACE_TO_BRAINTRUST=true, the resumed session is also traced regardless of the current environment.

MCP server

The Braintrust MCP server integrates with Codex, giving it access to your Braintrust projects, experiments, and logs.

Setup

Install Codex

If you haven’t already, install Codex.

Set your API key

Set the BRAINTRUST_API_KEY environment variable with your API key:

export BRAINTRUST_API_KEY="your-api-key-here"

Add the Braintrust MCP server

Edit ~/.codex/config.toml and add the Braintrust MCP server configuration:

[mcp_servers.braintrust]
url = "https://api.braintrust.dev/mcp"
bearer_token_env_var = "BRAINTRUST_API_KEY"

This configures Codex to read your Braintrust API key from the BRAINTRUST_API_KEY environment variable.

Verify the setup

Launch Codex with the environment variable set:

codex

Run the /mcp command to verify Braintrust is installed and accessible.

Usage

Once configured, Codex can access Braintrust data through the MCP server. You can fetch experiment results, query logs, log data, and more. See MCP documentation for details. Example prompts in Codex:

“Show me my recent Braintrust experiments”
“Query the last 10 logged requests with errors”
“What’s the average latency for the summarizer prompt today?”
“Compare accuracy scores between my GPT-4 and Claude experiments”

Troubleshooting

Traces not appearing in Braintrust

Verify TRACE_TO_BRAINTRUST is set to true.
Verify BRAINTRUST_API_KEY is set in the shell where you launch Codex.
Confirm the plugin is installed: run codex plugin list and check that trace-codex appears.
Check that you trusted the plugin’s hooks when Codex prompted you, or pass --dangerously-bypass-hook-trust for non-interactive runs.

BRAINTRUST_API_KEY not found

Verify the variable is set: echo $BRAINTRUST_API_KEY (macOS/Linux) or echo %BRAINTRUST_API_KEY% (Windows)
Ensure you’ve reloaded your shell after setting the variable
Check that Codex is launched from a shell where the variable is set

MCP server not appearing

Verify the TOML configuration syntax is correct
Check that the file path is exactly ~/.codex/config.toml
Run /mcp in Codex to see available MCP servers
Try restarting Codex

Authentication fails

Verify your API key is correct (no extra spaces)
Ensure you can log into Braintrust using the account associated with the API key
Generate a new API key if needed

Connection errors

Verify the URL is exactly https://api.braintrust.dev/mcp (no trailing slash)
Check your internet connection
Corporate networks may need to allowlist api.braintrust.dev and *.braintrust.dev

Next steps

Run evaluations: Check out the evaluation guide to learn evaluation patterns.
Explore MCP tools: See the MCP documentation for all available commands.
Query with SQL: Learn how to query with SQL for complex data analysis.
Browse the source: The braintrust-codex-plugin repository contains the plugin source code.

AI providers

SDKs

Developer tools

Trace Codex sessions

Setup

Configuration

Using in CI

Resuming sessions

MCP server

Setup

Usage

Troubleshooting

Next steps

​Trace Codex sessions

​Setup

​Configuration

​Using in CI

​Resuming sessions

​MCP server

​Setup

​Usage

​Troubleshooting

​Next steps

Trace Codex sessions

Setup

Configuration

Using in CI

Resuming sessions

MCP server

Setup

Usage

Troubleshooting

Next steps