Braintrust

Trace OpenAI Codex sessions in Braintrust with the `trace-codex` plugin. Add the plugin to your Codex setup to capture sessions as hierarchical traces. Works in interactive sessions and CI runs, with config options for projects, metadata, and flush behavior. Read more → https://lnkd.in/g6GKU6cH

Kimi K3 and DeepSeek V4 Flash are now available as built-in models on Braintrust, joining GLM-5.2. Open models keep improving, and it should be easy to test candidate models against your own prompts, datasets, and production traces. We ran all three on 327 of the hardest MathTutorBench dialogs. They all outperformed the human teacher baseline, but each had a different strength. GLM-5.2 led on quality, DeepSeek V4 Flash on speed, and Kimi K3 on token cost. Read more → https://lnkd.in/gTPinQr4

Eve's legal agents analyze thousands of documents, build medical chronologies, summarize eight-hour depositions, run legal research across case law and statutes, and draft work product for cases. To keep that quality consistent, they built Plaintiff Bench, a benchmark of 400+ legal tasks across 40+ states, tagged by failure mode. They use Braintrust to run the benchmark on every agent change, select models with evidence from lawyers, and turn every production failure into a future test case. Read more → https://lnkd.in/gk4zn8zn

New in the experiments page: - The analysis chart now includes options for All scores (avg) and Scale by axis, so you can plot duration, cost, or any metric against your scores - A new summary table that compares scores across all selected experiments and highlights the best and worst performers per row. Read more → https://lnkd.in/g2EXsJEu

Recent research by Alex Zhang found that training with an RLM harness can help models learn strategies that transfer to new tasks. In the RLM setup he studies, the harness lets the model decompose long tasks into smaller calls and combine their results, resulting in far better generalization to longer and unseen tasks than training the model alone. We wanted to test whether the same mechanism could carry the wrong strategy. So we evaled whether two tasks that look alike to the harness but require different ways of combining their results could help one but hurt the other. In our tests, we found no evidence that it did. This null result helps clarify the boundaries of the original finding. Read more → https://lnkd.in/gcxbcF8d

Cloudflare's dashboard agent spans its entire developer platform, from deploying Workers to debugging production instances. Keeping that much surface area performing well requires more than gut checks. Their team uses Braintrust to run LLM-as-a-judge scorers to measure conversation resolution, gates every skill and prompt change with evals in CI/CD, and benchmarks sub-agents against models of varying capacity before choosing which to use. Read more → https://lnkd.in/gqepH33C

Trust the agents you build on your data An online workshop from Braintrust and MotherDuck. Aug 12 at 10AM PT. Led by Jessica Wang and Jacob Matson. Learn how to: - Build an agent that answers questions over your data - Give that agent the context it needs to return accurate answers - Turn production data into an eval dataset Join us → https://lnkd.in/gywFbfwq

After a successful first edition in NYC, we’re bringing The AI Dev Stack to San Francisco. Join Fireworks AI, Braintrust, Browserbase & Antimetal on Aug 18 to see how teams are building the modern development stack beyond coding agents. RSVP Required. https://lnkd.in/g6-GG9gM

If you build agents with Cloudflare, you can send traces to Braintrust to see how these agents behave, eval them, and improve them over time. Export traces via OpenTelemetry, instrument your agents in JavaScript on the Workers runtime, or integrate Braintrust with Flue. Read more → https://lnkd.in/g8bARij8