Braintrust’s September 24 Nitro release is a useful signal for where agent infrastructure is heading. The feature sounds narrow at first: a new asynchronous query execution engine for Brainstore, the company’s trace storage and search system. But the reason Braintrust built it is broader than search latency. Production AI agents are creating observability workloads that look less like a developer opening a few trace pages and more like another automated system running hundreds of investigations against the trace corpus.
The central lesson is that agent observability is becoming an infrastructure problem, not only a dashboard problem. Once developers use coding agents and analysis agents to inspect other agents, every incident can trigger many parallel searches across prompts, model outputs, tool calls, retries, logs, and evaluations. Systems that were fast enough for human-paced debugging can become bottlenecks when the investigator is also automated.
Nitro addresses that pressure by separating object-storage reads from CPU-bound processing. Braintrust says the new engine keeps the existing Tantivy-based storage format but replaces the synchronous execution path with an asynchronous one that schedules storage and compute independently. In benchmarks across its customer base, the company says full-text searches ran more than twice as fast on average. Nitro is automatically enabled for Braintrust SaaS and bring-your-own-cloud customers, and is enabled for self-hosted customers starting with dataplane 2.15.
Why Agent Debugging Changes the Shape of Search
Traditional observability tools were designed around a familiar loop: a human sees an alert, opens a trace or log view, filters a few fields, searches for a phrase, and reads the result. Even when the data volume is high, the query pattern is often paced by human attention. The operator pauses to think. The system has time to return results before the next step begins.
Agentic debugging breaks that rhythm. A coding agent can investigate a failure by issuing many queries at once, comparing successful and failed traces, checking whether a tool retried in a loop, looking for context growth, finding semantically similar failures, and testing hypotheses against historical examples. The value of that workflow comes from breadth and speed. The agent does not need to wait politely between searches the way a human would.
That shift matters because AI trace data is unusually heavy. A single trace can include long system prompts, retrieved documents, tool inputs, tool outputs, model responses, screenshots, structured state, evaluator scores, and nested steps. A production support agent, coding agent, or browser agent may generate hundreds of spans for one task. Searching across those spans is not the same as querying a small metrics table.
Braintrust’s post describes this directly: agents such as Codex and Claude Code are increasingly being used to ask questions about other agents. They search trace history, run more investigations, and send off more queries. That is the workload Nitro was built for. The feature is not only about making one query faster; it is about avoiding collapse when many useful queries arrive together.
The Object Storage Tradeoff
Brainstore was built on object storage because trace history needs durability, low cost, and the ability to grow without forcing every query server to keep a full local copy of the data. That architectural choice matches the economics of observability: most traces may be stored for later investigation, while only a fraction are hot at any given time.
The hard part is that search engines and object storage have different instincts. Braintrust uses Tantivy to organize search indexes. Tantivy is efficient with local storage, where memory-mapped files let the operating system load data on demand and reuse recently read pages. Small sequential reads are cheap when the data is close to the CPU.
With object storage, a read that misses the local cache can become a network request. Braintrust gives the example of finding a posting list, the list of documents containing a particular word. The path can require several dependent reads: checking the end of a file to find footer size, reading the footer to locate the dictionary, reading the dictionary to locate the posting list, and then reading the posting list itself. Even if the amount of data is small, the dependency chain adds round trips.
In a synchronous execution model, workers alternate between waiting for storage and doing compute. Adding more workers can overlap more reads, but it also increases the amount of CPU work that can become runnable when those reads return. If storage and compute share the same concurrency limit, an attempt to hide latency can oversubscribe CPU and memory.
This is a familiar cloud systems tradeoff, but agent workloads make it more visible. When an automated investigator launches a burst of searches, the system needs to overlap network-bound work without letting the follow-on compute work flood the machine. Search latency, memory pressure, and query fairness become part of the agent reliability story.
What Nitro Changes
Nitro’s core design is to schedule storage and compute separately. A query waiting for object storage does not occupy a compute worker. Reads can overlap across index segments, and CPU work is dispatched when the required data is available. Braintrust describes phrase matching as one example: Nitro can use separate concurrency limits for reading word positions and checking those positions. Once reads finish, the read semaphore is released, and the compute-heavy position check waits for a compute slot.
That distinction is important for platform teams because it acknowledges that not all query work consumes the same scarce resource. A storage wait consumes time and network concurrency. A phrase match or decode step consumes CPU and memory. Treating them as separate queues lets the system use object storage aggressively without pretending every pending request should also be allowed to execute CPU-intensive work immediately.
Nitro also reduces unnecessary object-storage requests. Braintrust says it now prefetches more aggressively in cases such as reading the end of a file for footer information. If the file is small enough, it can read the whole thing in one request; for larger files, the data needed may still fit in the initial read. The engine also coalesces nearby reads, combining small adjacent reads into fewer requests when searching terms, postings, or positions.
These details are not glamorous, but they are exactly where production agent systems often succeed or fail. The more autonomous the debugging loop becomes, the more the bottleneck moves from the model alone to the data paths around the model. Agents need context, and context often lives in storage systems that were not tuned for automated, high-concurrency investigation.
Why This Matters for Agentic AI Teams
For teams building production agents, Nitro highlights a practical planning issue: observability costs and latency will rise with the sophistication of the debugging workflow. It is tempting to evaluate agent observability platforms by looking at trace capture, visualization, and evaluator configuration. Those features matter, but they are only part of the system. Once a team starts asking agents to investigate failures, query execution becomes a first-class capability.
This is especially relevant for teams moving from demos to production support, software engineering, commerce, or internal operations. Early agent failures are often inspected manually. A developer opens a few traces, finds an obvious prompt issue, and patches it. At higher volume, the failures become more varied: a tool times out only for certain accounts, a retrieval result changes a workflow path, a model chooses a costly fallback, or an agent loops because a tool response is technically valid but semantically incomplete.
Those cases benefit from broad historical investigation. The useful question is rarely, “What happened in this one trace?” It is more often, “How often does this happen, which accounts does it affect, what changed after the last release, and do successful traces show a better path?” Answering that quickly requires trace search, structured filters, evaluators, datasets, and release comparisons to work together.
Braintrust has been building around that connected loop. Earlier in September, the company announced Patterns, Debugger, and an enhanced Loop experience for agent observability. Patterns is designed to identify recurring behaviors across production traces, while Debugger explains likely failure modes in individual runs. Loop can investigate open-ended questions, repeat analyses on a schedule, and carry evidence into datasets, evaluators, dashboards, and monitors. Nitro strengthens the storage and query layer underneath that style of workflow.
The Practitioner Takeaway
The immediate takeaway is not that every team needs to build its own asynchronous search engine. Most should not. The lesson is to design agent operations with the assumption that agents will inspect agents. That means the observability substrate needs to handle bursts of automated investigation, not just human browsing.
Engineering leaders should ask different questions when selecting or operating an agent observability stack:
- Can the system search across full prompts, model outputs, tool calls, and long traces without forcing operators to sample away the evidence they need?
- Does query performance hold up when automated investigations run many searches in parallel?
- Are storage and compute scaled or limited independently, especially when trace history lives in object storage?
- Can findings move directly into datasets, evaluators, monitors, or regression tests?
- Is the cost of retaining trace history low enough that teams can investigate failures after the fact?
The answers will shape how confidently teams can deploy agents into workflows where failures are subtle and recurring. A customer support agent that invents a refund, a coding agent that repeats a tool call, or a research agent that silently drops a constraint may not trigger a simple uptime alert. The signal lives in traces and comparisons. If the system cannot search those traces quickly, the feedback loop slows down.
Agent Observability Is Becoming Active Infrastructure
The larger trend is that observability is moving from passive recordkeeping to active infrastructure. Logs and traces still need to be captured, but the competitive edge comes from turning them into questions, datasets, evals, fixes, and monitors. Automated investigation raises the ceiling on what teams can learn from production, but it also raises the floor for the storage and query systems that support that investigation.
Nitro is a specific implementation inside Braintrust, but the pattern is likely to spread. Agent platforms will need data systems that understand long traces, mixed structured and unstructured content, concurrent analysis, and the economic reality of object storage. Tooling that treats observability as a thin UI over logs will feel increasingly strained as agent workloads grow.
For practitioners, the important shift is mental. The agent is no longer only the thing being observed. It is becoming one of the observers. That changes the infrastructure requirements around trace search, evaluation, and incident response. Braintrust’s Nitro release matters because it treats that shift as a systems problem, and systems problems are where many production AI programs are now arriving.
