AWS has published two reference patterns for observing AI agents on Amazon Elastic Kubernetes Service: one sends telemetry from each agent directly to an observability backend, while the other routes it through a shared OpenTelemetry Collector. The diagrams may look like a familiar observability comparison, but the important change is what the telemetry represents. An agent request can cross model calls, tool invocations, gateways, retries, and fallback paths while accumulating latency and token cost. A healthy pod and an HTTP 200 response no longer prove that the workload behaved correctly.
That makes the choice between direct export and centralized collection more than a deployment detail. It determines who can govern sampling, redact sensitive attributes, change backends, enforce cost metadata, and diagnose a broken reasoning path. AWS’s blueprint is therefore useful beyond EKS: it shows that AI-agent observability is becoming a platform capability, and that Kubernetes teams should decide its control boundary before their agent fleet grows.
What AWS’s EKS reference architecture adds
The AWS design places agents built with the Strands Agents SDK, the Bifrost LLM gateway, Amazon Bedrock, and Langfuse on EKS. OpenTelemetry links the layers. Agent instrumentation creates spans for model and tool activity; trace context travels to the gateway; and the backend reconstructs the request as a connected trace. AWS then implements two ways to move those spans.
In the decentralized pattern, each agent exports telemetry directly to Langfuse over OTLP. The gateway also exports its spans to the same destination. This removes a shared collector from the data path and gives each workload team ownership of its exporter configuration.
In the centralized pattern, the agents and gateway send telemetry to a shared OpenTelemetry Collector. The Collector batches, processes, and exports the data to Langfuse. It becomes the point where operators can add attributes, filter data, apply sampling, or introduce another destination without rebuilding every agent.
The two patterns use essentially the same instrumentation and trace propagation. That is precisely why the comparison matters: platform teams can separate the question “Can we trace an agent?” from “Who controls what happens to those traces?” The first is an application instrumentation problem. The second is an operating-model decision.
Agent traces expose a different failure surface
Conventional Kubernetes monitoring starts with infrastructure and service health: CPU, memory, restarts, request rates, error rates, and latency. Those signals remain necessary, but an agent can be technically healthy while producing an expensive or incorrect result. It may call the wrong tool, repeat a tool call, choose an unexpectedly costly model, lose context between services, or follow a long path that still returns successfully.
A useful agent trace therefore needs more than a top-level request span. Operators need to correlate the agent’s execution with model identity, token consumption, tool selection, tool latency, gateway behavior, and the final outcome. When a fallback model is used after throttling, that transition should be visible in the same trace. When one tool dominates response time, the trace should make that obvious. When a request burns an abnormal number of tokens, teams need enough attributes to allocate the cost to a workload, namespace, team, or tenant.
Kubernetes adds another layer of correlation. The logical trace must connect to runtime facts such as deployment version, pod, cluster, namespace, and region. Without that bridge, an AI-observability interface can explain what the agent attempted but not whether the behavior coincided with a rollout, resource pressure, network fault, or a particular cluster. The real operational goal is a trace that crosses both semantic layers: what the agent decided and where the software executed.
Direct export optimizes for speed and team autonomy
Direct-to-backend export is attractive when a team is proving value or runs a small number of agents. There is no central Collector deployment to size, patch, scale, and monitor. An exporter failure in one workload does not automatically interrupt telemetry from every other workload. Teams can move independently and keep the data path easy to understand.
Those benefits are real, especially during early adoption. A platform group should not force a complex shared service into the critical path merely because centralization looks mature on an architecture diagram. If there are only a few agents, one backend, modest data volume, and clear workload ownership, direct export can be the most economical design.
The cost appears when local choices become fleet-wide policy. Changing an endpoint, adding a second backend, adjusting sampling, or applying a new redaction rule may require edits and rollouts across every agent. Export credentials and destination configuration spread through multiple deployments. Different teams may attach different attributes or use inconsistent names, making cross-service queries unreliable. The organization can end up with traces that exist but cannot be compared.
Direct export also shifts retry queues and backpressure closer to the application. Teams need to understand exporter behavior when the backend slows down or becomes unavailable. Telemetry must not consume unbounded memory, amplify an outage, or interfere with the agent’s primary work. A simple topology does not remove these failure modes; it distributes responsibility for them.
A shared Collector creates a governance boundary
The centralized pattern gives a platform team one place to govern telemetry after applications emit it. The OpenTelemetry project describes the Collector as a vendor-agnostic service for receiving, processing, and exporting telemetry, and specifically calls out retries, batching, encryption, and sensitive-data filtering as reasons to use it alongside services. For agent workloads, those functions map directly to platform concerns.
Central processing can enforce resource attributes, drop high-risk fields, route tenants to separate destinations, and sample routine traffic while retaining errors or costly outliers. It can buffer brief backend interruptions and allow a backend migration without coordinating a fleet-wide application release. A common pipeline can also make cost attribution more consistent by validating the metadata required for showback or chargeback.
But the Collector is not free control. It becomes shared production infrastructure. If undersized, it can drop the very evidence needed during an incident. If configured carelessly, it can expose prompt content or tool arguments to destinations that should not receive them. If every cluster exports through one fragile deployment, the architecture creates a broad failure domain.
Kubernetes teams should treat the Collector as a service with explicit capacity and reliability objectives. That means monitoring its queue depth, refused spans, export failures, memory use, and processing latency; setting resource limits with headroom; and testing backend outages. High-volume environments may need gateway collectors behind a service, workload-local or node-local collectors for fast offload, or separate pipelines for tenants with different compliance requirements. “Centralized” should describe policy ownership, not necessarily one global process.
The decision should follow scale, not fashion
A practical decision starts with ownership and expected change. Direct export is a reasonable default when one team owns the agents and backend, the environment is experimental or small, and telemetry policy changes infrequently. Central collection becomes more valuable as the number of agents, teams, clusters, and destinations increases.
Platform leaders should ask five questions before choosing:
- How many independent workloads and teams will emit agent telemetry? More owners increase the cost of coordinating exporter changes.
- Does telemetry contain prompts, tool arguments, customer identifiers, or retrieved data? If so, filtering and access boundaries need deliberate ownership.
- Will sampling depend on outcome, latency, or cost? Central processing can apply consistent rules, but the design must preserve the attributes needed to make those decisions.
- How many backends or tenants must receive data? Routing through a Collector reduces application coupling to destinations.
- What happens when the telemetry path fails? Teams need defined behavior for queues, drops, retries, and application isolation in either topology.
There is also a viable migration path between the patterns. Teams can begin with direct OTLP export to validate instrumentation, trace correlation, and operational value. They can then place a Collector at the same endpoint boundary as governance needs emerge. AWS’s comparison suggests this can be done without changing the conceptual trace model, provided applications use standard protocols and keep destination-specific logic out of business code.
What platform teams should standardize now
The first standard should be context propagation. Agent, gateway, and tool calls need to carry compatible trace context so the backend can reconstruct one request. A dashboard full of disconnected spans is not an operational view. Teams should test propagation across synchronous calls, asynchronous work, retries, and model fallbacks.
The second standard should be a minimal attribute contract. At a minimum, teams need stable workload ownership, environment, cluster, namespace, deployment version, model, and outcome dimensions. Token and cost fields should use consistent units and naming. High-cardinality or sensitive values should be governed explicitly rather than copied into every span by default.
The third standard should be data handling. Prompts and model responses can contain secrets, personal data, proprietary context, or customer content. Capturing everything may improve debugging while creating a larger security and compliance problem. Platform teams should define what is recorded, what is redacted, who can query it, and how long it is retained. The Collector can enforce part of that policy, but instrumentation must avoid emitting data that should never leave the process.
Finally, teams should measure the observability system itself. Trace completeness, dropped spans, end-to-end export delay, sampling rate, and telemetry cost deserve service-level indicators. Agent observability only helps if operators trust that an absent span means something other than an overloaded pipeline.
Why this matters beyond one AWS stack
The named components in AWS’s implementation are specific, but the architectural pressure is not. Any Kubernetes platform hosting agents will face the same tension between workload autonomy and shared governance. Models and agent frameworks will change faster than the platform underneath them. OpenTelemetry and OTLP offer a way to preserve a stable collection boundary while those higher layers evolve.
The most important takeaway is not that every EKS cluster needs a shared Collector. It is that the telemetry topology should be chosen consciously, with ownership, data sensitivity, reliability, and fleet scale in view. Agent workloads make application behavior harder to infer from ordinary service metrics, so observability architecture moves closer to the platform’s control plane. Teams that define that boundary early will be better positioned to manage cost, troubleshoot behavior, and change vendors without rebuilding every agent.
