Cloudflare Adds AI Spend Controls, Envoy Patches Security Holes, and Prometheus Drops v3.12.0: Cloud Native Gets Cost-Aware

The cloud-native stack has always been about two things: performance and reliability. But in 2026, a third dimension is demanding equal attention: cost control. As organizations pour AI workloads into Kubernetes clusters and route millions of tokens through service meshes, the infrastructure layer is being forced to evolve beyond raw throughput. It now needs to understand budgets, enforce spend limits, and attribute costs to individual users and teams.

Over the past two weeks, three major developments across the CNCF ecosystem have reinforced this shift. Cloudflare shipped identity-driven spend controls for AI Gateway, addressing the runaway-token-bill problem that keeps CFOs awake at night. Envoy released a security-focused patch that hardens the service mesh against HTTP/2 and OAuth2 attacks. And Prometheus dropped v3.12.0 with new PromQL functions, performance optimizations, and expanded service discovery. Together, these updates paint a picture of a maturing ecosystem that is learning to handle AI-era scale without sacrificing governance or security.

The AI Spend Problem Is Real

Every company that has deployed AI at scale has the same story. Engineering teams were given broad access to frontier models through shared API keys. Usage exploded. At the end of the month, finance pulled the invoice and nobody could explain where the money went. Was it the ML team training a new pipeline? Was it an intern experimenting with Claude Opus? Was it a CI/CD job that accidentally burned through fifty million tokens over a weekend?

Without per-user attribution, model routing logic, or budget enforcement, the rational move for any engineer is to reach for the biggest model available for every task. A log parser does not need the same model as a customer-facing content generator, but if there is no cost signal, there is no reason to differentiate. The result is predictable: ballooning bills, poor ROI, and a growing tension between engineering velocity and financial governance.

Cloudflare’s answer, announced on June 5, is AI Gateway spend limits.

Cloudflare AI Gateway: Budgets for Token Consumption

AI Gateway is Cloudflare’s unified routing layer for AI providers. Instead of calling OpenAI, Anthropic, Google, or any other provider directly, applications route requests through AI Gateway first. This gives organizations unified billing, cross-provider logging, response caching, rate limiting, and content guardrails in a single control plane.

What was missing until now was cost control at the granularity that enterprises actually need. The new spend limits feature changes that. Budgets are set in dollars, not tokens, and they track cumulative spend across all requests in real time. Limits can be scoped by model, provider, or custom attributes like user, team, or application. Windows can be fixed (resetting on the first of the month or at midnight) or rolling, with daily, weekly, or monthly cadences.

When a budget is hit, AI Gateway can block further requests by default. Alternatively, organizations can configure Dynamic Routes to fall back to a cheaper model, ensuring that a hard spending cap does not kill engineers’ workflows. Cloudflare is also working on alert notifications when limits are approached.

Crucially, the spend limits are in open beta for all AI Gateway users across all plans. This is not an enterprise-only feature. It is available now.

Identity-Driven Budgets: Who Spent What

The spend limits feature is powerful on its own, but it becomes genuinely transformative when combined with identity. Cloudflare is also running a closed beta for identity-driven budgets and policies, integrated with Cloudflare Access and existing identity providers.

Here is how it works in practice. When an employee authenticates via Cloudflare Access, their identity is extracted from the JWT and attached as metadata on the AI Gateway request. This enables per-user token consumption tracking, team-level usage breakdowns, and cost attribution across the organization, all visible in a single dashboard.

Platform teams can set granular policies: individual contributors get five hundred dollars per month on frontier models, senior engineers get two thousand. The ML team gets access to Claude Opus and GPT-4o. The brand design team can use generative image and video models. Interns are restricted to open-source models on Workers AI. These policies map directly to existing IdP groups.

For CI/CD pipelines and autonomous agents, Cloudflare Access service tokens give each agent a named identity. A code review bot that consumes five million tokens in a week is visible and budgetable separately from a documentation generator that uses five hundred thousand. If one agent runs out of control, a budget policy can be applied without affecting any others.

Every AI Gateway log entry includes the authenticated identity: email, IdP group, service token name. Export these to any analytics platform and you have a cost-by-user-by-team breakdown without building anything custom.

Cloudflare uses this internally already. Every Cloudflare employee routes millions of requests and billions of tokens per month through AI Gateway. The company faced the same attribution problem as its customers and solved it with the same tools it is now shipping.

What’s Next: Intelligent Task-Based Routing

Setting a budget is necessary, but optimizing within that budget is where the real value lies. Cloudflare is building intelligent, task-based routing in AI Gateway. For each request, the system will analyze the task and automatically route it to the model that delivers the best result at the lowest cost. A summarization task might run on a smaller, cheaper model without meaningful quality loss, while a large-scale code refactor gets the frontier model. This is in active development and should be watched closely.

Envoy 1.38.1: Security Hardening for the Service Mesh

While Cloudflare is tackling cost governance, Envoy is tightening security. The v1.38.1 release, published on June 4, is a security-focused patch that addresses multiple vulnerabilities and hardens the service mesh against real-world attack vectors.

The headline fix is CVE-2026-47774, an HTTP/2 vulnerability. Uncompressed cookies were not counting toward the configured maximum header list size limits, enabling an HPACK cookie-bomb that could cause excessive memory usage. The patch ensures cookies are included in the limits and resets streams that violate them. This is a classic example of how protocol-level hardening prevents resource exhaustion attacks that could take down an entire mesh.

The release also fixes a timing side-channel in OAuth2 HMAC verification that could leak HMAC secret validity, and a crash in AES-CBC decryption of token cookies that could spuriously succeed on secret mismatch. Both fixes close subtle but serious cryptographic attack surfaces in authentication flows.

Additionally, Envoy applied the nghttp2 CVE-2026-27135 patch and fixed a crash in the dynamic modules HTTP filter when streams were already above the downstream write-buffer high watermark at filter-chain construction time.

For operators, the release introduces two minor but notable behavior changes. The upstream transport failure reason is no longer included in HTTP response bodies sent to downstream clients, though it remains available in access logs. And load balancer rebuild coalescing during EDS batch host updates is now opt-in rather than default, giving operators more control over update timing in large clusters.

Envoy 1.38.1 is a reminder that as the service mesh becomes the backbone for AI traffic routing, its security posture must keep pace. A compromised mesh is not just a data leak; it is a cost leak.

Prometheus 3.12.0: PromQL Evolution and Performance Gains

Prometheus v3.12.0, released on May 28, is a substantial update that delivers security fixes, new PromQL functions, expanded service discovery, and significant TSDB performance improvements.

On the security front, the release patches a denial-of-service vulnerability in remote-write where snappy-compressed requests could declare inflated decoded lengths, and fixes a plaintext secret exposure in STACKIT service discovery via the configuration endpoint. Both fixes underscore the importance of treating metrics infrastructure as a security surface, not just an observability backend.

The most interesting new features are in PromQL. Four experimental functions have been introduced: start(), end(), range(), and step(). These give queries programmatic access to the evaluation window boundaries, enabling more dynamic and self-aware expressions. The resets() function has been updated to consider start timestamp resets, improving accuracy for cumulative metrics that span evaluation boundaries.

The rate(), irate(), and increase() functions now optionally use start timestamps for calculations, behind the use-start-timestamps feature flag. This addresses a long-standing source of inaccuracy in counter calculations when samples are sparse or irregularly spaced. For teams running Prometheus at scale, this is a meaningful correctness improvement.

Service discovery expands with support for DigitalOcean Managed Databases and Outscale VM targets. AWS EC2 discovery now supports IPv6 addresses, and the ECS, MSK, RDS, and ElastiCache integrations gain optional external_id fields for cross-account role assumption. The auto-reload-config feature has been promoted to stable, eliminating a common pain point for configuration management.

On the performance side, TSDB head chunk lookups in range queries have been optimized from quadratic to constant time, and mmap operations now skip clean stripes entirely, significantly reducing CPU utilization at production scales. A FloatHistogram performance regression on Go 1.26 has also been addressed.

Finally, the UI now includes a web interface for deleting time series and cleaning tombstones, accessible from the Status menu. This is a small but welcome quality-of-life improvement for operators managing long-running Prometheus instances.

The Bigger Picture: Cloud Native Meets Financial Engineering

These three updates are not isolated improvements. They are symptoms of a broader shift in how cloud-native infrastructure is being designed and operated.

For the past decade, the cloud-native stack optimized for throughput, resilience, and developer velocity. Kubernetes gave us self-healing workloads. Envoy gave us zero-trust networking. Prometheus gave us unified observability. These were the right problems to solve when the primary concern was keeping services available and developers productive.

But AI workloads introduce a new constraint: cost at scale. A single model inference can cost fractions of a cent, but multiplied across millions of requests, across dozens of services, across hundreds of developers, the numbers become material very quickly. The infrastructure layer must now understand not just how to route traffic, but how much it costs to route it, and who should pay.

Cloudflare’s AI Gateway spend limits are the most visible manifestation of this shift, but they are not the only one. The Envoy security patches are relevant because a compromised mesh could be used to exfiltrate data or burn tokens. The Prometheus updates are relevant because without accurate metrics and cost attribution, there is no feedback loop for optimization.

Platform teams should be thinking about three layers:

Cost visibility: Every request to an AI provider should be observable, attributable, and auditable. Prometheus’s new features and Cloudflare’s logging make this possible.
Cost control: Budgets should be enforceable at the infrastructure layer, not just aspirational finance policies. Cloudflare’s spend limits and identity-driven policies are a model.
Cost optimization: The right model for the right task, routed automatically. This is where Cloudflare’s intelligent task-based routing and Prometheus’s improved rate calculations converge.

The cloud-native stack is becoming financially aware. That is a necessary evolution for the AI era, and it is happening faster than many expected.