# vLLM v0.18.0 Ships gRPC Serving, GPU-Less Rendering, and Major KV Cache Improvements

- Link: https://thestackobserver.com/vllm-v0-18-0-ships-grpc-serving-gpu-less-rendering-and-major-kv-cache-improvements/
- Published: 2026-03-23T13:36:34-04:00
- Author: Stackxx

The vLLM project has released **version 0.18.0**, a substantial update featuring
445 commits from 213 contributors including 61 new contributors. This release significantly
expands deployment flexibility for production LLM serving with new protocol support,
architectural improvements for multimodal workloads, and substantial enhancements
to memory management that directly address operational pain points for large-scale
inference deployments.

## Headline Features

The v0.18.0 release introduces several major capabilities requested by production
operators:

**gRPC Serving Support:** The biggest infrastructure addition is native gRPC serving
via the new –grpc flag. This enables high-performance RPC alongside the existing
HTTP/REST interface, addressing latency and throughput requirements for internal
service-to-service communication. gRPCs binary protocol and streaming capabilities
make it well-suited for high-volume inference scenarios.

**GPU-Less Render Serving:** The new vllm launch render command enables GPU-less
preprocessing and rendering, allowing separation of multimodal preprocessing from
GPU inference. For vision-language models and other multimodal workloads, this means
CPU resources can handle image decoding, resizing, and tokenization while GPU memory
remains dedicated to model inference. This architectural separation can improve 
overall throughput and resource utilization.

**NGram GPU Speculative Decoding:** NGram speculative decoding now runs on GPU and
is compatible with the async scheduler, significantly reducing speculative decoding
overhead. This optimization helps reduce latency for token generation by predicting
and validating multiple tokens simultaneously.

**Ray Dependency Change:** Ray has been removed as a default dependency. Users requiring
Ray for distributed serving will need to install it explicitly. This reduces the
default installation footprint for users who dont need distributed capabilities.

## KV Cache Offloading Gets Smarter

The KV cache management received substantial attention in this release with three
significant improvements:

**Smart CPU Offloading:** The new smart offloading feature stores only frequently-
reused KV blocks rather than the entire cache. Traditional offloading moves all 
KV cache blocks to CPU memory when GPU memory is constrained, but not all blocks
have equal reuse probability. The smart approach analyzes access patterns and prioritizes
high-value blocks, potentially maintaining more of the working set at GPU speed 
while still accommodating longer contexts.

**FlexKV Backend:** FlexKV joins the list of supported offloading backends, providing
additional flexibility for different hardware and workload configurations. The modular
backend approach allows vLLM to adapt to various system architectures.

**Multiple KV Groups:** Support for multiple KV groups in offloading configurations
enables more granular memory management. This is particularly relevant for larger
models where different attention heads or layers might have different caching requirements.

These changes directly address one of the primary challenges in long-context inference:
efficiently managing the KV cache memory footprint without sacrificing performance
for context-heavy workloads. For production deployments running models with 128K
+ context windows, these optimizations can translate to serving more concurrent 
requests with the same hardware.

## Elastic Expert Parallelism Advances

For Mixture-of-Experts (MoE) models, v0.18.0 completes Milestone 2 of the Elastic
Expert Parallelism (EP) initiative. The NIXL-EP integration enables dynamic GPU 
scaling for MoE experts—meaning the system can adjust expert parallelism based on
actual load rather than being fixed at deployment time.

A new –enable-ep-weight-filter CLI option accelerates EP model loading by filtering
expert weights during the loading process. For MoE-based workloads like Mistral,
DeepSeek, and other popular architectures, these improvements translate to more 
efficient resource utilization and faster startup times.

The elasticity aspect is particularly interesting for cloud deployments where GPU
availability fluctuates. Rather than failing when the expected expert count cant
be satisfied, the system can adapt to available resources while maintaining correctness.

## Expanded Model Support

The release adds support for several new architectures, expanding vLLMs coverage
of the open-source model landscape:

 * **Sarvam MoE**—Indian-language-focused MoE architecture
 * **OLMo Hybrid**—Allen Institutes open language model
 * **HyperCLOVAX-SEED-Think**—32B VLM and 14B language models from Naver
 * **Kimi-Audio-7B-Instruct**—Moonshots audio-capable model
 * **ColPali**—Late-interaction retrieval models
 * **ERNIE Pooling Models**—Baidus embedding architectures

Eagle3 speculative decoding is now available for Qwen3.5 models as well, providing
additional latency reductions for that popular architecture.

## API and Tooling Improvements

The OpenAI-compatible Responses API now supports streaming tool calls (function 
calling), enabling real-time tool use in streaming contexts. This is essential for
agentic workflows where tools are invoked mid-generation.

For audio workloads, vLLM adds beam search support for encoder-decoder models in
both offline and online transcription scenarios. This improves accuracy for speech
recognition use cases by considering multiple hypotheses during decoding.

FlashInfer has been updated to version 0.6.6, bringing performance and correctness
improvements to the attention backend.

## Known Issues and Deployment Notes

Operators should note degraded accuracy when serving Qwen3.5 with FP8 KV cache on
NVIDIA B200 GPUs. This is a known issue being tracked by the project.

Users who previously encountered CUBLAS_STATUS_INVALID_VALUE errors in v0.17.0 should
reinstall PyTorch 2.10.0. An updated wheel addressing this issue has been published,
so the previous workarounds should no longer be necessary.

## Sources

 * GitHub: vLLM v0.18.0 Release Notes (March 20, 2026)
 * VLLM Project Documentation