vLLM
High-throughput open-source LLM inference with PagedAttention
vLLM Project (UC Berkeley / community) · Infrastructure
Launched Jun 2023
Quick Answer
vLLM is high-throughput open-source llm inference with pagedattention, made by vLLM Project (UC Berkeley / community). It is free and open source. Key uses: Self-hosted LLM serving, High-throughput inference, Production LLM deployment.
Last updated:
Users
30K+ GitHub stars; deployed at major AI labs and enterprises
Valuation
$1B (seed, a16z, 2025)
What is vLLM?
vLLM is an open-source, high-throughput and memory-efficient inference engine for large language models, built by UC Berkeley. Its PagedAttention algorithm manages GPU memory like an OS manages RAM, enabling 2-4× more throughput than standard HuggingFace inference. Provides an OpenAI-compatible server for drop-in deployment.
What's new in vLLM in 2026?
vLLM 0.6 (2024) adds speculative decoding, lora serving, prefix caching; 30K+ GitHub stars; vLLM Production Stack for Kubernetes; used by major AI companies for model serving; vLLM team raised $100M seed at $1B valuation (Andreessen Horowitz, 2025)
How much does vLLM cost?
Open-Source
$0
- Apache 2.0 license
- All features
- Community support
- Commercial use
What can you do with vLLM?
- Self-hosted LLM serving
- High-throughput inference
- Production LLM deployment
- Multi-GPU serving
- Cost-efficient inference
Pros
- 2-4× throughput vs HuggingFace
- OpenAI-compatible API
- Continuous batching
- Multi-GPU support
- All major open models
Cons
- Requires ML expertise
- GPU hardware needed
- No GUI
- Setup complexity
What are the best alternatives to vLLM?
Compare vLLM Head-to-Head
Frequently Asked Questions about vLLM
How much does vLLM cost?
vLLM pricing: Open-Source at $0.
Is vLLM free?
Yes, vLLM is fully free and open source.
What is vLLM used for?
vLLM is used for: Self-hosted LLM serving, High-throughput inference, Production LLM deployment, Multi-GPU serving, Cost-efficient inference.
What are the best alternatives to vLLM?
Top alternatives to vLLM include Replicate, Groq API, Ollama.
What platforms does vLLM support?
vLLM is available on: Linux (CUDA/ROCm), AWS, GCP, Azure, On-premise.
Details
Vendor
vLLM Project (UC Berkeley / community)
Category
Infrastructure
Pricing Model
Launched
Jun 2023
Website
vllm.aiUnderlying Models
Platforms
Integrations
Ideal For
- MLOps engineers
- Platform teams
- High-volume inference
- GPU cluster operators