vLLM
High-throughput open-source LLM serving engine
vLLM is an open-source engine for serving language models. It uses paged attention and continuous batching to raise throughput, and exposes an OpenAI-compatible server.
More in Inference platforms
Featured
Ollama
Inference platforms
Run open models locally with one command
Amazon Bedrock
Inference platforms
Managed access to many foundation models on AWS
Azure AI Foundry
Inference platforms
Microsoft's platform for building with models and agents
Baseten
Inference platforms
Production inference for open and custom models
Cerebras
Inference platforms
Inference on wafer-scale chips
Cloudflare Workers AI
Inference platforms
Run models on Cloudflare's network