llama.cpp
C/C++ inference for GGUF models on everyday hardware
llama.cpp runs quantized language models on CPUs and many GPU backends. It defines the GGUF format and ships a server with an OpenAI-compatible API, and many local tools build on it.
More in Inference platforms
Featured
Ollama
Inference platforms
Run open models locally with one command
Amazon Bedrock
Inference platforms
Managed access to many foundation models on AWS
Azure AI Foundry
Inference platforms
Microsoft's platform for building with models and agents
Baseten
Inference platforms
Production inference for open and custom models
Cerebras
Inference platforms
Inference on wafer-scale chips
Cloudflare Workers AI
Inference platforms
Run models on Cloudflare's network