LiteLLM
One OpenAI-compatible gateway for every LLM provider - virtual keys, budgets, spend tracking - with no percentage fee.
One-click deploy, from $25/mo on a Miget plan.
Every team using LLMs converges on the same needs: one endpoint instead of five SDKs, per-team API keys instead of the shared provider key in a Slack DM, budgets before the invoice surprise, and a log of who spent what on which model. LiteLLM is the open-source gateway that does exactly this - an OpenAI-compatible proxy in front of 100+ providers.
This template runs the database-backed mode on a managed Postgres: virtual keys, teams, budgets, and spend logs all persist, and models are managed in the admin UI (/ui) with no config file - STORE_MODEL_IN_DB keeps provider credentials encrypted in the database.
The hosted alternatives take a cut or meter your logs; OpenRouter adds 5.5% on credits, Portkey meters logged requests. A self-hosted gateway charges neither - your provider keys, your spend caps, LAN latency from the agents in your project.
Upstream project: LiteLLM
#what you get
- OpenAI-compatible /v1 endpoint for 100+ providers (Anthropic, OpenAI, Gemini, Bedrock, Mistral, …)
- Virtual keys with per-key/team budgets, rate limits, and model allowlists
- Spend tracking per key, team, and model in Postgres
- Fallbacks, retries, and load balancing across deployments
- Admin UI at /ui - add models and mint keys without redeploys
- No config.yaml: models and encrypted provider keys live in the DB
#topology
| Service | Role | Public |
|---|---|---|
| litellm | proxy + admin UI (:5000, key-authenticated) | yes |
| db | Postgres - managed service on Miget, container locally | no |
#miget sizing
// this stack needs
3 GiB RAM · 5 GB disk · 2 services
2 GiB suits a single proxy instance for team workloads (official prod guidance is 4 GiB per worker - size up under heavy concurrency). The container is stateless; Postgres holds everything.
Hobby - recommended fit
$25/mo
2 vCPU · 4 GiB · 80 GiB disk
Headroom for your own apps: 8 GiB at $49/mo
Professional - production
$43/mo
2 vCPU · 4 GiB · 25 GiB disk
Dedicated resources, production SLOs - plan details
One Miget plan is a fixed pool of compute - the whole stack (managed databases included) deploys inside it, and anything left over runs your other apps. No per-service or per-seat math.
#vs. the managed service
What the hosted equivalents charge, against the flat Miget plan this stack fits on. Prices as of June 2026, sources linked.
| Service | Plan | Monthly | What you get |
|---|---|---|---|
| LiteLLM on Miget ★ | 4 GiB plan | $25 | this whole stack, flat - no usage meters, and room left for your own apps |
| OpenRouter | pay-as-you-go | usage-based | 5.5% fee on credit purchases on top of provider model prices |
| Portkey | Production | ~$49 | 100k logs/mo, then $9 per extra 100k; 30-day retention |
A self-hosted gateway adds no percentage fee and no log metering - your provider keys, your spend caps, LAN latency to your apps.
#vs. other PaaS
Estimated monthly cost of running this exact stack (3 GiB RAM, 5 GB disk, 2 containers) elsewhere, from published June 2026 rates.
| Platform | Est. monthly | Notes |
|---|---|---|
| Miget ★ | $25 flat | compose stacks first-class: one deploy, dedicated vCPU, managed Postgres/Valkey, volumes and TLS all included in the plan |
| Heroku | ~$150 | no volumes; nothing between 1 GB ($50) and 2.5 GB ($250) dynos - 2 GB containers cost far more than shown |
| DO App Platform | ~$41 | no persistent volumes - stateful containers need managed DBs/Spaces (base $5 Spaces included here) |
| Render | ~$39 | per-service instances (0.5 GB $7, 2 GB $25) - every container is its own paid service |
| Railway | ~$31 | usage-based ($10/GB RAM-mo); vCPU billed separately at $20/vCPU-mo on top |
| Fly.io | ~$18 | cheapest sticker price - but burstable shared CPUs (1/16 core; dedicated vCPUs cost ~2-3×), no compose deploys (one app per container, manual wiring), managed DBs billed extra |
Estimates assume RAM fully allocated at published on-demand rates - and sticker price isn't the whole comparison: the cheaper rows buy burstable shared CPUs, per-service wiring instead of a compose deploy, and managed databases billed separately. Heroku and DO App Platform have no persistent volumes at all - stateful stacks like this one need workarounds there.
#deploy it
On Miget
- Create a Compose Stack in app.miget.com pointing at the templates repository
- Set the stack path to
litellm -
Set the required variables:
LITELLM_MASTER_KEY, admin key - must start with sk-LITELLM_SALT_KEY, encrypts stored provider keys (openssl rand -hex 32); set before adding the first model, cannot rotate after
- Deploy. Miget layers
compose.miget.yaml(RAM, privacy, volumes, managed services) automatically
Locally first?
Every template is portable, vanilla Docker Compose - the Miget overrides are ignored locally:
git clone https://github.com/deployable-sh/stacks
cd miget-compose-templates/litellm
docker compose up -dSame files, same behavior. The template README covers connection strings and scaling notes.
#faq
How does this compare to OpenRouter?
OpenRouter is excellent for instant multi-model access but adds a 5.5% fee on credits and routes your traffic through their infrastructure. Self-hosted LiteLLM uses your own provider keys at raw provider prices, keeps prompts inside your network, and adds the team controls (budgets, virtual keys) OpenRouter does not.
Do my existing OpenAI-SDK apps work with it?
Yes - that is the design. Point base_url at your gateway (http://litellm:5000/v1 in-project, or the public domain) with a virtual key, and the same SDK call can hit Claude, GPT, or Gemini depending on the model name and your routing config.
Where do provider API keys live?
Encrypted in Postgres using LITELLM_SALT_KEY, managed through the admin UI. Generate the salt key before adding the first model and never rotate it - it cannot be changed once credentials are stored.
Is it safe to expose the proxy publicly?
Every request requires a key (master or virtual), so the public domain is usable by external clients. For purely internal use, in-project apps can call http://litellm:5000 and you can keep usage gated by virtual keys per team or agent.
Does it pair with the other LLM templates here?
Naturally: point agent-box, OpenClaw, or Hermes at the gateway for spend caps per agent, and send LiteLLM callbacks to the langfuse template for tracing - the whole LLM platform, self-hosted in one project.
Ship LiteLLM today
One compose stack, 3 GiB of RAM, from $25/month flat, and it runs on your laptop with the same files.