GoModel

GoModel is an open-source AI gateway written in Go and released under the MIT license. It places a single OpenAI- and Anthropic-compatible endpoint in front of 31 AI model providers, so applications keep the request shape they already use while the gateway handles authentication, workflow resolution, provider routing, caching, budgets, guardrails, failover, audit logging, and usage tracking behind that endpoint. It is built for engineering teams, platform teams, and developers who want the provider-switching and governance logic that would otherwise leak into application code to live in one self-hosted layer instead. GoModel ships as one small binary with an embedded admin dashboard, so there is nothing else to deploy. The gateway exists to solve a specific set of problems that appear once AI workloads reach production. Without a gateway layer, provider switching, debugging, and usage tracking start leaking into application code, and teams that integrate a provider directly find that switching vendors becomes a code project rather than a configuration change. A single behavior rarely fits every team or every app: one path may need cache, another audit logging, another guardrails. Identical prompts can be paid for twice when duplicate requests are dispatched. Provider dashboards show one aggregate total, which makes it hard to attribute spend to teams, tenants, and features. When a fallback fires during an incident, nobody can reconstruct why it happened. And running a gateway can become its own scaling project if the software sitting on every request is heavy to operate. GoModel moves that logic into one gateway layer so that provider choice is decoupled from the application. GoModel's routing and provider layer covers a broad range of models behind a single endpoint. OpenAI, Anthropic, Gemini, Bedrock, Vertex, Azure, Groq, Ollama, vLLM, and more are supported, with round-robin rotation across multiple keys per provider, and suffixed environment variables that register extra instances of the same provider type. Aliases and virtual models let teams publish stable names such as smart-chat and remap the real provider and model behind them, which is a config change rather than an application change. Load balancing spreads a virtual model across targets with weighted round-robin, or lets cost-based routing pick the cheapest capable model for each request. Automatic failover sends availability errors to the next model or provider, with retries using backoff and a circuit breaker absorbing flaky upstreams. Provider passthrough lets you call any provider's native API through /p/:provider/* while keeping GoModel's auth, usage tracking, and audit on the way through. Control and safety features are configured through scoped workflows. A workflow toggles cache, audit, usage, budgets, guardrails, and failover per provider, model, or user path, and the most specific matching scope wins; workflow versions are immutable so you can see exactly which policy a given request ran under. Guardrails can inject system prompts or rewrite messages with an LLM before dispatch, running as ordered steps that execute in parallel groups. Virtual API keys hand teams managed keys bound to a user path and labels instead of raw provider credentials, and those keys can be revoked and rotated from the admin UI. Rate limits cap request rate and concurrency per user path, provider, or model, and saturated routes are routed around when alternatives exist, returning 429 with Retry-After when they do not. On the cost side, budgets enforce hard spend limits per user path or label, evaluated from tracked usage cost and enforced before a request is dispatched, so the run stops at the cap rather than at the invoice. Response caching works in two ways: exact-match caching returns identical non-streaming requests straight from the gateway with no provider call and no cost, while semantic caching matches similar prompts and is backed by Qdrant, pgvector, Pinecone, or Weaviate. Usage and cost tracking records token and dollar accounting per request, user path, and label, with per-model pricing overrides for when list prices do not match your contract. Cache lookups run after alias and workflow resolution, so policy decisions still apply, and cache hits are visible in the dashboard. Observability and surface area extend well beyond chat completions. Audit logs capture every request with its resolved route, workflow, cache result, and provider attempts, with bodies and headers logged only when explicitly enabled. The embedded admin dashboard shows live request logs, usage breakdowns, keys, budgets, workflows, and provider status without a separate deployment. Request tagging flows labels from headers or key metadata into usage and audit, so spend and incidents map to teams, tenants, and features. Prometheus /metrics exposes request, provider, and circuit-breaker gauges alongside health endpoints and optional pprof profiling, and OpenTelemetry traces and metrics cover every inbound request and provider call on the GenAI semantic conventions, readable by Jaeger, Tempo, Honeycomb, or Datadog as they are. GoModel also serves the full OpenAI surface including chat, embeddings, the Responses API with gateway-managed conversations, files, and batches, plus the Anthropic Messages API with native /v1/messages and token counting, audio and realtime features such as text-to-speech, transcription, and realtime speech over WebSocket and WebRTC, an MCP gateway that aggregates MCP servers behind one endpoint with namespaced tools, and a built-in playground for sending a real request against any model or alias. Overall, GoModel authenticates each request, applies the matching workflow with its guardrails, cache, budgets, and rate limits, and routes it to the right provider with automatic failover, all behind OpenAI- and Anthropic-compatible APIs. Every response records usage and cost, writes an audit trail, and updates the live dashboard, while cache hits return instantly without a provider call. Because the gateway runs as one Go binary with Docker, Compose, and Helm recipes and the admin UI is embedded, deployment is minimal. Storage starts on SQLite with zero setup and moves to PostgreSQL or MongoDB when traffic and retention demand it, using the same binary with a different config. Session keeping makes requests from one conversation or agent task stick to the target and key that served the first, which keeps provider prompt caches warm, audit logs threaded, and cost attributable per session, with zero configuration required. The benefits follow from that architecture. Teams get one stable API that decouples provider choice from the application, so models can be swapped with a config change. Duplicate prompts stop paying full price twice because caching returns them faster and cheaper. Spend becomes attributable per team, tenant, model, and label instead of appearing as a single provider total. Incidents become reconstructable: a fallback that fired can be traced through audit logs and runtime metadata showing the resolved route and provider attempts. Compliance reviews can replay any request, including guardrail versions and full bodies where logging is explicitly enabled. And the gateway itself stays lightweight: a single binary with storage that grows with the workload, so the gateway does not out-scale the application it serves. Concrete workflows show how it is used. A multi-tenant SaaS issues a virtual key per customer, tracks usage by user path, and enforces per-tenant budgets so invoices come from the dashboard rather than guesswork. A platform team publishes aliases like smart-chat with scoped workflows behind them, letting product teams ship features without ever holding provider keys. Production traffic rides failover chains with retries and circuit breakers, turning a provider incident into a routing event instead of a customer-facing one. Caching absorbs duplicate prompts, cost-based routing picks the cheapest capable model, and budgets stop end-of-month surprises without code changes. Compliance reviews replay any request with its resolved route, guardrail versions, provider attempts, and full bodies where logging is explicitly enabled. Developers run Ollama or vLLM locally behind the same endpoint the cloud providers serve in production, so moving from laptop to production is config, not code. GoModel supports 31 providers, including OpenAI, Anthropic, Google Gemini and Vertex AI, Azure OpenAI, Amazon Bedrock, OpenRouter, Cohere, Groq, xAI, DeepSeek, Fireworks AI, Alibaba Bailian, MiniMax, Z.ai, ElevenLabs, Ollama, vLLM, and any OpenAI-compatible backend registered as its own instance. Hundreds of models are read from live provider catalogs, and connection is typically a single environment variable. Deployment options include a one-command binary install for macOS, Linux, and Windows, a compressed 14.4 MB Docker image, Docker Compose, and Kubernetes with a Helm chart. The gateway is MIT licensed, and GoModel Pro is a commercial distribution that adds prompt compression, OIDC single sign-on, per-child quota templates, and intelligent routing for $4,999 per year or $499 per month, flat per company, with a 30-day money-back guarantee and an offline signed license token. A roadmap toward v0.2.0 tracks remaining work such as plugins starting with guardrails, guardrails hardening with custom and response-side guardrails, passthrough for every provider, and failover and streaming for image endpoints. In short, GoModel is an open-source AI gateway that collapses provider routing, governance, cost control, and observability into a single small binary with one OpenAI- and Anthropic-compatible endpoint, giving teams a self-hosted alternative to OpenRouter and LiteLLM without running a Python service on the hot path.