LiteLLM AI Gateway

Berri AI Incorporated develops LiteLLM, an open-source AI gateway (LLM proxy) that exposes many model providers through a single OpenAI-compatible API. The About page says it was founded in 2023 in the Y Combinator Winter 2023 batch and is based in San Francisco, California.

The gateway sits between applications and model providers and adds spend tracking per key, user and team, budgets, rate limits, SSO, guardrails, routing and caching. It can be self-hosted with Docker, Helm or Terraform, including in air-gapped environments, and is also offered as a Python SDK and a Rust gateway. Testimonials on the site come from NVIDIA, Netflix, Okta and AT&T.

• Unified API: OpenAI-compatible access to 140+ providers and 1,800+ models.
• Spend control: spend tracking per key, user, team and tool with budgets and rate limits.
• Routing and caching: load balancing, lowest-cost routing, fallbacks and semantic caching.
• Security: SSO, JWT, RBAC, audit logs, PII and prompt-injection guardrails.
• Flexible deployment: Docker, Helm and Terraform, self-hosted or air-gapped.
• Observability hooks: integrations with Langfuse, Arize Phoenix, OpenTelemetry and Datadog.

A Vietnamese bank platform team can run LiteLLM in its own cluster to give internal teams budgeted access to several LLMs. A telecom or e-commerce group can route chatbot traffic to the cheapest adequate model with fallbacks. A system integrator can offer a managed gateway with per-client chargeback.

One API for 140+ providers and 1,800+ models reduces vendor lock-in. Per-key and per-team spend tracking with hard budgets gives finance and platform teams cost control. Self-hosting with no telemetry keeps prompts and data inside the buyer network. A free open-source tier allows low-cost evaluation before an enterprise contract.

Enterprises adopting several LLM providers need a central layer for access, cost allocation and governance. AI gateways are becoming standard infrastructure as agent and MCP traffic grows. In Vietnam and Southeast Asia, banks and large groups piloting generative AI often require self-hosted control over data and spend.

Legal name Berrie AI Incorporated (site footer spelling varies), San Francisco, founded 2023, YC W2023, $1.6M seed per About page. Enterprise tier adds SSO, JWT auth, audit logs and SLAs. Security incident: a malicious v1.107.0 was pushed to PyPI and Docker on 2026-10-21 and is marked contained; enterprise release not affected.