TrueFoundry AI Gateway and AI Deployment Platform

TrueFoundry, operated by Ensemble Labs Inc in San Francisco, builds enterprise infrastructure for deploying and governing generative AI applications and agents. It targets the fragmentation companies face when many teams call different LLM providers, MCP tools and self-hosted models without central control over cost, access and data.

The platform combines an AI Gateway that routes requests across LLM providers with fallbacks, rate limits and budgets; an MCP Gateway and Agent Gateway that govern tool access and agent identity; prompt management; and an AI deployment platform for training, fine-tuning and serving models on Kubernetes. It runs in public cloud, customer VPCs, on-premises or air-gapped environments, with RBAC, SSO, audit logs and OpenTelemetry tracing. Customers named on the site include NVIDIA, ResMed, Siemens Healthineers, Automation Anywhere, Cargill and NetApp.

• AI Gateway: unified API with routing by cost, latency and availability, fallbacks and budgets.
• MCP Gateway: discovery, authentication and access control for tools and MCP servers.
• Agent Gateway: governs agent identity, usage policies and tracing.
• Guardrails and governance: RBAC, SSO, quotas, rate limits and immutable audit logs.
• AI deployment platform: training, fine-tuning and model serving on Kubernetes clusters.
• Observability: OpenTelemetry tracing exported to Grafana, Datadog and Prometheus.

A Vietnamese bank can route all internal GenAI use through one gateway with per-department budgets and PII controls. Software outsourcing firms can give developers governed access to multiple coding models. Retailers and telecoms can host fine-tuned models in their own VPC while still using external LLMs as fallback.

One control plane for all LLM, MCP and agent traffic gives security and finance teams visibility into usage and cost. The gateway adds about 3-4 ms latency and handles 350+ requests per second on one vCPU, per the vendor. Customers report up to 80% higher GPU cluster utilization and 35% lower cloud cost than a previous SageMaker setup. Self-hosted deployment keeps data inside the customer's own environment.

Enterprises are moving from single chatbot pilots to many AI agents using several model providers and internal tools, raising concerns about cost, data leakage and compliance. AI gateways and agent governance layers are becoming standard infrastructure, especially for regulated sectors in Asia that require data residency.

Headquartered at Ensemble Labs Inc, San Francisco, operating across North America, Europe and Asia-Pacific. Raised a 19 million USD Series A in February 2025 led by Intel Capital with Peak XV, Eniac Ventures and Jump Capital. Acquired UK-based Seldon AI, maker of Seldon Core, in June 2026. Lists SOC 2, HIPAA and GDPR compliance.