Opik AI Observability and Evaluation

Comet ML, Inc. publishes Opik, an open-source observability and evaluation platform for agentic AI systems, with a New York address listed in its privacy policy. It addresses the problem of silent errors and untracked cost in LLM applications. Comet announced the launch of Opik in December 2024 and joined Amazon SageMaker Partner AI Apps the same month.

Opik logs each step an agent takes as a trace, scores outputs with 30+ LLM-as-a-judge metrics, runs production monitoring with PII and policy guardrails, and tracks token cost. A June 2026 release added Cost Intelligence for engineering teams using Claude Code and Codex. The page names AssemblyAI, NatWest, Stellantis, Uber, Netflix and Etsy.

• Tracing and debugging: visualizes agent steps, annotations and audit logs.
• Evaluation: 30+ LLM-as-a-judge metrics and plain-text assertions through Test Suites.
• Production monitoring: real-time evaluation, alerts and PII or policy guardrails.
• Cost Intelligence: tracks Claude Code and Codex usage and spend by team.
• Agent Playground: runs end-to-end agent tests with versioned prompts.
• Prompt Optimizer: six optimization algorithms plus the Ollie AI debugging assistant.

A Vietnamese software outsourcer can track Claude Code spend across teams. A bank can run hallucination and relevance metrics on a RAG assistant before launch. A startup can self-host Opik to monitor an AI chatbot for free.

The core is open source and free, with a hosted free account and enterprise plans on request. Every agent step is traceable for debugging. Guardrails can flag PII and policy violations in production. Cost Intelligence shows spend on coding agents across engineering teams.

Open-source LLM observability is a crowded field, and buyers increasingly ask for agent-level tracing, automated evaluation and cost control for coding assistants. Vietnamese enterprises adopting generative AI often prefer open-source tools they can self-host.

Legal entity Comet ML, Inc. (New York address in privacy policy). Opik launched December 2024; Comet joined Amazon SageMaker Partner AI Apps in December 2024; Cost Intelligence press release June 2026. HIPAA, AICPA and ISO badges shown in the site footer.