Datumo Eval LLM Evaluation and AI Red Team

SelectStar, which uses the global brand Datumo, is a Seoul-based AI data and evaluation company established in November 2018. It started with AI training data and now focuses on making LLM services trustworthy, building Korea's first LLM trustworthiness benchmark dataset and co-hosting generative AI red team challenges with the Korean government and GSMA.

Datumo Eval lets teams upload source documents, automatically generate industry-specific evaluation questions, and score model answers with default or custom metrics. Multiple LLM agents collaborate on question generation and judging, and a Human Alignment step narrows the gap between evaluation intent and automated scores. The AI Red Team offering probes models for harmful or unsafe outputs, and the company also supplies custom and premium datasets, including LLM harmlessness evaluation data. Logos on the site include Samsung SDS, NAVER, LG CNS, SKT, KT, Hana and Shinhan.

• Question generation: creates industry-specific evaluation questions from uploaded source documents.
• Answer evaluation: scores model responses with default or custom metrics.
• Multi-agent judging: several LLM agents collaborate on generation and evaluation.
• Human Alignment: calibrates automated scores against human evaluation intent.
• AI Red Team: adversarial testing for harmful and unsafe model behaviour.
• Datasets: custom datasets, premium dataset store and LLM harmlessness data.

Vietnamese banks can generate test sets from product documents to check whether customer chatbots answer accurately. Korean-invested electronics and telecom firms in Vietnam can red-team internal assistants before rollout. Legal and education service providers can define custom metrics for answer quality.

Teams build evaluation sets from their own documents in hours rather than writing test questions by hand. Custom metrics let each industry, such as finance or legal, define what a good answer means. Human Alignment keeps automated judging consistent with expert opinion. Red teaming and harmlessness datasets help catch unsafe outputs before customers see them.

As companies in Asia build domain-specific chatbots on top of local and global LLMs, evaluation is shifting from generic English benchmarks to criteria tied to each business and language. Korean and regional regulators are also promoting red teaming as part of AI safety, a trend Vietnamese enterprises are likely to meet as they deploy GenAI.

SelectStar Inc. is based in Gangnam, Seoul, and DATUMO Inc. was established in November 2018. Funding includes a seed round in 2019 and Series A extensions of 6.1 million USD in 2022 and 2.7 million USD in 2023. Recognition includes Forbes 30 Under 30 Asia 2021, Forbes Korea AI 50 in 2025 and KPAS 2024. In 2025 it co-hosted a GenAI Red Team Challenge with GSMA at MWC and joined Korea's proprietary AI foundation model consortium.