# Arize AI's SallyAnn DeLucia Introduces Decision Judges Built on OpenAI's Decision Model API

> Arize AI introduced Decision Judges, a new eval type tied to OpenAI's decision model API, as its head of product argued evals are the primary signal for agents.

- Published: 2026-10-25T23:56:05+00:00
- Section: Product Launches
- Author: News Desk
- URL: https://newsforagent.com/news/2026-10-25/arize-ai-decision-judges-openai-api
- Source: [StartupHub.ai](https://www.startuphub.ai/ai-news/technology/2026/arize-s-eval-demo-skips-the-hard-tradeoffs)

## Key takeaways

- SallyAnn DeLucia, Head of Product at Arize, introduced Decision Judges, a new eval type tied to OpenAI's decision model API released about two weeks prior.
- DeLucia said Decision Models return a probability and a true or false with no explanation, and her team started Alex with a handful of evals and now runs 32.
- Arize raised $131 million in total equity funding as of March 2026 and was ranked third in valuation among 100 AI governance startups in 2026.

Arize AI introduced Decision Judges, a new eval type tied to [OpenAI](https://akta.pro/company-profiles/0000008/openai)'s decision model API released about two weeks prior, during a walkthrough of writing a first AI eval from scratch. [StartupHub.ai](https://www.startuphub.ai/ai-news/technology/2026/arize-s-eval-demo-skips-the-hard-tradeoffs) covered the session, in which SallyAnn DeLucia, Head of Product at Arize, framed evals as the primary signal for agents.

## DeLucia's Decision Judges and the OpenAI API

DeLucia, four years at [Arize AI](https://akta.pro/company-profiles/00002ik/arize-ai) and builder of its agent Alex and its evals product, said vibe checks do not scale. Without evals, she said, teams cannot catch regressions, compare prompts or models objectively, or put quality gates in CI. She drew a line between observability and testing: "Traces are the logs for AI, evals are the actual tests."

She listed LLM as a Judge, which returns a label and explanation and fits QA or relevance checks; Decision Models, which are cheap and fast and return a probability and a true or false with no explanation; code-based checks for deterministic rules like JSON format or Rouge and Blue scores; annotations on golden or synthetic datasets; Agent as a Judge for complex criteria; plus user feedback and business metrics in production. She said to use the first four across the lifecycle, and to judge one quality per eval for LLM as a Judge because research shows single-criterion judges perform better, while Decision Models and Agent as a Judge can carry multiple.

OpenAI, whose decision model API the new eval type is tied to, was named an OpenAI Select Partner within the OpenAI Partner Network on October 25, 2026, according to EIN Presswire.

## Arize AI's Funding and Scale Records

Arize AI raised $131 million in total equity funding as of March 2026, the most-funded startup in the AI governance sector ranking, according to akta.pro [company records](https://akta.pro/company-data). The same records rank it third in valuation among 100 AI governance startups in 2026, behind Braintrust at $800 million and Protect AI at $650 million to $700 million, and note that no AI governance startup had reached unicorn status. Its $70 million Series C in February 2025 was the largest-ever investment in the AI observability space at the time of the announcement, led by Adams Street Partners.

## Arize AI's Path to the Eval Demo

Dynatrace completed its acquisition of Arize, an AI observability and evaluation platform, in October 2026, with Arize continuing to support its open-source and enterprise platforms while integration proceeds, Stock Titan reported. In September 2026, LG Uplus signed a strategic memorandum with Arize AI to monitor AI agents, focusing on performance, response quality, and safety, according to Koreatimes. Arize AI announced a partnership with Google Cloud in May 2026 to promote standardized telemetry for enterprise AI agents using OpenTelemetry and OpenInference standards, The New Stack reported.

## What DeLucia Said Comes Next

DeLucia said her team started Alex with a handful of evals and now runs 32, and that Arize's Alex can draft and iterate the prompt itself. She said a specific eval beats a generic one, but any eval beats none, and Eval Hub templates are fine to start.

## Companies in this story

- [Arize AI company profile](https://akta.pro/company-profiles/00002ik/arize-ai) · website: https://arize.com
- [OpenAI company profile](https://akta.pro/company-profiles/0000008/openai) · website: https://openai.com

---
News for Agent · https://newsforagent.com/llms.txt