Evaluation & testing
Vendor offering managed evaluators plus simulation environments for agent testing: hosted judges score hallucination and unsafe output, red-teaming algorithms probe for weaknesses, and simulated digital workflows exercise long-horizon agent tasks. Public pages document scoring and simulation but not audit records, retention or access control.
commercial · generally available · Research snapshot 2026-09-06
Visit the official product source ↗Evaluation & testing · Observability & traceability
Useful conversation with: AI engineering lead, Model risk analyst.
Demonstrate a simulated multi-step workflow run where my agent is scored for hallucination and unsafe output, and show what evaluation evidence I can export.
Support labels reflect the supplied research. Documentation and vendor claims are not independent product tests. “Not established” means the researcher did not find support; it does not prove a capability is absent.
Documented by provider
Documentation describes evaluating and monitoring LLM and agent interactions in production through tracing, logging and alerts, with in-house evaluators such as Lynx and Glider for hallucination and unsafe output, LLM-as-judge with custom criteria, human-in-the-loop annotations and dataset generation.
Limit: Documentation overview does not state whether tool calls or multi-agent handoffs are captured as distinct spans.
Documented by provider
Patronus documents red-teaming algorithms that automatically expose weaknesses in AI systems, alongside turnkey metrics covering RAG, agents, NLP and OWASP categories.
Limit: No documented attack taxonomy, coverage list or independent validation of the red-teaming results.
Vendor claim
The company's product page states that digital world models predict and simulate agent actions in digital workflows, covering multi-turn dialogue, long-horizon tasks spanning days to months, memory and UI/UX navigation.
Limit: Simulation figures such as feature parity and model lift are vendor-stated on a marketing page with no methodology or third-party verification.
Listing does not imply partnership, supplier status, a working DutyGraph integration, or a compliance certification.
Suggest a correction