Evaluation & testing
Apache-2.0 LLM evaluation toolkit from Singapore's AI Verify Foundation that combines benchmark testing across safety and performance metrics with manual and automated red-teaming, offers guided workflows for IMDA's starter kit for LLM app testing, and produces shareable scoring reports usable in CI pipelines.
open_source · generally available · Research snapshot 2026-09-06
Visit the official product source ↗Evaluation & testing · AI risk & compliance management
Useful conversation with: AI evaluation lead, Compliance manager, LLM application developer.
Show me running IMDA's starter-kit benchmarks plus an automated red-team attack module against our chatbot and the scoring report it produces.
Support labels reflect the supplied research. Documentation and vendor claims are not independent product tests. “Not established” means the researcher did not find support; it does not prove a capability is absent.
Documented by provider
The repository states Moonshot brings benchmarking and red-teaming together for LLM applications, tests bias, toxicity and hallucination alongside metrics such as accuracy and BLEU, supports guided workflows for IMDA's Starter Kit for LLM-based App Testing, and provides attack modules and custom recipes.
Limit: Benchmarks and attack modules are as good as their datasets; results are not certifications and dataset currency is not guaranteed.
Documented by provider
AI Verify Foundation states Project Moonshot is an open-source LLM evaluation toolkit with a Python library and web UI, pre-built evaluators and benchmark datasets, CI/CD pipeline integration and shareable reports.
Limit: Foundation page is promotional in tone and does not quantify coverage or validity of the evaluators.
Documented by provider
GitHub metadata records Apache-2.0 licensing, a non-archived repository and last push June 2026.
Limit: Activity gap of several months; maintenance cadence unclear.
Listing does not imply partnership, supplier status, a working DutyGraph integration, or a compliance certification.
Suggest a correction