Evaluation & testing
Apache-licensed LLM vulnerability scanner maintained by NVIDIA. It fires static, dynamic and adaptive probes at a model or dialog system to test for jailbreaks, prompt injection, toxicity, data leakage and misinformation, logs each generation and detector verdict, and outputs a report with failure rates and hit logs.
open_source · generally available · Research snapshot 2026-09-06
Visit the official product source ↗Evaluation & testing · Agent security & threat detection
Useful conversation with: AI security engineer, Red team lead.
Show me a garak scan of my chatbot with the probe-by-probe failure rates and the hit log for successful jailbreaks.
Support labels reflect the supplied research. Documentation and vendor claims are not independent product tests. “Not established” means the researcher did not find support; it does not prove a capability is absent.
Documented by provider
garak combines static, dynamic and adaptive probes to test for hallucination, data leakage, prompt injection, misinformation, toxicity and jailbreaks, evaluates each probe with recommended detectors, and records generations both as received and when evaluated, with a separate hit log for successful attacks.
Limit: The README does not document tool-call or agent-trajectory testing, nor any coverage guarantee.
Documented by provider
Supported targets include Hugging Face Hub models, OpenAI chat and completion APIs, AWS Bedrock foundation models, Replicate, LiteLLM, GGUF models via llama.cpp and arbitrary REST endpoints returning plaintext or JSON.
Limit: Support for a target does not establish probe effectiveness against it.
Documented by provider
Documentation states garak is Apache 2.0 licensed and produces a full report detailing what worked and what needs improvement when scanning a chatbot or model.
Limit: The documentation site describes itself as a work in progress and states no governance or multi-user features.
Listing does not imply partnership, supplier status, a working DutyGraph integration, or a compliance certification.
Suggest a correction