Attest

Testing framework that checks what AI agents do, not just what they say

Visit website →

About

Attest is an open-source testing framework for AI agents in Python and TypeScript. It checks an agent reply, a sequence of tool calls, or a change to external state using pytest or Vitest.

It follows a rec, replay, gate pattern: capture the real incident, replay the agent against recorded model responses with zero live model calls, then gate the merge. This removes live-model variability from tests while keeping other dependencies under control.

Attest provides six evaluation layers: schema validation, cost and token constraint checks, trace and behavioral checks, content pattern matching, embedding similarity, and LLM-as-judge. The first four layers are free and fully deterministic.

It ships adapters for OpenAI, Anthropic, Gemini, LangChain, CrewAI, Ollama, and more, with beta support for multi-agent trace delegation and simulation mode with mock tools and seeded faults.

AI-assisted draft, human-reviewed before publishing — see how we choose & review tools.

Why we picked it

Testing AI agents is hard because models are non-deterministic. Attest solves this elegantly with strict replay: record model responses then replay with zero live calls. The four free deterministic layers mean most tests need no API key. A mature framework that fills a real gap in agent testing tooling.