What an assistant evaluation receipt should contain

Record the conditions

A result needs a task definition, an initial state, the user's constraints, and the permissions the assistant received. Record the product and model versions where available, subscription, tools, memory state, region, and date. If a provider does not expose a version, preserve that uncertainty.

A transcript alone leaves too much implicit. For an email task, the receipt should distinguish a saved draft from a sent message and identify the destination using a privacy-preserving reference.

Verify the outcome independently

Keep the assistant's completion claim separate from the observed final state. Record assertions about both intended effects and prohibited effects. A successful booking with an unauthorized payment is not an unqualified success.

Retain failed and interrupted trials. A denominator that excludes difficult cases produces a misleading success rate. Reset the environment between independent repeats, and document what could not be reset.

Make corrections inspectable

Version the task and grader, record adjudication, and preserve superseded results. A correction should explain which claim changed and why. Redact secrets and personal information before publication; reproducibility does not require exposing someone's inbox.

For a builder, begin with a small set of consequential failure modes and verifiers that a second evaluator can inspect. Expand coverage after checking whether those verifiers accept valid alternative solutions and reject deliberately incorrect outcomes.

Further reading

• τ-bench: tool-agent-user interaction ↗
• Anthropic: demystifying evals for AI agents ↗

This is methodological commentary, not a report of completed Personaeval product trials. Corrections can be made through the repository owner.