PersonaevalPersonal AI Assistant Benchmarks & EvalsResearch preview
Personal assistants · everyday tasks · inspectable evidence
Personaeval

Personaeval

Research & settings

Keep Personaeval on your home screen

Open it in its own window on your phone or tablet.

Installation guide →

Research & protocols

Personal assistant benchmarks

Find relevant research, understand what each evaluation measures, and inspect the limits of its claims.

External research · retrieved 2026-10-31

Travel, use cases and reports

These are source-reported facts and links from Assistant Benchmark, not Personaeval trials. The source’s detailed evidence and writing remain on its own site.

Travel results

The source travel grid, labeled a design preview, contains 5 assistants. The source assigns booking speed 40% of its composite; compare task results individually if speed is not your priority. Miso’s reported booking time excludes earlier profile setup, so it is not directly comparable to an end-to-end onboarding time. Read the source for the task and evidence.

Source-reported travel scores; booking times are approximate
AssistantTravel overallBooking timeSpeed scoreOnboarding flowBooking executionSeat selectionTravel profileCancellationRebookingProactiveness
Miso9.3/10~3 min *10/1010/107/1010/1010/107/1010/108/10
Instinct8/10~6 min9/107/107/1010/1010/107/103/107/10
Grok Bot7.9/10~15.5 min7/107/1010/109/1010/109/107/107/10
Dots7.1/10~33 min5/107/107/1010/1010/107/1010/108/10
Muse6.9/10~16 min7/107/1010/103/107/1010/103/108/10

Use-case index

Of 59 indexed cases, 14 have at least one source-labeled tested result and 38 have reported examples but no source-labeled test. Counts describe the source’s labels, not independent verification or the number of successful attempts. Open each source page for its scope and evidence.

59 matching cases

Reports and writing

Source titles are indexed here; full reports and articles are read at the source.

Primary sources · Reviewed 17 September 2026

Find an evaluation for your use case

Our proposed task suite ↓

A curated starting point for builders. Review scope and limitations are shown for every source. We have not reproduced these benchmarks or imported their scores.

8 of 8 sources

Tool workflowsAbstract reviewed

PAUSE ↗

Benchmark paper

Personal-assistant evaluation with stateful services, persistent user settings, permissions, and simulated users.

Useful for
Design tasks that carry user context and constraints across service interactions.
Scope limit
The paper has not been reproduced here. Its reported results are not scores for products in our directory.
Read primary source ↗
Computer useAbstract reviewed

MyPCBench ↗

Benchmark paper

184 personalized computer tasks across 17 simulated web applications in a Linux desktop environment.

Useful for
Study assistants that coordinate work across applications using computer and shell interfaces.
Scope limit
A single constructed persona does not establish coverage of a diverse user population. Not reproduced here.
Read primary source ↗
MemoryAbstract reviewed

LongMemEval ↗

Benchmark paper

500 questions covering extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention.

Useful for
Evaluate recall and correction of user information over conversation histories.
Scope limit
A memory component benchmark does not establish reliable end-to-end task execution. Not reproduced here.
Read primary source ↗
Tool workflowsAbstract reviewed

τ-bench ↗

Benchmark paper

Interactive tool use with simulated users and domain policies; checks database outcomes and repeated reliability.

Useful for
Separate successful conversation from a correct final state, and measure consistency across trials.
Scope limit
Retail and airline tasks are domain-specific; their scores do not transfer directly to personal assistants. Not reproduced here.
Read primary source ↗
Permissions & securityAbstract reviewed

AgentDojo ↗

Benchmark paper

A sandbox for evaluating useful tool execution under prompt injection in untrusted external content.

Useful for
Test whether malicious tool output changes an assistant's authorized behavior.
Scope limit
Injection resistance is one security property, not a complete privacy or reliability assessment. Not reproduced here.
Read primary source ↗
Evaluation designArticle reviewed

Demystifying evals for AI agents ↗

Engineering guide · Anthropic

Practical guidance on tasks, trials, graders, transcripts, and outcomes in agent evaluation.

Useful for
Reset trial environments, validate graders, inspect traces, and distinguish eventual success from consistent success.
Scope limit
Engineering guidance is not empirical evidence about any assistant listed on this site.
Read primary source ↗
Evaluation designArticle reviewed

Trustworthy third-party evaluations ↗

Research guidance · OpenAI

Evaluation design that makes claims, elicitation methods, harness choices, and resource budgets explicit.

Useful for
Predeclare the comparison being made and disclose tools, scaffolding, scoring, and inference budgets.
Scope limit
Guidance does not certify this platform or establish that different product configurations are comparable.
Read primary source ↗
Evaluation designCollection page reviewed

Google DeepMind evaluations ↗

Research collection · Google DeepMind

Primary evaluation resources with datasets, papers, and supporting materials across several capabilities.

Useful for
Find task-specific resources and inspect how evaluation targets and data quality are defined.
Scope limit
This collection is not a personal-assistant product benchmark. Individual resources require separate review.
Read primary source ↗

Proposed protocol · v0.1-preview

Everyday Delegation

Ten task families for studying completion, repeatability, privacy, approval boundaries, and recovery.

Synthetic research preview

The suite currently illustrates the method with fictional assistants and six authored run receipts. A task family with no receipt is explicitly marked; it is not evidence of tested capability.

Read the evaluation protocol →
Task 01

Plan my day

Build a realistic day plan around my meetings and commitments

No bundled run receipt
Explore task evidence →
Task 07

Coordinate a group

Find a dinner time without exposing individual calendar details

No bundled run receipt
Explore task evidence →
Task 08

Deal with a business

Call about an appointment and return a verifiable outcome

No bundled run receipt
Explore task evidence →