Benchmark

AI scribe benchmark: Hanah and seven other AI scribes

We played the same 13 recorded consultations into Hanah, Heidi, Lyrebird, Freed, PatientNotes, CliniScripts, Preve and Arlo, then graded every note against what was actually said. Every transcript, note and grade is published on this page, with the reason for each one.

92% of Hanah's notes free of major hallucination flags The average across the other seven scribes was 42%.
61% fewer clinically significant transcript errors 0.85 per consultation against an average of 2.17.
13% more required clinical details captured 93.1% against an average of 82.3%.

Results

Scribes are listed by the share of required clinical details their notes captured. Each scribe's note for the first prompt of each recording is scored.

AI scribe Clinical details captured Notes with every clinical detail Major hallucination flags per note Major hallucination flags, all notes Major hallucination flags per document, including summaries and letters Major hallucination flags, including summaries and letters Notes with no major hallucination flag Significant transcript errors per consultation Word Error Rate, English consultations
Hanah13 notes 93.1% 62% 0.23 3 0.17 4 92% 0.85 1.6%
Heidi13 notes 91.4% 38% 0.31 4 0.22 5 69% 1.08 3.3%
CliniScripts13 notes 90.8% 62% 0.54 7 0.35 8 62% 1.08 2.1%
Lyrebird13 notes 89.7% 54% 0.85 11 0.74 17 38% 2.08 3.4%
Freed†12 notes 84.5% 50% 0.50 6 0.43 9 58% 1.67 19.4%
PatientNotes13 notes 75.3% 15% 2.46 32 2.43 56 15% 2.38 3.9%
Arlo13 notes 72.4% 31% 1.00 13 1.04 24 23% 3.62 5.2%
Preve*10 notes 72.2% 30% 2.40 24 2.40 24 30% 3.31 6.7%
Average of the other seven 82.3% 40% 1.15 13.9 1.09 20.4 42% 2.17 6.3%

* Preve was scored on its standard clinical note, because it follows a custom instruction only after a sample PDF is uploaded to build a template. Preve doesn't generate a note from less than 600 characters of transcript, and its three shortest recordings have no note.

† Freed kept recording after Stop on the shortest recording and produced no transcript or note, on two attempts. Freed is scored on the other 12.

A significant transcript error is one a blind rater judged could mislead a clinician: a wrong drug, dose, number, side or symptom, a lost negation or red flag, or clinical content dropped. Word Error Rate counts every word the same, fillers included.

The jaw surgery review was spoken mostly in Mandarin. Each scribe ran it with its input language set to Mandarin where it offers that setting. Heidi made the fewest significant transcript errors on it (2), then Hanah (5) and CliniScripts (6). Freed made 10, PatientNotes 12, Lyrebird 14, and Arlo and Preve 18 each. Arlo and Preve have no input language setting, and Freed detects the language itself.

By consultation

Clinical details captured, with the number of major hallucination flags underneath. Select a consultation to read every note and grade.

ConsultationHanahHeidiCliniScriptsLyrebirdFreedPatientNotesArloPreve
ICU handover dictation 100%no major 100%no major 100%no major 100%no major 100%no major 83%3 major 83%1 major no note
ICU handover dictation, ward noise 100%no major 100%no major 100%no major 92%1 major 75%1 major 83%2 major 83%1 major no note
Community nurse home visit 100%no major 100%no major 86%1 major 100%no major 100%no major 64%no major 93%1 major 93%2 major
Dietitian, IBS review 100%no major 100%no major 100%1 major 92%1 major 100%no major 83%2 major 100%no major 92%4 major
Knee pain, short consult 83%no major 83%no major 83%no major 83%no major no note 100%no major 100%no major no note
Knee replacement prehab 100%no major 88%no major 100%no major 100%1 major 100%1 major 100%1 major 100%no major 88%1 major
Myotherapy, 50 minutes in a noisy room 100%no major 88%no major 56%no major 100%no major 81%1 major 88%1 major 81%1 major 100%no major
OT home safety assessment 100%no major 100%1 major 100%no major 100%1 major 100%no major 86%3 major 93%1 major 100%7 major
Child with asthma, parent with limited English 100%no major 86%1 major 100%no major 100%1 major 100%no major 93%3 major 93%2 major 71%4 major
Jaw surgery review in Mandarin and English 81%3 major 84%no major 84%2 major 66%4 major 62%2 major 47%11 major 6%2 major 22%3 major
Quad bike fall, outdoors 86%no major 86%1 major 100%2 major 86%1 major 79%1 major 64%3 major 71%1 major 64%3 major
Rugby knee prehab 67%no major 83%no major 100%no major 100%1 major 50%no major 33%2 major 100%1 major 100%no major
Rugby hamstring, sideline 93%no major 93%1 major 93%1 major 86%no major 86%no major 93%1 major 71%2 major 79%no major

Every note, transcript and grade

Choose a consultation and a scribe. You can play the recording, read the reference transcript, and see each checklist verdict and hallucination flag with the reason it was given.

Loading…

How it was run

The recordings

The 13 recordings are Hanah's accuracy corpus, the same set behind the release results on our transparency page. They cover an ICU handover dictation with and without ward noise, physiotherapy, dietetics, occupational therapy, myotherapy and community nursing consults, a child with asthma whose parent speaks limited English, a quad bike fall recorded outdoors, a 50 minute myotherapy session with about seven minutes of speech in a noisy treatment room, and a jaw surgery review spoken mostly in Mandarin. Each has a verbatim reference transcript and a checklist of the details a note must capture.

Capture

Each scribe ran in its own web app in Chrome, signed in to a standard account. The recording was played through the browser's microphone input, the same file for every scribe. After recording, each recording's prompts were typed into the scribe's own document or chat feature word for word, for example "Generate SOAP notes please" and "Summarise this consult in two sentences for a referral letter". Each scribe was run once per recording on 6 October 2026, and Arlo on 20 October 2026, with English as the input language where a scribe asks for one. For the Mandarin review, scribes with an input language setting were set to Mandarin: Heidi to English and Chinese (Mandarin), Lyrebird and PatientNotes to Chinese (Mandarin, Simplified). CliniScripts produced no transcript with Chinese and English selected on two attempts, so its standard setting is used. Arlo has no input language setting. Arlo builds documents from saved templates rather than a free-text box, so each prompt was typed into Arlo's own template generator word for word and the template it built was used unedited.

Grading by a Claude jury

The notes were graded by Claude, Anthropic's model. For each recording, every scribe's output was shuffled behind a letter with product names removed, and graded against the reference transcript with a fixed rubric. Each checklist item is marked yes, partial or no. Every statement the transcript doesn't support is listed as a hallucination flag: major when it could change care or misstate the clinical picture (an invented symptom, finding, drug, diagnosis or plan, or a wrong side or drug name), minor otherwise. On 20 October 2026, when Arlo was added, every scribe's notes and transcripts were graded again together in the same blind packets, so all eight are scored by the same grader. The grades were unblinded only after every recording was graded. The full rubric is in our methodology.

Transcript errors

Each transcript was aligned word by word with the verbatim reference. Differences in filler words were set aside, and Claude judged each remaining difference blind, using the rule above for what makes a transcript error significant. Word Error Rate is calculated by code with the same strict normalisation as our own release reports, treating equivalent written forms as the same words: 22 mm and 22 millimetres, US and UK spellings, Chinese and Arabic numerals, and Simplified and Traditional characters. Chinese is scored per character.