A study found that clinical notes generated by artificial intelligence tools consistently ranked lower in quality and accuracy compared to those written by clinicians.
RT’s Three Key Takeaways:
- Lower Quality Scores: AI-generated clinical notes scored lower than human-written notes across 10 quality domains, including accuracy, usefulness, and organization.
- Standardized Comparison: Researchers compared 11 AI scribe tools against 18 clinicians, including physicians and nurse care managers, using audio recordings of standardized patient visits.
- Drafting Tool Only: The study authors recommend that AI scribes be used only for generating draft documentation that requires review and editing by a clinician.
Clinical notes generated by ambient artificial intelligence (AI) scribe tools are consistently lower in quality than those produced by human clinicians, according to a cross-sectional evaluation from the Veterans Health Administration, according to research were presented at ACP Internal Medicine Meeting 2026 and published in Annals of Internal Medicine.
While ambient AI scribes have been shown to reduce administrative burden for physicians, researchers sought to compare documentation quality between AI and humans in a healthcare setting. The team used audio recordings of five standardized primary care visits and asked 11 AI scribe tools and 18 human clinicians—including physicians, pharmacists, and nurse care managers—to generate clinical notes.
Thirty blinded raters scored each note using the modified Physician Documentation Quality Instrument (PDQI-9), which measures quality across 10 domains. In every case, human notes scored higher, while AI notes consistently scored lower in accuracy, thoroughness, usefulness, organization, and “comprehensiveness.”
“Rigorous and ongoing testing of their accuracy and quality is critical before relying on them in clinical care,” said the study authors, researchers from the Veterans Health Administration, in a news release.
The researchers concluded that AI scribes should be regarded as tools for generating draft documentation that requires review and editing, rather than as a substitute for clinician-authored notes.
