Browsing by Author "Amenyo, Solomon"

Now showing 1 - 2 of 2

Assessment of AI-Generated Pediatric Rehabilitation SOAP-Note Quality
(University of Waterloo, 2025-02-19) Amenyo, Solomon; Grossman, Maura; Brown, Daniel; Wylie-Toal, Brendan
This study explores the integration of artificial intelligence (AI) or large language models (LLMs) into pediatric rehabilitation clinical documentation, focusing on the generation of SOAP (Subjective, Objective, Assessment, Plan) notes, which are essential for patient care. Creating complex documentation is time-consuming in pediatric settings. We evaluate the effectiveness of two AI tools; Copilot, a commercial LLM, and KAUWbot, a fine-tuned LLM developed for KidsAbility Centre for Child Development (an Ontario pediatric rehabilitation facility), in simplifying and automating this process. We focus on two key questions: (i) How does the quality of AI-generated SOAP notes based on short clinician summaries compare to human-authored notes, and (ii) To what extent is human editing necessary for improving AI-generated SOAP notes? We found no evidence of prior work assessing the quality of AI-generated clinical notes in pediatric rehabilitation. We used a sample of 432 SOAP notes, evenly divided among human-authored, Copilot-generated, and KAUWbot-generated notes. We employ a blind evaluation by experienced clinicians based on a custom rubric. Statistical analysis is conducted to assess the quality of the notes and the impact of human editing. The results suggest that AI tools such as KAUWbot and Copilot can generate SOAP notes with quality comparable to those authored by humans. We highlight the potential for combining AI with human expertise to enhance clinical documentation and offer insights for the future integration of AI into pediatric rehabilitation practice and other settings for the management of clinical conditions.
Bridging Technology and Therapy: Assessing the Quality and Analyzing the Impact of Human Editing on AI-Generated SOAP Notes in Pediatric Rehabilitation
(University of Waterloo, 2025-04-07) Amenyo, Solomon
This thesis explores the integration of artificial intelligence (AI) into clinical documentation, focusing on evaluating AI-generated SOAP (Subjective, Objective, Assessment, and Plan) notes in pediatric rehabilitation settings. AI-powered tools, such as Microsoft's Copilot and the University of Waterloo-developed KAUWbot, offer potential efficiencies by automating aspects of clinical documentation. However, their quality, reliability, and applicability to clinical practice have remained largely unexamined. The research aims to assess and compare the quality of human-authored SOAP notes and AI-generated notes across five key criteria: clarity, completeness, conciseness, relevance, and organization. A dataset of 432 SOAP notes, divided into four pools, was evaluated using a custom rubric. The pools included human-authored notes, Copilot-generated notes edited by occupational therapists, unedited KAUWbot-generated notes, and KAUWbot-generated notes edited by occupational therapists. A rigorous anonymization process ensured evaluator impartiality. Findings indicate that AI-generated notes, particularly when edited by clinicians, achieve comparable or superior quality to human-authored notes. After editing, notes generated by KAUWbot, a model fine-tuned on pediatric occupational therapy data, exhibited notable improvements in relevance and organization. Statistical analyses demonstrated some differences among note pools, with edited AI-generated notes consistently receiving the highest ratings. This highlights the importance of human oversight in enhancing AI output and tailoring it to specific clinical needs. The research shows the potential of AI to augment clinical documentation processes, reduce clinician workload, and improve documentation quality. However, it also emphasizes the necessity of human-AI collaboration and robust training to mitigate limitations such as contextual inaccuracies and misclassifications. These findings provide a foundation for future research and practical recommendations for integrating AI into healthcare documentation workflows.