The Use of Artificially Intelligent Symptom checkers by University Students – An Exploratory Sequential Mixed Methods Study

Abstract

Rising healthcare costs, wait times, unnecessary care, and lack of access to a primary care provider, are pressing issues encountered by various health systems and in part, are a result of misinformed patient demand. The literature suggests that one way to economize the healthcare system is to provide patients with reliable tools to inform better decisions on when to seek care. The Internet is often used as a source of health information. University students, a population group considered to be technology savvy, often browse the Internet for health topics and self-diagnosis; however, the information is abundant and may not be reliable which can have negative consequences on health. Relatively new artificially intelligent (AI)-enabled symptom checkers seek to address this limitation by enabling self-triage and self-diagnosis based on data inputted by users. Given the direct-to-consumer nature of this technology and availability, little to none is known about the factors associated with the behavioural intention to use this technology. This thesis focuses on university students between the ages of 18 and 34 – a demographic that is technology savvy and undergoing a critical transition period as they start making individual decisions regarding their health. Inspired by the Technology Acceptance Model (TAM), the overarching aim of this dissertation was to understand university students’ use of AI-enabled symptom checkers for self-triage and self-diagnosis.

A scoping review conducted as part of this work helped inform the following research questions: (i) What are university students’ perspectives towards the use of AI-enabled symptom checkers for self-triage and self-diagnosis? (ii) What are university students’ perspectives on the platform’s influence on the use of health services? (iii) What are the population profiles (latent classes) associated with the intent to use AI-enabled symptom checkers? This study received ethics clearance from the Research Ethics Board (#41366) and approval from the Institute of Analysis and Planning at the University of Waterloo.

A two-phased mixed methods sequential exploratory research design was used for objectives (i) and (ii) using qualitative research methods and procedures (i.e., semi-structured interviews and a think-aloud exercise). A total of 24 participants were recruited across faculties at the university to address research question 1 of which 22 were included in the sample to address the second research question. Using NVivo software, inductive thematic analysis, informed by the factors identified in the UTAUT, was used to analyze qualitative data.

Findings from the qualitative phase suggests that more than half of participants (n=13) were unaware about the existence of symptom checkers prior to the study. Most participants had a positive outlook on the use of AI in healthcare due to the use of big data and pattern recognition; however, skepticism regarding the quality of data used and biases against minority groups emerged. Based on participants’ experience using a symptom checker during the interview, the platform was perceived to be more personalized and interactive in nature as compared to using the Internet search engine for seeking health information. Symptom checkers, however, were believed to be unreliable if it limits a user’s input of data and were thus more accepted for self-triage rather than self-diagnosis. Many barriers and enablers – related to the individual, disease, healthcare system, or the symptom checker itself – for using symptom checkers were identified. Some enablers included trust, curiosity, having pre-existing or “embarrassing” health conditions, being uncertain about the care required, experiencing symptoms that can be easily described, endorsement by doctors and health organizations, and increased awareness regarding their existence. Identified barriers included the use of medical jargon, lack of explanation as to why certain questions are being asked, disclaimer undermining the credibility of the platform, skepticism from the media regarding the use of AI and lack of human interaction. Following the use of a symptom checker, participants mentioned various areas of improvement that would enhance the user experience – these included having the ability of entering symptoms as free text, the use of visuals to pinpoint affected areas, tailoring the experience based on a user’s health literacy, providing an option to speak to a health provider following the initial assessment, providing information related to other users who reported similar symptoms, providing information on the conditions listed, and recommending nearby locations for accessing health services.

Symptom checkers were perceived to have a positive effect on health of university students through the integration of health reminders, enablement of proactive care seeking, and mental health. A few participants believed that it may have a negative influence on health due to a suboptimal understanding of the user’s contextual factors, overall health status, and the reactive nature of the platform (i.e., focus on symptoms). As for the platform’s influence on health services, symptom checkers were perceived to affect three main areas that include the reduction in unnecessary medical visits, increasing patient engagement and improving access to care. To have any influence on the use of health services, symptom checkers must be adopted by the general public with the top five factors identified by participants to be important for adopting a symptom checker for self-triage being trust towards the platform, perceived credibility, demonstrability, perceived accessibility (for self-triage), and output quality (for self-diagnosis). Given its centrality, trust was explored further – participants believed that symptom checkers could be trusted for minor conditions. Moreover, various factors related to the input, process, and output were considered to influence a user’s level of trust in the platform.

To address the third objective, findings from the first phase and input from the Survey Research Center, were used to develop a survey. University students were notified of the survey through an email invitation sent by the Registrar’s office and an announcement made in a newsletter. A total of 1,547 students opened the survey link of which some were screened out due to ineligibility (n=14) and others were removed due to their prior use of symptom checkers (n=180). The remaining sample who had not used the platform in the past year were identified as “non-users” and were the focus of this thesis. Quantitative analyses were conducted on complete cases (n=1,305) with the sample being approximately evenly split between men and women, healthy, non-white, enrolled in an undergraduate program, and often have access to the Internet. Latent Class Analysis (LCA) was used to understand response patterns and define the population profiles (latent classes) which were identified based on attitudes towards symptom checker functionality and AI – these five classes were labeled as tech acceptors, tech rejectors, skeptics, unsure acceptors, and tech seekers. Using a General Linear Model (GLM), these latent classes were regressed on the intent to use symptom checkers (the outcome had three categories with the category of interest being the use of symptom checkers as compared to the neutral referent group) while controlling for confounders (i.e., gender, self-perceived health, race, healthcare use, wait time, health literacy). Results suggest a significant effect of latent classes on the intent to use symptom checkers, even when controlling for other variables (p-value <.0001). As compared to tech rejectors, the odds of future symptom checker use are 7.6, 5.6, 2.6, and 2 times higher in tech seekers, tech acceptors, skeptics, and unsure acceptors, respectively. Interestingly, tech seekers – categorized as a latent class that has positive attitudes about functionality and AI but do not perceive to have accessibility to symptom checkers – was the class with the highest odds of intending to use a symptom checker. This may suggest that the perception of not having access to symptom checkers increases the odds of intending to use a symptom checker. Moreover, addressing the variables that categorize “skeptics” and “unsure acceptors” will be important to understand which aspects should be addressed to increase symptom checker use. For example, skeptics were defined as the group that perceive symptom checkers to be easy to use but have negative attitudes towards the output of the platform whereas unsure acceptors show negative attitudes towards the output but perceive the platform difficult to use.

Findings from this work have theoretical, methodological, and practical implications that will inform the use of an understudied technology. This thesis contributes to the technology acceptance literature by focusing on a relatively new technology that has not been studied in this population all the while employing a sophisticated statistical technique (i.e., LCA) that provides valuable insights about the population profiles among subjects. Methodological implications include the interview protocol, survey development, and survey data analysis, which could be employed in future studies to assess symptom checker acceptance and use among other population groups. Moreover, policymakers, health professionals, health institutions, technology companies, and the general public may be interested in understanding the perceptions of AI-enabled health diagnostics, as well as the factors and profiles associated with its intended use. Importantly, understanding end-user reception of this technology will inform the integration of AI-enabled symptom checkers by healthcare systems as a potential approach to economize and reduce the burden on these systems.