Learning Trustworthy Web Sources to Derive Correct Answers and Reduce Health Misinformation in Search
dc.contributor.author | Zhang, Dake | |
dc.contributor.author | Vakili Tahami, Amir | |
dc.contributor.author | Abualsaud, Mustafa | |
dc.contributor.author | Smucker, Mark | |
dc.date.accessioned | 2022-05-11T18:27:14Z | |
dc.date.available | 2022-05-11T18:27:14Z | |
dc.date.issued | 2022-07 | |
dc.description | Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. SIGIR ’22, July 11–15, 2022, Madrid, Spain © 2022 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-8732-3/22/07...$15.00 https://doi.org/10.1145/3477495.3531812 | en |
dc.description.abstract | When searching the web for answers to health questions, people can make incorrect decisions that have a negative effect on their lives if the search results contain misinformation. To reduce health misinformation in search results, we need to be able to detect documents with correct answers and promote them over documents containing misinformation. Determining the correct answer has been a difficult hurdle to overcome for participants in the TREC Health Misinformation Track. In the 2021 track, automatic runs were not allowed to use the known answer to a topic’s health question, and as a result, the top automatic run had a compatibility-difference score of 0.043 while the top manual run, which used the known answer, had a score of 0.259. The compatibility-difference measures the ability of methods to rank correct and credible documents before incorrect and non-credible documents. By using an existing set of health questions and their known answers, we show it is possible to learn which web hosts are trustworthy, from which we can predict the correct answers to the 2021 health questions with an accuracy of 76%. Using our predicted answers, we can promote documents that we predict contain this answer and achieve a compatibility-difference score of 0.129, which is a three-fold increase in performance over the best previous automatic method. | en |
dc.description.sponsorship | Natural Sciences and Engineering Research Council of Canada, RGPIN-2020-04665, RGPAS- 2020-00080 || Mitacs || Compute Canada || University of Waterloo. | en |
dc.identifier.uri | https://doi.org/10.1145/3477495.3531812 | |
dc.identifier.uri | http://hdl.handle.net/10012/18257 | |
dc.language.iso | en | en |
dc.publisher | ACM | en |
dc.relation.uri | https://github.com/UWaterlooIR/golden-gaze | en |
dc.subject | Health Misinformation | en |
dc.subject | Stance Detection | en |
dc.subject | Web Search | en |
dc.title | Learning Trustworthy Web Sources to Derive Correct Answers and Reduce Health Misinformation in Search | en |
dc.type | Conference Paper | en |
dcterms.bibliographicCitation | Dake Zhang, Amir Vakili Tahami, Mustafa Abualsaud, and Mark D. Smucker. 2022. Learning Trustworthy Web Sources to Derive Correct Answers and Reduce Health Misinformation in Search. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’22), July 11–15, 2022, Madrid, Spain. ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/3477495.3531812 | en |
uws.contributor.affiliation1 | Faculty of Engineering | en |
uws.contributor.affiliation1 | Faculty of Mathematics | en |
uws.contributor.affiliation2 | David R. Cheriton School of Computer Science | en |
uws.contributor.affiliation2 | Management Sciences | en |
uws.peerReviewStatus | Reviewed | en |
uws.scholarLevel | Faculty | en |
uws.scholarLevel | Graduate | en |
uws.typeOfResource | Text | en |