Browsing by Author "Frasheri, Kris"

Filter results by typing the first few letters
Now showing 1 - 1 of 1
  • No Thumbnail Available
    Item
    PERSONA: A Tool for Generating Algorithmic Personas for Reflective Annotations
    (University of Waterloo, 2025-01-02) Frasheri, Kris
    The domain of machine learning (ML) has grappled with the challenge of curating subjective datasets, where there can be many equally valid labels due to differences in perspectives and a significant technical gap remains in how we can effectively incorporate multiple subjective viewpoints into the labelling process. We contribute PERSONA, a dataset labelling tool that presents LLM-generated personas with diverse labelling per- spectives to encourage annotators to consider different human values during the dataset labelling process. We studied how interactions with these personas affect the annotator’s decision-making patterns. Based on a two-part user study, our evaluation shows that PERSONA enriches the labelling process by prompting the annotators to reflect on differ- ent viewpoints, showing the potential value of integrating LLMs in machine learning data generation pipelines.