Adaptive Human-Chatbot Interactions: Contextual Factors, Variability Design and Levels of Automation

Abstract

The landscape of software development is undergoing a significant transformation characterized by various factors. A notable shift is the surging demand for software developers, driven by industries' increasing reliance on software solutions to support their operations. This increased demand is accompanied by an escalation in the complexity of software development projects. In this dynamic environment, modern software systems interact with numerous external systems, interfaces, data sources, and work practices. This complexity requires developers to navigate a complex environment while creating software.

Adding to this landscape is the emergence of AI-based conversational systems, a transformative trend that is shaping software development processes. These systems, powered by artificial intelligence and natural language processing, enable human-like interactions through chatbots and virtual assistants. Software developers are increasingly turning to AI-powered chatbots to support their work. These chatbots play diverse roles, ranging from technical query resolution or load testing to providing project management insights and automating routine tasks. By harnessing the capabilities of these AI-driven tools, developers can potentially enhance productivity, access pertinent information swiftly, and optimize their workflows.

However, amid these developments, many challenges arise due to the intricate web of contextual factors that influence software development processes, especially when chatbots come into play. These contextual factors act as distinct pieces of a puzzle, each altering how software development functions in the presence of chatbots. Unfortunately, the existing research landscape has a limited understanding of these contextual intricacies, resulting in insufficient design methods to adequately support developers using chatbots. Moreover, addressing the customization of automation levels in these interactions remains unexplored.

With the growing complexity of software development, coupled with the emergence of advanced, AI-based conversational systems, the integration of chatbots to support developers in their work has become prominent. There is a pressing need to address the challenges in human-chatbot interactions, particularly in leveraging conversational agents’ advances to tailor interactions to developers’ specific contexts and desired levels of automation. This research explores the design of context-based adaptive interactions between software developers and chatbots. By understanding and integrating the contextual factors that influence software development with chatbots, we aim to gain novel insights into developers’ expectations regarding these interactions and the levels of automation involved and advance the design of human-chatbot adaptive applications.

First, I perform a user study to investigate the requirements of conversational agents in software development. I uncovered a vast list of desired requirements and insights from participants, including that they are interested in working with such tools, in various parts of the development lifecycle such as managing their tasks and version control. One of the insights of this study was that contrary to the authors' beliefs, not all developers were interested in automating all possible tasks. This insight led me to the next part of this thesis, which was the investigation of the factors that influence how much automation is desired in systems.

I then perform a literature review focused on studies about taxonomies of levels of automation. I aimed to uncover from these studies, the factors that influence systems switching from one level of automation to a different level. I identified these factors and composed a list of 61 factors, which we divided into five categories, system, task, human, environment, and quality. I propose feature model designs to represent these factors and their relationships and instantiate this model with use cases.

This research provides a roadmap for the design of adaptive chatbot interactions that align with developers' specific needs and workflows. Empirical studies are conducted to gain insights from developers' experiences and expectations, ultimately driving the design of context-aware chatbot interactions. Additionally, by examining the influence of varying levels of automation on these interactions, I sought to identify factors that shape the variability of automation levels, bridging the gap between human-system interactions and autonomous systems.