Does Impact Analysis Support the Review of Changes to Build Specifications?

Abstract

Build systems require maintenance. Since automated quality assurance rarely targets build specifications, peer review remains the key mechanism for detecting faults; yet, previous research reports that reviews of build changes are hindered by limited tool support and the scarcity of developers with build expertise. Moreover, build changes often introduce cascading effects that propagate beyond the modified files. Such effects make it difficult for reviewers to understand the consequences of a change to build specifications, particularly without dedicated tooling. In this paper, we propose a controlled experiment to examine whether access to Build Change Impact Analysis (BCIA) enhances developer understanding and assessment of changes to build specifications. BCIA summarizes how changes propagate through build specifications via data-and control-flow relationships. We conduct this study using BuiScout, a prototype implementation of BCIA for CMake-based build systems. Using a within-subjects experimental design, participants will perform comprehension and reasoning tasks with and without access to BuiScout. These tasks will evaluate their ability to assess change impact and reason about its implications. We will measure and compare response accuracy, explanation quality, and task completion time, along with self-reported estimates of confidence and perceived difficulty. Together, these measurements will allow us to study whether BCIA (implemented in BuiScout) enhances the effectiveness and efficiency of reviewing changes to build specifications.