Not All Pull Request Rejections Are The Same

Abstract

In the Open Source Software (OSS) development landscape, evaluating pull requests extends beyond code quality assessment. Recent research has revealed the significant influence of social dynamics and perceptions on pull request evaluations, a notion our study seeks to expand upon. By examining the intricate reasons behind pull request rejections, we aim to move beyond the traditional view of rejections as a monolithic category. Utilizing a dataset comprising of 52,829 pull requests across 3,931 projects, we conduct a large-scale comprehensive analysis identifying twelve distinct categories of rejection rea- sons. Our findings underscore that although social ties and technical abilities are factors that influence pull request decisions, they are not consistent across all rejection reasons. Notably, certain characteristics, such as extensive line changes and team size, exhibit varied impacts on different types of rejections, indicating the complex interplay between social and technical factors in pull request assessments. This study provides a multifaceted understanding of the OSS contribution evaluation process, highlighting the complexity and diversity of rejection reasons. By describing the specific features that influence distinct types of rejections, we contribute to the development of more nuanced strategies for managing contributions. Our findings offer valuable insights for both contributors and project maintainers, emphasizing the need for a tailored approach to understanding and enhancing the pull request evaluation process in OSS projects.