Assessing Count Measurement Systems

Abstract

Measurement system analysis provides a method to quantify the sources of variation in observed data collected from a measurement system. Extensive literature describes plans and analyses for assessing a continuous measurement system. However, there has been limited research on how to assess a measurement system when the outcome is a count. Processes with a count outcome occur across various industries, such as manufacturing and medicine. In this thesis, we propose a new model and approach for assessing count measurement systems.

First, we introduce measurement system analysis and provide a general framework for this thesis. Then, we introduce a flexible count measurement system assessment model and approach when a gold standard or the true count is available. In this model, we assume that the true counts arise from a Poisson distribution and that the observed counts are subject to two types of errors. We may miss a true defect, and we may obtain a false positive by labeling something flawless as a defect. We initially consider a gold standard plan where we have access to all the information. Then, we consider a second plan where we only know the true count. We compare and discuss these plans empirically and theoretically.

We then extend this framework to allow for a scenario in which no gold standard is available. To evaluate the count measurement system, we propose a single-phase and two-phase sampling plan. In the single-phase plan, a small sample of parts is selected at random from the manufacturing process and each selected part is measured repeatedly. With the two-phase plan, we begin with an initial measurement of a larger set of parts, from which a subset of these parts is then selected for additional measurements. In the second phase, the selection of parts for remeasurement is not random, but guided by the initial observations, allowing us to focus on parts that are most informative for assessing measurement system performance. Three selection strategies for the two-phase plan are investigated. We compare and discuss these plans empirically and theoretically. Finally, we show how these methods work in practice by applying them to real experimental data.

Finally, we relax the assumption that each defect has the same probability of detection. We assume that each defect has a varying probability of detection based on its severity, which is assumed to follow a Beta distribution. We propose five plans, each with varying levels of information, to assess a count measurement system under these conditions. We then compare these plans empirically and theoretically. Additionally, these plans are compared to the analogous case where the probability of detection is fixed. Finally, we analyze a data example and provide practical insights.