A new statistical research article is published by B&B faculty Huiman Barnhart, PhD in Statistics in Medicine.
There has been substantial statistical literature in the last several decades on assessing agreement, and coverage probability approach was selected as a preferred index for assessing and improving measurement agreement in a core laboratory setting. With this approach, a satisfactory agreement is based on pre-specified high satisfactory coverage probability (e.g., 95%), given one pre-specified acceptable difference. In practice, we may want to have quality control on more than one pre-specified differences, or we may simply want to summarize the agreement based on differences up to a maximum acceptable difference where the coverage probability should be 100%. Relative area under the coverage probability curve is proposed for the summary of overall agreement. The coverage probability curve provides an intuitive display on the full spectrum of measurement error over a range of observed differences/disagreement. The relative area under the curve provides a new way to compare different intra-methods or inter-methods/labs/observers’ agreements for measuring the same variable. The new index can even be used to compare agreement/reproducibility of methods across different variables if the chosen maximum acceptable differences have the same interpretation in practice. This new approach provides a better tool for the investigators to understand the magnitude of the measurement error or the extent of agreement than the traditional approaches. Estimation, Inference and simulation studies are provided in the paper. The methodology is illustrated with a data example to determine if a digital device can be used to replace nurses on measuring blood pressure.