Inter-reader agreement of the Prostate Imaging Quality (PI-QUAL) score: A bicentric study

Purpose: To investigate the inter-reader agreement of the Prostate imaging quality (PI-QUAL) for multiparametric magnetic resonance imaging (mpMRI). Methods: We included 66 men who underwent 1.5 T mpMRI in June 2020-July 2020 in center 1, with no exclusion criteria. mpMRI included multiplanar T2-weighted imaging (T2WI), Diffusion-weighted imaging (DWI), and dynamic contrast-enhanced imaging (DCE). Two readers from center 2 (experience < 1000 to > 1500 examinations), blinded to clinical history but not to acquisition parameters, independently assessed PI-QUAL qualitative/anatomical items of each mpMRI sequence, final PI-QUAL score (1-5), and the Prostate imaging reporting and data system version 2.1 (PI-RADSv2.1) category of the index lesion. Cohen's kappa statistics (k) or prevalence-adjusted-bias-adjusted kappa (PABAK) were used to calculate the inter-reader agreement in assessing the PI-QUAL (1-to-5 scale and 1-2 versus 3 versus 4-5), the diagnostic quality of each mpMRI sequence, qualitative/anatomical items, and PI-RADSv2.1 category. Results: The inter-reader agreement for PI-QUAL category was moderate upon 1-5 scale (k = 0.55; 95%CI 0.39-0.71) or 1-3 scale (k = 0.51; 95%CI 0.29-0.72), with 90.9% examinations categorized PI-QUAL > 3 by readers. The agreement in assessing a sequence as diagnostic was higher for T2WI (k = 0.76) than DCE (PABAK = 0.61) and DWI (k = 0.46), ranging moderate-to-substantial for most of the items. Readers provided comparable PI-RADSv2.1 categorization (k = 0.88 [excellent agreement]; 95%CI 0.81-0.96), with most PI-RADSv2.1 > 3 assignments found in PI-QUAL > 3 examinations (43/46 by reader 1, and 47/47 by reader 2). Conclusions: The reproducibility of PI-QUAL was moderate. Higher PI-QUAL scores were associated with excellent inter-reader agreement for PI-RADSv2.1 categorization.