Understanding PT performance assessment

Significance of the topic

Proficiency testing (PT) is a cornerstone of analytical quality assurance: it verifies laboratory competence, supports accreditation, and fosters comparability of measurement results across organizations. Understanding how PT providers derive assigned values and assess participant performance is essential for laboratories to interpret scores correctly, to identify measurement problems, and to use PT outcomes for continuous improvement and metrological traceability.

Objectives and overview of the leaflet

The document summarizes key concepts from ISO 13528 relevant to quantitative PT schemes and explains how PT performance is assessed. Its objectives are to clarify: how assigned values (x_pt) are selected, how the uncertainty of those values u(x_pt) is estimated, options for setting the standard deviation for proficiency assessment (s_pt), the different scoring metrics used to judge participant results, and the conventional interpretation of those scores.

Methodology and assignment strategies

The leaflet highlights that PT providers may determine the assigned value x_pt in several ways (ISO 13528 lists five common approaches). The chosen strategy must reflect the scheme’s objectives and the desired traceability. Typical approaches include using a certified reference value or deriving x_pt statistically from participant data (robust estimators, means, or medians).

The uncertainty of the assigned value, u(x_pt), must be estimated consistently with the chosen assignment method. ISO 13528 provides multiple methods to evaluate u(x_pt); this uncertainty can be reported as a standard uncertainty or as an expanded uncertainty U(x_pt)=k·u(x_pt) with a selected coverage factor (commonly k≈2 for ~95% confidence). The PT provider is responsible for documenting the chosen approach and its implications for interpretation of results.

Standard deviation for proficiency assessment (s_pt)

ISO 13528 suggests multiple options to set s_pt, and the provider must choose the one aligned with the scheme’s aims. Options include using an expert-specified target relative standard deviation, deriving s_pt from participant results, or basing it on historical or method-specific performance data. The choice of s_pt directly affects the z-score denominator and therefore the sensitivity of performance assessment.

Performance metrics and how they are calculated

The leaflet summarizes four main normalized performance indicators used in PT:

• D%: percent difference between the participant result x_i and x_pt, expressed relative to x_pt. The provider may specify a maximum permissible relative error (dE%).
• z-score: (x_i - x_pt) / s_pt. This unitless score compares deviation to the proficiency standard deviation. It is appropriate when u(x_pt) is negligible relative to s_pt.
• z-score: a modified z that incorporates the uncertainty of the assigned value in the denominator when u(x_pt) is not negligible (recommended when u(x_pt) > 0.3·s_pt).
• zeta (ζ) score: (x_i - x_pt) divided by the combined standard uncertainty sqrt[u(x_pt)^2 + u(x_i)^2]. This assesses agreement accounting for both the PT-assigned value uncertainty and the laboratory’s reported measurement uncertainty.
• En score: similar in concept to ζ but uses combined expanded uncertainties (approximately 95 % confidence interval). En is commonly used in metrology and calibration comparisons.

Interpretation of scores and conventional thresholds

The conventional interpretation from ISO 13528 is:

• |z|, |z|, |ζ| <= 2.0: satisfactory performance.
• 2.0 < |score| < 3.0: warning/questionable performance.
• |score| >= 3.0: unsatisfactory performance (action signal).

For D% and En the analogous acceptability criteria are |D%| < dE% and |En| < 1 respectively.

Main results and discussion (key messages)

The leaflet emphasizes several practical implications rather than presenting experimental data:

• The method used to assign x_pt and to estimate u(x_pt) materially affects participant scores and their interpretation; schemes that derive x_pt from participant data must consider bias and outliers and provide robust uncertainty estimates.
• If the uncertainty of the assigned value is non-negligible compared with s_pt, ignoring u(x_pt) can lead to misleading z-scores; z, ζ or En should be used to incorporate that contribution.
• zeta and En require that laboratories report realistic measurement uncertainties u(x_i) or expanded uncertainties; underestimation of these uncertainties will produce falsely unsatisfactory ζ or En results.
• The provider’s choice of s_pt controls the stringency of the assessment; transparent documentation of how s_pt was chosen is crucial for fair interpretation.

Benefits and practical use of the methods

• PT scoring provides objective, standardized indicators for interlaboratory comparability, regulatory compliance, and internal quality management.
• Different scores serve different purposes: D% is simple and useful when absolute tolerances are set; z-scores are effective for statistical interlaboratory comparisons; ζ and En connect performance assessment to reported measurement uncertainty and are therefore valuable in metrology and calibration contexts.
• When used correctly, PT results guide corrective actions, method validation/verification, estimation of measurement uncertainty, and identification of systematic errors or methodological drift.

Recommendations for laboratories participating in PT

1. Understand the scheme design: ask how x_pt and s_pt were obtained and whether u(x_pt) was evaluated.
2. Report realistic, well-justified measurement uncertainties u(x_i) when required, since ζ and En depend on them.
3. Use PT outcomes as triggers for investigation rather than sole proof of competence: apply root-cause analysis, repeat measurements, and review calibration and SOPs when scores indicate issues.
4. Maintain records of PT participation and trends (control charts) to detect long-term biases or variability changes.

Future trends and potential applications

• Greater emphasis on uncertainty-informed scores (ζ, En) as accreditation and metrology communities push for measurement-evidence-based assessments.
• Wider use of robust statistical estimators and advanced outlier detection when deriving x_pt from participant data to reduce bias from poor-performing laboratories.
• Development of digital reporting and meta-data standards so that scheme design choices (assignment method, uncertainty models) are machine-readable and reproducible.
• Integration of PT results with laboratory information management systems (LIMS) and quality dashboards for real-time monitoring of performance and trends.

Conclusion

Accurate interpretation of PT performance requires knowledge of how the assigned value and s_pt were established and whether the uncertainty of the assigned value has been accounted for. Choosing the appropriate score (D%, z, z, ζ, En) depends on scheme objectives and the relative magnitude of uncertainties. Transparent documentation by PT providers and realistic uncertainty reporting by laboratories are essential for fair and useful assessments that support continuous improvement and metrological traceability.

References

1. ISO 13528:2022 - Statistical methods for use in proficiency testing by interlaboratory comparison.
2. B. Brookman and I. Mann (eds.) Eurachem Guide: Selection, Use and Interpretation of Proficiency Testing (PT) Schemes (3rd ed. 2021).
3. Eurachem leaflet: How can proficiency testing help my laboratory.
4. Eurachem leaflet: Understanding PT statistics.