6. FRAMEWORK

One of the objectives of the project was to develop novel protocols to use ongoing EQA results to monitor performance.

Before this project was commenced, RCPA QAP reviewed how performance of EQA was monitored in other jurisdictions. The framework subsequently developed, was modelled on the UKNEQAS approach and adapted for the Australian environment. Extensive consultation was undertaken with Managers and Chairs of programs, with participants throughout Australia and with Fellows and Trainees of the RCPA. The framework will enable the RCPA QAP to offer assistance to participants to enable early corrective action, in the first instance.

The framework has three stages. At each stage a laboratory must meet one of the benchmarks (fall within one of the criteria for unacceptable performance) in order to trigger for a letter of notification to a laboratory.

Performance Monitoring Process for Participants (Figure 3)

1. Enrolment process

At the end of every year, participants enrol for the following year. Each participant is given a unique identifying code. When enrolment packages are sent out, participants will be sent a copy of the new framework outlining the process when participants’ results are identified as performing within the criteria for unacceptable performance. As part of the enrolment process, the performance monitoring project will be outlined in the general information with a statement explaining that enrolment in the RCPA QAP acknowledges that performance monitoring of EQA results is now part of RCPA QAP processes. The enrolment process captures participants enrolling for education only, specialisation of Pathologists’ for Anatomical Pathology diagnostic modules and contact details of supervisors including Director of Pathology.

2. EQA results fall within the criteria for unacceptable performance the first time

When unacceptable performance has been identified, according to the criteria developed for each module, letters will be sent to the participant and supervisors detailed on the contact page at enrolment. The letter offers assistance to review the QAP results.

Results within the criteria for ‘participation indicators’

• The QA Program Manager will send a letter of notification.

Results within the criteria for unacceptable performance for ‘test performance indicators’

• The EQA results are reviewed by the Program Performance Review Committee. The experts on each committee will review all EQA results for that module in their context.
• A letter of notification is sent to those participants / supervisor(s) whose results are considered to fall within unacceptable performance. 29
• A report prepared by the Program Performance Review Committee will also be sent to the participant and supervisor(s) and is available to the RCPA QAP Board and NATA on request.

Potentially critical situations as defined by the performance criteria

• When the Committee have greater concerns regarding performance of a participant, the EQA report, letter and Committee report are referred to NATA.
• Participants falling within the criteria for unacceptable performance will be monitored for one year, to review the progress of their EQA results.

3. EQA results fall within the criteria for unacceptable performance a second time

Those participants whose results fall within the criteria for unacceptable performance a second time will be escalated to the next level of action by the RCPA QAP.

All results within the criteria are reviewed by the Program Performance Review Committee.

• The experts on the committee will consider all EQA results for that module in their context.
• A letter of notification, from the Program Performance Review Committee Chairperson, is sent to those participants / supervisor(s), detailed on the contact page at enrolment, whose results are considered to fall within unacceptable performance.
• A report prepared by the Program Performance Review Committee will be sent to the participant and supervisor(s) and is available to the RCPA QAP Board on request.
• Where appropriate, the Program Manager may also phone the participant to offer assistance to review any problems.
• Results of participants falling within the criteria for unacceptable performance will be monitored for one year, to review the progress of their EQA results.
• A copy of the letter sent to the participant, EQA report(s) and report from the Program Performance Review Committee will be sent to NATA for their consideration and follow‐up directly with participants.

4. EQA results fall within the criteria for unacceptable performance a third or subsequent time

• After results have been reviewed by the Program Performance Review Committee, the Board of RCPA QAP will formally review all results.
• EQA results from other relevant Quality Assurance Programs will also be reviewed by the Board at this time to ascertain if there is unacceptable performance in other testing or if what has been reported is isolated to one discipline or module.
• The findings of the RCPA Board will be communicated to the Director of Pathology of the organisation concerned.
• Copies of all letters and Committee reports will also be sent to the participant and supervisors detailed on the contact page at enrolment.
• In addition a copy of the letter sent to the participant, supervisors and Director of Pathology, EQA report(s) and report from the Program Performance Review Committee(s) will be sent to NATA for their consideration and follow‐up with the participant.