SECTION I: EXECUTIVE SUMMARY

Project aim

This project aimed to assess the impact of electronic ordering systems, on the quality use of pathology services across six hospital sites and different pathology departments, for the following areas:

• the legibility and completeness of laboratory test orders and the impact on Central Specimen Reception work processes (Quality of test orders).
• the volume and mix of tests ordered examined by such factors as Diagnosis-related Groups (DRGs), adjusted for clinical activity where appropriate, and the prevalence of add-on and repeat testing (Effectiveness).
• the timeliness of the pathology laboratory process (Turnaround time).
• the impact of pathology performance (e.g., laboratory test turnaround times) on the duration of patient stay in the emergency department (Patient outcome).

The project also produced a benefits realisation framework, made up of performance indicators, that can be used to guide the assessment of electronic ordering in a pathology service and to monitor what works (or doesn’t work), where, and in what circumstances. The project was funded by an Australian Government Department of Health and Ageing, Quality Use of Pathology Program grant.

Project setting

An electronic medical record (EMR) system utilising Cerner PowerChart became available at Hospitals A, B, and C, on 26 October 2009; Hospital D on 29 June 2009, Hospital E on 1 October 2008, and Hospital F on 9 March 2009. The EMR allowed the clinicians to create electronic orders. In 2011, electronic ordering was used for approximately 66% of pathology test orders across the six hospitals.

Laboratory test order errors

Electronic ordering systems (referred to as EMR in the settings involved in this study) are expected to eliminate legibility problems in handwritten orders and to reduce errors, particularly during the pre-analytic phase involving patient identification and specimen collection and labelling. They are also able to contribute to improvements in the quality of the information provided to the laboratory, thus increasing efficiency and effectiveness in the laboratory.

A longitudinal analysis of laboratory errors including a period after the implementation of the EMR showed an increase in the number of errors, both as raw frequency and as a rate per 1000 test order episodes. This increase was accounted for by the introduction of a new class of errors associated with the EMR and the processes surrounding its use. A cross-sectional analysis, comparing the error rate for EMR orders with that for paper orders (for the same period of time), indicated that the overall error rate for many categories of error was lower for EMR orders than for paper orders. Critically, this pattern was consistent for all three Incident Information Management System (IIMS) categories of errors that relate particularly to patient safety issues.

Test volume

A series of analyses across the six hospital sites was undertaken to compare test volumes and aspects of the effectiveness of the test order process. A comparison of the rates before (2008) and after (2011) the implementation of the EMR, indicated that the mean number of tests ordered in each test order episode decreased significantly at each of the hospitals. Taken for all hospitals, the mean number of tests for each episode fell from 4.63 in 2008 to 4.36 in 2011.

Diagnosis-related Group casemix

Our comparison of the number of tests undertaken per admission and grouped in DRG categories provided examples such as A06B (Tracheostomy w/ventilation >95hrs) where the mean number of tests per admission fell from 181.10 in 2008 to 156.77 in 2011, but where the corresponding mean length of stay rose from 646 hours to 696 hours. Alternatively, for E62A (Respiratory infections) the numbers were 40.60 to 42.81 for mean number of tests and 305 to 289 hours for mean length of stay. The use of DRGs also provided a valuable means to examine test ordering patterns across hospitals. Our analysis of the test ordering profiles for F74Z (Chest pain) at four hospital emergency departments (EDs) highlighted similar test ordering patterns (e.g., Troponin, EUC, and Automated Differential tests were consistently the most frequently ordered tests). There were some differences in test ordering profiles, especially for the lower volume tests, between hospitals compared across the pre- and post-EMR periods. The mean number of C-Reactive protein tests per ED presentation varied both between hospitals and between years. At three of the EDs the mean number of C-Reactive protein tests per ED presentation was higher in 2011 than in 2008.

Add-on testing

Add-on tests are test assays that are performed on an existing specimen within the pathology service. The reasons for ordering an add-on test may include; requiring a base-line test result in cases where treatment has already commenced, the ordering clinician neglecting to order all relevant tests in the first instance, or simply to avoid subjecting certain vulnerable patients to additional phlebotomies. Add-on tests are labour-intensive and disruptive and place a disproportionate burden on laboratory resources. The add-on rates between hospitals ranged from 0.61% (Hospital B; specialist hospital) to 2.24% (Hospital F; metropolitan general hospital). The clinical chemistry and haematology departments, combined, accounted for 70% of all add-on test volume. In the clinical chemistry and haematology departments, add-on tests accounted for 2.56% and 0.69%, respectively, of all ordered tests.

Repeat tests

We compared the rate of paper and EMR-ordered EUC tests which were ordered within one hour and 24-hours of the previous EUC test. In 2011, the overall proportion of repeat EUC testing occurring within one hour of the previous EUC test was significantly greater for paper tests than EMR tests (0.69% and 0.25%, respectively). While, for tests ordered within 24 hours, there was a significantly lower proportion of repeat tests with paper orders than for EMR orders (11.68% and 34.04%, respectively).

Test turnaround time

Laboratory turnaround time (TAT) is the time taken by the laboratory to complete the entire testing process (from when the specimen arrives in the CSR to when a result is available to the clinician). TAT is often used as a key performance indicator of laboratory performance. Our analyses showed that the median data entry time (the time from when the specimen arrives in the CSR until the order is entered into the Laboratory Information System), for all hospitals combined, was three minutes shorter for EMR than paper. This difference was consistent and significant for both EUC and Automated Differential in 2010 and 2011. These decreases contributed to significantly lower median Total Laboratory TATs for EMR orders than for paper orders (for EUC tests, the difference in medians was 12 minutes in 2010 and six minutes in 2011; for Automated Differential tests, the difference in medians was four minutes in 2010 and two minutes in 2011).

Patient outcomes – Emergency Department length of stay

This project used multilevel linear regression modelling to examine the relationship between length of stay (LOS) in the ED along with pathology testing characteristics such as TAT and the volume of tests. The final model, accounting for 24% of the variation in ED LOS, showed that after controlling for the effect of patient age, triage category, number of tests in the test order episode, and ED mode of separation, the ED LOS on average, increased by 9.8% for every 60 minutes increase in the test turnaround time.

Benefits realisation framework

The evidence provided by this research (as summarised previously) has highlighted the value of a set of key performance indicators that can be used to measure major features of electronic ordering and its effect on the laboratory processes (predominantly the pre-analytical processes). These indicators can be used for comparisons between hospitals, wards etc., to help monitor and improve the overall safety of patient care, efficiency in the wards, and to help enhance the quality of pathology provided.

In this project, the utilisation of these indicators provided valuable empirical information about the EMR and its impact on pathology services and clinical work processes. Within the CSR they revealed the impact of errors associated with the introduction of the EMR but also showed how the EMR-ordering was associated with significantly fewer IIMS-related errors when compared with paper orders. The introduction of EMR was connected to a significant decrease in the mean number of tests for each test order episode across each hospital when compared before and after EMR implementation. This project used DRG categories to compare the number of tests per admission and to examine test ordering patterns across hospitals. Add-on test rates were investigated between hospital departments to provide benchmarks for future analyses. The analysis of repeat tests for EMR-ordered EUC tests showed that the overall proportion of repeat EUC tests which occurred within one hour of the previous EUC test was significantly lower for EMR than for paper orders. The project identified a significant decrease, for all hospitals, in the median time taken from specimen arrival in the CSR to the time the order was entered in the Laboratory Information System. This decrease contributed to the significantly lower median laboratory TAT measured from the time a specimen arrived at CSR to the time a result was available to the clinician. The project’s multi-level linear regression modelling examined the relationship between LOS in the ED along with pathology testing characteristics such as TAT and the volume of tests, and produced a model that accounted for 24% of ED LOS variation.