Statistical Methods for Hospital Monitoring with R

Preface

This book had its beginnings when, as director of an intensive care unit, I became interested in analysing acute care hospital outcome data and in trying to understand how such monitoring could prevent care from becoming unsafe. Attempts at data analysis were initially made using a spreadsheet. I retired from clinical practice in 1989 and for several years I worked part-time as a research associate at the then Australian Centre in Strategic Management. In 1994 Professor Michael Whitby invited me to work with him in his department, Infection Management Services at the Princess Alexandra Hospital. I then became acutely aware of the challenges presented by the clustering, high variability and sometimes autocorrelation among adverse event data, particularly but not only that which is due to hospital-acquired infection and colonisation.

Professor Kerrie Mengersen and I first became associated when a combined Queensland University of Technology and Princess Alexandra Hospital Australian Research Council Grant was received over a decade ago to study hospital infection adverse events. Since then we have been fortunate to be able to work on these and similar data. In 2011 Michael Whitby left Princess Alexandra Hospital to become Professor of Medicine at Greenslopes Hospital in Brisbane and Geoffrey Playford became his successor as Director of Infection Management Services at Princess Alexandra Hospital.

This book represents a journey of discovery involving infectious diseases physicians and other infection management staff at the Princess Alexandra Hospital, statisticians particularly at the Queensland University of Technology and the Queensland Health Centre for Health Related Infection Surveillance and Prevention (CHRISP), and others interested in what makes hospitals safer places. This journey continues. The work has been described in a series of journal articles that we acknowledge in the chapters and references. Our principal aim in this book is to demonstrate how the R statistical analysis software can be used to analyse these hospital adverse events (AEs). The data come predominantly from the hospital infection area but also from the wider hospital environment and we gratefully acknowledge the people who have made data available to us.

As we become more involved in attempting to understand what causes hospitals to be safe, we are increasingly reminded that the collection of data and their analysis, vital though it may be, does not alone make for a safe environment. We have to admit our ignorance and learn about safe systems, not just concentrate on outcomes that may be the result of unsafe practices; preventing errors should take precedence over counting them. Evans and her colleagues have recently discussed the role of epidemiologically sound clinically relevant data from clinical-quality registries as opposed to the current practice of monitoring numerous indicators. We have to understand that safety is impossible without discipline. We have to act on evidence and to work to improve that evidence. We must understand that AEs may be the result of the interaction of agents that may not appear individually to be of great concern. It is necessary to ensure that the complex system of interacting people, places and processes that make up a hospital is understood and made as safe as possible. The essential components of evidence-based systems to prevent AEs are being gathered together by organisations such as the Institute of Healthcare Improvement (IHI), for example as bundles in the IHI Improvement Map. In some cases a checklist may be valuable and the use of simulators may be able to detect and remedy unsatisfactory knowledge, skill, behaviour or judgement. Increasingly, engineering advancements that physically prevent specific AEs are becoming available.

There needs to be some practically workable surveillance mechanism to measure compliance with evidence-based systems and there must be some form of disciplinary action available to deal with people who persistently fail to implement them. Counting failures and comparing hospitals using conventional indicators do not do these things.

Analysis of aggregated indicator data usually involves displaying rates for individual hospitals together with prediction or precision limits relative to some average value for the group of hospitals to which they belong. The objective is to identify those hospitals with rates that differ from the average to the extent that predictable, mostly random, variation is an unlikely cause. As MohammedB and colleagues point out, this does not necessarily mean that they are unusually good or bad; it means that they may be atypical to the extent that further investigation, often of a non-statistical nature, is required. If it is found that substandard systems exist or individual performance is unsatisfactory, remedial action can then be taken. This process has nothing to do with comparing hospitals. In addition, it has its limitations. The average for the group of hospitals that may differ markedly in the services they provide increasingly involves some form of risk adjustment and for common count data AEs such as bacteraemias it is in the early stages of development. Within-hospital data typically display much less variability. Ideally, when outcomes are worrying it should be possible to identify this quickly, search for causes and institute remedial action. Sequential within-hospital methods of data analysis complemented by such audits as occur with properly performed mortality and morbidity (M&M) meetings, as described, for example, by Singer, frequently aid in such early detection. Data that are aggregated, for example by years, may hide runs of unsatisfactory outcomes or, if this is not the case, causes may be difficult or impossible to determine when considerable time has elapsed prior to reporting.

It is not uncommon to hear that we must collect data from hospitals so people can compare them and choose the safest. However, to make such comparisons, it would be important to adjust for different patient populations within the institutions and for multiple testing among institutions. Ignoring these adjustments may lead to quite erroneous comparisons and decisions. In addition, simple random variation may influence a hospital's ranking from year to year without any change in its systems. Finally, most people just want their local hospital to treat them courteously as individuals and to perform competently and safely. Implementation of appropriate systems based on evidence does this.

AEs in hospitals result in additional suffering and economic loss for their victims, as well as occasional potentially preventable death, and this burden may extend into the community if there is residual discomfort, delay in regaining function and continuing financial difficulty with reliance on family, community or government for support. Hospital-acquired infections are a major, but by no means the only, cause of hospital AEs. During the past 20 or so years, there has been an explosion in hospital complexity in the face of considerable financial constraint. The last thing any hospital needs in this situation is re-work, having to deal with complications, or to re-do surgery that has not resulted in satisfactory outcomes. When bed occupancy is high and beds are scarce, it is not good practice to have any of them occupied by patients recovering from potentially preventable complications. Yet Eshani and colleagues have reported that over 18% of hospital inpatient budgets can be consumed treating patients with adverse outcomes.

As we begin to understand the science of complexity, it has become apparent that making systems super-efficient may not be a good idea. Although efficiency may be desirable, complex system science has shown that with increasing efficiency there tends to be increasing fragility and instability (Cook and Rasmussen). For resilience, some redundancy is needed. If a ward is full, patients become outliers in other wards and staff must traverse extended networks to care for them. If hand hygiene is substandard and high-touch surfaces are imperfectly cleaned, transmission of hospital-acquired organisms will be enhanced. If the ward in question is the infectious diseases ward, it may become impossible to isolate patients who are carriers of potentially dangerous hospital-acquired organisms, thus resulting in increased transmission to other patients. It is well documented that access block in emergency departments is dangerous and that it impairs the functioning of ambulance services. A super-efficient hospital, the aim of current managerialist management practice, may in fact be involved in enough extra re-work to nullify any gains due to its super-efficiency.

AEs are not confined only to hospital-acquired infections. For example, delay in comprehending the seriousness of a patient's condition may result in the patient's potentially preventable collapse and admission to intensive care. Unconscious patients whose backs and airways are not managed effectively can end up with painful pressure ulcers or an inhalation injury. Stroke or dementia patients who are not managed optimally can fall and sustain injury. Errors with medications can harm patients. Harried staff are more likely to sustain a needlestick injury. These are but a small sample from the list of potentially preventable AEs.

There has, of course, been much work done to improve hospital safety. In particular, there are mandated systems of error reporting. These can be driven from within the hospital, with a view to improving its systems, or they can be externally driven for the purposes of independent monitoring and public reporting. However vital the latter may be for transparency and accountability, there is as yet little evidence that in general rates of potentially preventable AEs are declining, although obviously there are exceptions. The increasing implementation of simple evidence-based groups of interventions that have been placed in bundles, for example for minimising complications associated with the use of intravenous devices, appears to be a major improvement. However, strong leadership and discipline are needed to ensure they become habit, implementation must be sustained until this occurs and, when new evidence becomes available, modification must be possible. The increasing use of checklists and, for some specialties, the employment of simulators are positive steps towards achieving a safer environment. Better understanding of complexity and how agents interact will be a major step forward in the future. The idea of root-cause analysis seems inadequate when an AE may represent the emergent behaviour and self-organisation of an unsatisfactory complex system. The Pareto principle which decrees that most problems have few major causes must also be subjected to scrutiny. Excessive numbers of AEs may occur in complex systems because of the interaction of many agents that, when examined individually, may seem relatively innocuous.

At the level of data analysis, there are problems peculiar to hospitals. The most important methods involve the sequential analysis of AE data. This is often accomplished by adapting quality monitoring and improvement tools from manufacturing industry, where they have a long and successful history (Deming). However, this adaptation is not straightforward: in hospitals these time-series AE data frequently display clustering, high variability, marked skewness, many zero counts and occasionally autocorrelation. The transmission of an infectious agent is not an independent process. Patients vary in their susceptibility to the occurrence of an AE and risk adjustment, although useful, is by no means perfect. Length of stay in hospital contributes to occupied bed-days, a frequently employed denominator and it is also frequently a risk factor. Although these issues also arise in manufacturing processes, they need to be carefully understood and accounted for in this new setting of hospital monitoring. Risk adjustment is available for many binary AEs such as complications of surgical procedures and, although very useful, it is not infallible. With count data AEs, where risk adjustment usually involves the hospital's services rather than individual patient characteristics, it is in the early stages of development. When patients with differing potential to be harmed are in the wards of a hospital, such grouping is often not random, and this can result in AE variation that is sufficiently excessive to invalidate elementary approaches.

Control charts are the most frequently employed quality monitoring tools. These rely on there being some sort of expected value, perhaps a mean value during a known period of stability, and some measure of variability, that is, there are available data that behave predictably and that enable location and spread to be estimated. It is then possible to determine whether or not this month's result is within those predictable limits. If it is not, an audit can be performed to search for causes. A problem with some hospital data is that such expected values may not exist. For example, how can it be possible to determine the expected usage of an antibiotic or the expected prevalence or burden of colonisation with an organism each month? When a quality improvement activity is instituted, the count of AEs may actually increase due to better reporting. Another difficulty arises with some highly skewed data like ICU length of stay that appear, at least approximately, to sometimes follow fractal (power law) distributions and to have no mean value in the usual sense. When this is the case, using familiar control charts can give meaningless misinformation. In addition, we once again mention the limitations of finding obvious causes for apparent signals in the control charts when so many trends and other changes represent emergent behaviour.

This book has been written in light of the above considerations. The aim is to provide statistical tools for monitoring the type of processes that are typically encountered in a hospital context. We devote much attention to addressing the difficulties described above, but we acknowledge that our approaches can certainly be improved. We hope that our book will stimulate interest in improving the methods that are currently available. During the past decade we have published several papers to illustrate how available methods can be adapted to monitor hospital AE data, especially AEs due to hospital-acquired organisms. Spreadsheet software is no longer adequate for analysing hospital AE data and in these notes we endeavour to show how the R statistical analysis system can be employed to undertake these analyses.

Thus we aim to introduce hospital scientists to the statistical software program R for the analysis of their data. Since specialised statistical software can require considerable training and continuity of use, some method has to be found to make software like R useful to hospital scientists who increasingly have postgraduate degrees but often little training in statistics, or continuity of use of statistical software. We have endeavoured to address these difficulties by providing simple to use functions for getting data into and out of R in a standard format and for analysing those data. The earlier parts of Chapters 1 and 4 should be of use to hospital scientists who often have to analyse proportions and rates. Chapters 3 and 6 and parts of Chapter 7 deal with control charts and charts based on generalised additive models. In each case, a menu is provided. Chapters 2 and 5 deal with aggregated data and will be of less interest to hospital scientists. However, we feel that these chapters are necessary for continuity and that they may prove to be of interest to staff whose job it is to present analysed data from groups of hospitals, or units within them that perform similar functions. The Introduction describes how hospital scientists might use R. Chapter 8 is a summary of the very important non-statistical aspects of hospital safety.

In conclusion, it is of interest to speculate on what future developments will occur in this area.

New insights into effective monitoring tools and approaches are being developed by hospital-based researchers such as Mr Ian Smith at St Andrews Hospital and Medical Institute (Smith and colleagues). Of particular importance is their experience regarding implementation and uptake of these approaches. As more AE data collection is mandated by central authorities and punitive action like withholding funding or administering fines may be involved, it is very important that the corresponding statistics are presented fairly. Good performance is dependent on justice, learning and discipline and without justice there cannot be the trust that is vital for high levels of performance. Systems of reward and punishment that are based on random variation in the data, or inappropriate statistical analysis or interpretation, or that involve AEs that occur as a result of high-level decisions or that seek scapegoats, destroy trust and morale and guarantee mediocre performance.

Risk adjustment is another important area that, while relatively well developed, still needs statistical attention. This is particularly true for count data adverse events (AEs) where risk adjustment is still undergoing development. Edward Tong's bacteraemia study was the first we were involved with and the work is being continued by Ms Mohana Rajmokan MSc (Biostats), statistician at the Queensland Health Centre for Health Related Infection Surveillance and Prevention (CHRISP) who has recently applied Tong's methods for risk adjusting hospital antibiotic usage.

As hospital complexity continues to increase in tandem with financial restraint necessitated by increasing population, depletion of resources, environmental degradation and deteriorating climate, change in many areas will become necessary. Understanding hospitals as complex systems within a complex environment will assume ever increasing importance. Our early work on this, employing a Bayesian network, involved Dr Mary Waterhouse (Waterhouse and colleagues) with St Andrews Medical Institute and the Wesley Research Institute in Brisbane and has again been continued and extended by Ms Rajmokan at CHRISP.

Hospitals vary greatly in the specialist services they provide and this can have a large effect on expected rates. For example, one hospital may have a large haematology/oncology or renal dialysis unit and another may have a large maternity unit. Bacteraemias are uncommon in maternity units. A major difficulty for us has been the limited number of hospitals available to us, for example Queensland has one spinal injuries unit so the possible effect of such units on bacteraemia rates or antibiotic use cannot easily be established. In addition, many smaller hospitals have few bacteraemias due, for example, to Staphylococcus aureus, so they are of little use in determining which services are associated with these infections. It would seem to be sensible to omit hospitals reporting only a few AEs each year and require them to perform M&M audits when those AEs occur. It would also seem desirable for a standardised evidence-based bundle to be devised for performing an M&M audit, such as that described by Singer. Another area of potential interest is the development of charts for composite measures. For example, track, trigger and report (TTR) systems (Mitchell and colleagues) for the early detection of acute deterioration may benefit from the development of a composite measure that encompasses changes in blood pressure, pulse rate, temperature, respiratory rate, oxygenation and level of consciousness. These are but a few of the areas where future developments could be beneficial.

We wish to acknowledge the valued collaboration of Dr David Cook, Dr David Looke, Dr Margaret Lindsay, Ms Mohana Rajmokan, Professor Tony Pettitt and Dr Anna Barker. We must also acknowledge the selfless work performed by those who developed and sustain R. We are indeed fortunate to be able to take advantage of their work. We also include a list of the libraries we have employed and thank their authors. R can be a wonderful gift to those of us who work with clinical people to make hospitals safer places.

When R starts it indicates that R is free software and comes with no warranty. The R scripts and functions to accompany this book that are available on the internet are similarly free software and come with no warranty (www.wiley.com/go/hospital_monitoring). Our primary aim is to show how a complex statistical analysis package can be used by hospital scientists who are not statisticians and who often have to deal with atypical data and we hope that others will build on these notes, scripts and functions. We recommend that before our functions are implemented users seek specialist advice. Hospital adverse event data from other hospital systems may have characteristics that we have not encountered. Inevitably, in spite of our best efforts, errors will be found and we ask that they be brought to our attention so that corrections can be made available via the internet. Further aims are to suggest possible solutions to problems such as the analysis of very low rate adverse events and data that lack predictable measures of location and spread. We hope that others will investigate and build on these proposals.

Anthony Morton, Tarragindi, February 2013

Introduction

0.1 Overview and rationale for this book

0.1.1 Motivation for the book

Quality management (QM) and quality improvement (QI) are fundamental tenets of almost every business, industry, government and institutional enterprise. Established in the 1920s and popularised by Shewhart, Deming and Juran, among others, QM is now recognised as aiming to produce a consistent outcome of interest, whereas QI seeks to improve the consistency and/or quality of the outcome. This often involves changing the physical, technical, environmental or human factors or processes that generate, or impact on, the desired quality and consistency. The importance and pervasiveness of QI are illustrated by the many QI standards that have been established across the world that include ISO guidelines, the Japanese Kaizen program, Six Sigma, Taguchi methods, and others (Wadsworth and colleagues). In the hospital setting, QM and QI mean achieving safety primarily by the disciplined application of systems based on evidence.

One of the more recent applications of QI is in the improvement of hospital acquired infection (HAI) rates and other adverse events that occur in hospitals such as patient falls, pressure ulcers, medication errors, needlestick injuries, complications of treatment and mortality. HAIs are a major contributor to hospital adverse events (AEs): approximately one in every 20 hospitalised patients will contract an HAI (US Centers for Disease Control www.cdc.gov/hai/). They can hinder patient recovery, increase hospital stay, increase costs and can lead to death. The cost of HAIs is considerable: in the United States the annual overall direct cost to hospitals is in the order of 30 billion dollars. In that country in 2002 the estimated number of HAIs was 1.7 million with 99, 000 deaths. GravesA and colleagues have predicted that in Australia there are approximately 175, 000 cases annually resulting in an additional 850, 000 bed-days. In another paper, GravesB and colleagues express caution concerning the estimation of costs. Nevertheless, reducing HAIs would clearly make available badly needed hospital beds.

Effective infection management (IM) includes understanding the factors involved in the acquisition and spread of the organisms responsible for HAIs, and implementation of systems for managing and reducing infections in hospitals. This requires a strong evidence base, which in turn requires the collection of epidemiologically sound and clinically relevant data (Evans and colleagues), and employment of appropriate statistical methods for their analysis.

These ideas apply equally to other AEs such as patient falls, medication errors and pressure ulcers. An example is the reduction in patient falls resulting in injury following the determined implementation of a suitable evidence-based system (Barker and colleagues).

In the past, spreadsheets have been used regularly for statistical analysis in QI in general, and for IM in hospital departments in particular. This practice can be constraining and possibly misleading due to the limited statistical capabilities of the spreadsheet software, particularly for typical HAI data which can be characterised by small rates, a large number of zeros, and variation within and between hospital units and patient cohorts. This excessive variation occurs because infections are not biologically independent. Frequently, a stable average value is lacking, for example, in antibiotic usage and some prevalence and other data, they may be correlated over time and they are sometimes highly skewed. This means that the well-known statistical approaches, for example simple normal approximation methods for calculating confidence intervals and performing significance tests for proportion and rates may not be appropriate.

A major focus of the book is to provide a suite of methods that we have found to be useful in analysing these types of data. Moreover, we provide the software tools to implement these methods using the freely available statistical package, R. Although the approaches presented in the book are mainly focused on IM, the methods and software tools may be generally applicable across a range of situations for which QI in hospitals is a focus.

0.1.2 Why R?

We choose the software package R (R Core Team (2012). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-project.org/.) because it is freely available and includes very powerful methods such as those based on the beta and gamma distributions. Wherever possible, we employ the functions already available in R and its libraries. We acknowledge that it takes an investment of time to learn R, but we suggest to the reader that this investment is worthwhile. In order to facilitate the learning and implementation of this package, we have developed two menus (IMenu() and CCMenu(), in rprogs.RData) that enable the selection of frequently required methods for the AEs of interest. Each function in these menus is accompanied by a script available in the accompanying code file on the internet that includes the corresponding lines of R code. The functions in the menus can be modified if required for access to other R functions if different analyses are of interest. There is a short Appendix entitled Menus that explains the use of these menus.

0.1.3 Other reading for R

There is excellent R documentation available from the R website http://cran.r-project.org/. For example there are R for Beginners by Emmanuel Paradis and Simple R by John Verzani, both available at cran. Other documentation that may be of interest includes Analysis of Epidemiological Data Using R and Epicalc by Virasakdi Chongsuvivatwong, Statistics Using R with Biological Examples by Kim Seefeld and Ernst Linder, An Introduction to R: Software for Statistical Modelling and Computing by Petra Kuhnert and Bill Venables, all available at cran. There is a useful download by Lawrence Joseph available at www.medicine.mcgill.ca/epidemiology/joseph/EPIB-621.html together with a downloadable introductory statistics textbook http://www.medicine.mcgill.ca/epidemiology/Joseph/courses/EPIB-621/rosenberg.joseph.barkun.pdf).

Excellent introductory books are available such as Dalgaard's Introductory Statistics with R 2nd Edition (2008), Using R for Introductory Statistics by Verzani (2004), Statistics, An Introduction Using R by Crawley (2005), The R Book also by Crawley (2007), A Handbook of Statistical Analysis Using R 2nd Edition by Everitt and Hothorn (2010), and Data Analysis and Graphics Using R by Maindonald and Braun 3rd Edition (2010). Books at a higher level include Statistical Computing by Crawley (2003), Modern Applied Statistics with S by Venables and Ripley (2002) and Data Analysis Using Regression and Multilevel/Hierarchical Models (Gelman and Hill 2007). Myatt's Open Source Solutions–R (2005) explains the use of R for analysing epidemiological data including advice on the use of logistic regression for analysing outbreaks and epidemics (www.brixtonhealth.com). The R meta, rmeta, Epi, epicalc, epibasix, epiR, epitools and other packages mentioned in these notes have functions that may prove useful to hospital epidemiologists and scientists in IM and QI departments.

Other possibly useful programs are Abramson's WINPEPI (www.brixtonhealth.com/), CIA (Altman, Machin, Bryant and Gardner 2000), EpiInfo 3.5.4 (Dean, Arner, Sunki, Friedman, Lantinga, Sangam, Zubieta, Sullivan, Brendel, Gao, Fontaine, Shu, Fuller, Smith, Nitschke and Fagan. Epi Info™, a database and statistics program for public health professionals. Centers for Disease Control and Prevention, Atlanta, Georgia, USA, 2007. www.cdc.gov/epiinfo/) and EpiData (www.epidata.dk/).

Several R libraries are required in addition to those in