TMJ 2009 0169
TMJ 2009 0169
Michelle Glaser, M.P.H.,1 Tom Winchell, M.P.A.,2 technology was adequate to conduct visit (93.4%), and a follow-up
Patty Plant, B.S.N., N.P., M.S.N.,3 Wayne Wilbright, M.D., M.S.,3 telemedicine appointment was requested (90.8%). Most providers
Michael Kaiser, M.D.,3 Michael K. Butler, M.D., M.H.A., C.P.E.,3 were satisfied with telemedicine for the visit overall (87.0%), be-
Matthew Goldshore, M.P.H.,1 and Manya Magnus, Ph.D., M.P.H.1 lieved that telemedicine improved patient prognosis (88.2%), and
perceived that the patient was satisfied (83.0%). This study suggests
1
Department of Epidemiology and Biostatistics, The George that telemedicine was an effective and accepted method of healthcare
Washington University School of Public Health and Health provision.
Services, Washington, DC.
2
LSU Health Sciences Center, New Orleans, Louisiana. Key words: telemedicine, health information technology, outcomes
3
LSU Health Care Services Division, Baton Rouge, Louisiana. evaluation
Introduction
T
Abstract elemedicine represents a unique opportunity to enhance
Health information technology including telemedicine offers poten- healthcare delivery in situations where distance or travel
tial to improve patient care outcomes. As part of the response to challenges present barriers to specialty or other care. In-
Hurricanes Katrina and Rita in 2005, the Louisiana State University carcerated individuals represent a specific population where
Health Care Services Division expanded its statewide telemedicine there may be benefits of telemedicine are accentuated; benefits may
program. The aim of this study was to evaluate provider satisfaction result from the elimination of need to travel great distances with
and patient outcomes associated with telemedicine when used for the security measures in place, halt regular clinics to accommodate
administration of prisoner medical care. Providers completed a prison patients, or provide specialty services to prisoners that would
survey following each patient encounter in real-time; questions were otherwise be unavailable.1–4 Between transportation and extra
adapted from standard satisfaction indices. Statistical methods in- guards, bringing a prisoner to a doctor’s appointment can be ex-
cluded uni-, bi-, and multivariable including ordinal regression pensive and has potential to be dangerous. Telemedicine offers a way
methods to characterize unadjusted and adjusted factors associated to decrease medical costs and improve care among incarcerated
with telemedicine use and provider satisfaction, and patient out- populations.1–4
comes. Data were collected between December 2007 and May 2008 The field of telemedicine, also known as telehealth, ranges from
and were analyzed using SAS and Stata. Out of 737 patient visits, systematic phone contact between providers and patients for mental
the majority of patients were African American (68.6%), men healthcare provision, to complex video visits with technological at-
(92.9%), seen for either infectious disease or mental health (46.2% tachments to provide clinical assessments such as electrocardiogram
and 50.2%), with most surveys completed by a physician (63.1%). (ECG) monitoring, otoscopes, dermatologic viewing, and more.5–7
Most telemedicine encounters were completed (92.8%), a treatment Uses of telemedicine include connecting prison populations to care as
plan was established (97.0%), the provider perceived that the well as facilitating contact between providers and patients in rural
areas where travel is a barrier to care. Multiple evaluations and re- lenges in obtaining real-time or near-real-time evaluation data re-
views have been conducted in the use of telemedicine for provision garding provider perceptions of the system, including information on
of mental healthcare,1,2,8–15 prison care,16 HIV=AIDS care,17–24 and satisfaction and patient outcomes, have been limited by poor re-
dermatology,25–28 among other disciplines. These consistently suggest sponse rates of providers or lengthy delays between point of service
an increasing acceptance of telemedicine as well as efficacy of the and data collection. For example, rural Canada32 and Alaska33 have
technology toward patient care improvements. Adolescent mental telemedicine programs that reach citizens who are too remote from
healthcare in detention centers and other settings has been shown to be doctors and hospitals to receive medical treatment; however, the
particularly responsive to the use of telemedicine technology.2,13,15 system has not yet published an extensive outcomes-based evalua-
In response to Hurricanes Katrina and Rita in 2005, Louisiana State tion of its services. The Alaska Federal Health Care Access Network
University (LSU) Health Care Services Division enhanced and ex- system has one of the country’s oldest telemedicine systems, which
panded its telemedicine program to facilitate continuity of care be- was designed to overcome physical geographic barriers to care.33
tween clinics as well as respond to the unique needs of prisoner Despite the system’s ability to generate system-based process eval-
health. Over 48 video endpoints in 8 public medical centers and 13 uation data, satisfaction and outcomes data were collected by paper-
prisons were created.17,29 The initial set of services included based survey, resulting in a response rate of just 40%.33
HIV=AIDS care, mental health including juvenile services from de- Additional studies are needed to track real-time provider percep-
tention centers, otolaryngology (ear, nose, and throat), dermatology, tion of prison system use of telemedicine and provider satisfaction
and neurology. At each prison telemedicine unit, self-contained carts with telemedicine, in conjunction with patient outcomes. Provider
were provided to link the prisoner patient with the provider. The satisfaction with health information technology including tele-
equipment includes a primary video conferencing system (Polycom), medicine is a central component needed for adoption and im-
with a secondary camera (AMD) used for dermatology and wound plementation of any health information technology (HIT) to be
care appointments as either a hand held or tripod-mounted lens. A successful.34–39
video monitor=television screen, with audio capabilities, is used for The aim of this study was to evaluate provider satisfaction and
all clinical, administrative, and educational procedures that are part patient care outcomes associated with use of telemedicine in
of the telemedicine encounter. An AMD=Welch Allyn otoscope and Louisiana prison clinics, using a method that captures real-time,
laryngoscope is used for the ear, nose, and throat appointments and electronically collected data.17,29
the AMD 3700 Telephonic Stethoscope is used for the cardiology
appointments. Additionally, there is a document camera capable of Materials and Methods
capturing, analog to digital conversion, and transmitting clinical In concert with a multidisciplinary team and based on a review of
images, radiographs, ECGs, and videos where digital copies do not the existing literature, an electronic survey was developed that would
exist. be automatically presented to providers at the closure of each tele-
In conjunction with the initial rollout in the prison health centers, medicine visit. The survey contained five Likert questions regarding
an electronic provider survey was developed to capture the provider the provider’s satisfaction with the telemedicine visit, perception of
experiences. The aim of the survey was to obtain real-time satisfac- prognosis, and patient satisfaction with telemedicine, as well as six
tion and outcomes data directly from providers at the time of tele- questions regarding patient outcome and disposition; questions are
medical care. Although multiple evaluations have supported displayed in Table 1. The questions were adapted from other evalu-
telemedicine’s association with patient and provider satisfaction and ations and literature, publicly available forms, reports, and publi-
positive care outcomes in prison settings and the general popula- cations.32,33,40–43 Data obtained from all visits prison and nonprison
tion,1,5,7,8,10,11,13,15,16,20,28,30,31 there remain gaps in telemedicine between December 2007 and May 2008 were analyzed.
evaluation research, many stemming from difficulties in adequate
response rate and reaching providers to gather data as well as ANALYSIS
sometimes minimal dissemination of results. Univariate and bivariate methods were used to characterize pro-
In a systematic review of the literature, one study found that be- vider satisfaction and perception of telemedicine at each specific
tween 1966 and 2005, 47 articles contained information on a com- visit, and its association with patient disposition and visit outcomes.
plete technical evaluation of a telemedicine program and that only 3 Responses on the questionnaire were first assessed for global asso-
of those used objective methods to complete the evaluation.5 Chal- ciations with characteristics of the visit (location, clinic, and purpose)
Patient no show
Clinician unavailable
What information was not available but was needed to either make a diagnosis
and=or treat the patient?
What is the patient’s disposition? Inappropriate consultation—requires additional consult by referring clinician
Other
Based on your perception about today’s telemedicine visit, clinical decision making Completely agree=agree=neutral=disagree=completely disagree
was successfully accomplished?
Based on your perception, today’s telemedicine visit may have improved the
patient’s prognosis?
Based on your perception about today’s telemedicine visit, how satisfied are you Completely satisfied=generally satisfied=neutral=generally dissatisfied=completely
today’s telemedicine outcome? dissatisfied
What is your perception about overall patient satisfaction with today’s telemedicine
visit?
What is your perception about overall patient satisfaction with today’s telemedicine
visit?
and patients (age, race, and sex). Those found significantly associated All survey protocols and instrumentation were approved by the LSU
were then assessed to ensure assumptions underlying ordinal re- Health Sciences Center and George Washington University Medical
gression were met; if they were, ordinal regression was used to model Center Institutional Review Boards.
characteristics associated with agreement to the Likert statements.
Logistic regression was used to model characteristics associated with Results
completion of the visit, perception that technology was adequate for As shown in Table 2, the majority of patients were men (92.9%),
the visit, and whether a treatment plan was established. For all re- African American (68.6%), and over 18 years of age at the time of the
search questions, a was set to 0.05. Stata 9.0se was used for analysis. visit (54.1%). Visits were nearly evenly split between mental health=
more likely to have providers perceive that the technology was not
Table 3. Provider Perception of Telemedicine and Patient
Outcomes, December 2007 to May 2008 (n ¼ 737) adequate for the visit at hand. This may reflect the difficulties in
treating this adolescent population in juvenile detention settings
N (%)
more than the telemedicine itself, but suggests that the technology
Based on your perception, today’s telemedicine visit may
should be carefully monitored for satisfactory outcomes among this
have improved the patient’s prognosis
Completely agree 279 (37.9) population. This is contrary to the findings of other investiga-
Agree 371 (50.3) tors2,13,15 who found telemedicine to be successful with adolescents,
Neutral 71 (9.6) including those in detention centers. Although women composed a
Disagree 3 (0.4) very small proportion of the sample, our measures suggest that the
Completely disagree 13 (1.8)
providers perceived the use of telemedicine to be more satisfactory
Based on your perception about today’s telemedicine visit, and with less satisfactory patient outcomes than that for their male
clinical decision making was successfully accomplished counterparts. Future studies are needed among samples with more
Completely agree 283 (38.8)
women to examine this more fully.
Agree 369 (50.6)
Neutral 62 (8.5) This study has several strengths and limitations. The survey was
Disagree 2 (0.3) inserted into the telemedicine electronic interface that sets ap-
Completely disagree 13 (1.8) pointments and contains patient data as a part of the electronic
Based on your perception about today’s telemedicine visit,
close-out procedures, which ensured maximum participation
how satisfied are you with today’s telemedicine outcome? among the providers; this increases the generalizability of the
Completely satisfied 281 (38.1) findings to the population of providers using the telemedicine
Generally satisfied 335 (45.5) program. The survey was developed by a multidisciplinary team
Neutral 65 (8.8) and was evaluated by sentinel providers before launch, to ensure
Generally unsatisfied 42 (5.7)
Completely unsatisfied 14 (1.9)
acceptability of the tool. During that period, however, the length of
the survey was found to be too long and the final survey omits
Based on your perception, what is your overall satisfaction several questions that could have lent context or depth to the re-
with the telemedicine system for this telemedicine visit?
sponses. Other than location, type of clinic, and the type of provider
Completely satisfied 283 (38.4)
Generally satisfied 323 (43.8) closing out the survey, few characteristics are available regarding
Neutral 73 (9.9) the providers. Future studies may examine whether provider char-
Generally unsatisfied 44 (6.0) acteristics such as comfort with computers or technology are as-
Completely unsatisfied 14 (1.9) sociated with satisfaction; if so, additional training may be
What is your perception about overall patient satisfaction provided to those most at risk of dissatisfaction with the system. Of
with today’s telemedicine visit? particular interest may be differentiating frequent telemedicine
Completely satisfied 267 (36.2) users from occasional users. The relative impact of satisfaction with
Generally satisfied 345 (46.8)
telemedicine may depend on how frequently the provider uses the
Neutral 75 (10.2)
Generally unsatisfied 37 (5.0) technology, as well as how comfortable the patient is with the
Completely unsatisfied 13 (1.8) system. In addition, more information beyond demographic char-
acteristics of the patients, such as diagnosis codes or other clinical
information, would enrich our understanding of the relationship
between telemedicine use in the prisons and provider satisfaction
adequate to meet the clinical needs of the patients. These results with evolving technologies. Additional research into the patient
suggest that the providers are successfully using the telemedicine perspective of telemedicine and satisfaction with the system are
technology to treat prisoners using these methods in mental needed. An additional limitation of the current cross-sectional
health=psychology and HIV=infectious disease. The type of ap- study is the inability to track trends in satisfaction or utilization
pointment that resulted in the greatest challenges included mental over a longer time frame. This temporal information would be
health televisits for residents of juvenile detention centers. These helpful to see if as the providers become more accustomed to the
patients were least likely to have their visits completed, and were telemedicine procedures, their levels of utilization or satisfaction
Table 4. Adjusteda Characteristics Associated with Provider Satisfaction and Patient Outcomes,
December 2007 to May 2008 (Odds Ratio [95% Confidence Interval]) (n ¼ 737)
SATISFIED WITH
TELEMEDICINE TECHNOLOGY CLINICAL TODAY’S SATISFIED WITH PERCEPTION
VISIT TREATMENT PLAN ADEQUATE FOR IMPROVED DECISION MAKING TELEMEDICINE TELEMEDICINE OF PATIENT
COMPLETED ESTABLISHED TODAY’S VISIT PROGNOSISb FACILITATEDb OUTCOMEb SYSTEMb SATISFACTIONb
Patient sex
Male 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
Female 1.09 (0.24–4.97) 0.34 (0.06–1.94) 0.82 (0.23–2.94) 0.41 (0.22–0.75)c 0.49 (0.27–0.89)d 0.38 (0.20–0.71)c 0.35 (0.18–0.66)c 0.40 (0.22–0.75)c
Patient race
Non–African 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
American
African 1.27 (0.71–2.25) 4.64 (1.54–13.98)c 0.95 (0.52–1.75) 1.10 (0.82–1.46) 1.12 (0.87–1.56) 1.08 (0.82–1.44) 1.05 (0.79–1.39) 1.25 (0.94–1.66)
American
Patient age
(years)
18 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
<18 0.35 (0.13–0.96)d 0.15 (0.18–1.34) 0.26 (0.07–0.90)d 1.59 (1.06–2.39)d 1.56 (1.03–2.35)d 1.73 (1.16–2.59)c 1.54 (1.03–2.29)d 2.14 (1.43–3.20)e
Provider type
Nonphysician 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
Physician 1.01 (0.34–2.98) 2.24 (0.27–20.61) 2.73 (0.79–9.53) 1.46 (0.97–2.20) 1.87 (1.24–2.82)c 1.64 (1.10–2.47)d 1.89 (1.27–2.81)c 1.61 (1.08–2.40)d
a
Adjusted for all other variables listed.
b
OR > 1.0 indicates increasing disagreement with statement and dissatisfaction with system; OR < 1.0 indicates increasing agreement with statement
and satisfaction with system.
c
p < 0.01.
d
p < 0.05.
e
p < 0.001.
change. As the telemedicine system in Louisiana evolves, time- over time and in provider and patient satisfaction with the technol-
trend analyses can be conducted to see if there are trends in sat- ogy as it broadens and is implemented elsewhere throughout the
isfaction and usage over time. state.
REFERENCES 20. Danahar D. Louisiana update. Delta to explore new venues in 1999.
Fac Notes (New Orleans La) 1999;11:4.
1. Fox KC, Somes GW, Waters TM. Timeliness and access to healthcare services via
21. Caceres C, Gomez EJ, Garcia F, Gatell JM, del Pozo F. An integral care telemedicine
telemedicine for adolescents in state correctional facilities. J Adolesc Health
system for HIV=AIDS patients. Int J Med Inform 2006;75:638–642.
2007;41:161–167.
22. Vazquez E. HIV treatment in prison. Posit Aware 2001;12:32–34.
2. Fox KC, Whitt AL. Telemedicine can improve the health of youths in detention.
J Telemed Telecare 2008;14:275–276. 23. Mahoney MR, Khamarko K, Goldschmidt RH. Care of HIV-infected Latinos in the
United States: A description of calls to the National HIV=AIDS Clinicians’
3. Goodale J, Menzel D, Hodgson G. High-tech prisons: Latest technologies
Consultation Center. J Assoc Nurses AIDS Care 2008;19:302–310.
drive cost savings and staff efficiencies. Corrections Today 2005;67:78–81.
24. Makulowich JS. AIDS and telemedicine. AIDS Patient Care STDS 1996;10:
4. Watson W, Stimpson A, Hostick T. Prison health care: A review of the literature.
387–388.
Int J Nurs Stud 2004;41:119–128.
25. Chanussot-Deprez C, Contreras-Ruiz J. Telemedicine in wound care.
5. Clarke M, Thiyagarajan CA. A systematic review of technical evaluation in
Int Wound J 2008;5:651–654.
telemedicine systems. Telemed E Health 2008;14:170–183.
26. Wurm EM, Hofmann-Wellenhof R, Wurm R, Soyer HP. Telemedicine and
6. Field MJ. A guide to assessing telecommunications in health care. Washington,
teledermatology: Past, present and future. J Dtsch Dermatol Ges 2008;6:
DC: National Academies Press, 1996.
106–112.
7. Mair F, Whitten P. Systematic review of studies of patient satisfaction with
27. Wurm EM, Campbell TM, Soyer HP. Teledermatology: How to start a
telemedicine. Br Med J 2000;320:1517–1520.
new teaching and diagnostic era in medicine. Dermatol Clin 2008;
8. Godleski L, Nieves JE, Darkins A, Lehmann L. VA telemental health: Suicide 26:295–300, vii.
assessment. Behav Sci Law 2008;26:271–286.
28. Mofid M, Nesbitt T, Knuttel R. The other side of teledermatology: Patient
9. Yellowlees P, Marks S, Hilty D, Shore JH. Using e-health to enable culturally preferences. J Telemed Telecare 2007;13:246–250.
appropriate mental healthcare in rural areas. Telemed J E Health 2008;
29. The Louisiana State University Health Care Services Division Medical Informatics
14:486–492.
and Telemdicine Program. Telemedicine Hardware. http://medinfo-telemed.
10. Mitchell JE, Crosby RD, Wonderlich SA, Crow S, Lancaster K, Simonich H, lsuhsc.edu/ (last accessed April 4, 2010).
Swan-Kremeier L, Lysne C, Myers TC. A randomized trial comparing the efficacy
30. Harrison R, Macfarlane A, Murray E, Wallace P. Patients’ perceptions of joint
of cognitive-behavioral therapy for bulimia nervosa delivered via
teleconsulations: A qualitative evaluation. Health Expect 2006;9:81–90.
telemedicine versus face-to-face. Behav Res Ther 2008;46:581–592.
31. Love W. Patient and provider satisfaction with the use of telemedicine:
11. Ross JT, TenHave T, Eakin AC, Difilippo S, Oslin DW. A randomized controlled
Overview and rationale for cautious enthusiasm. J Postgrad Med 2005;51:
trial of a close monitoring program for minor depression and distress.
294–300.
J Gen Intern Med 2008;23:1379–1385.
32. Nova Scotia Department of Health. Remote specialist consultation and
12. Mozer E, Franklin B, Rose J. Psychotherapeutic intervention by telephone.
continuing medical education pilot project. Halifax: The Department, 1997. Last
Clin Interv Aging 2008;3:391–396.
accessed September 9, 2007.
13. Cloutier P, Cappelli M, Glennie JE, Keresztes C. Mental health services
33. University of Alaska Statewide Health Programs and University of Alaska
for children and youth: A survey of physicians’ knowledge, attitudes
Anchorage Center for Human Development. Evolution & summative evaluation
and use of telehealth services. J Telemed Telecare 2008;14:98–101.
of the Alaska Federal Health Care Access Network telemedicine project 2004.
14. Yellowlees P, Burke MM, Marks SL, Hilty DM, Shore JH. Emergency 2004. http://www.alaska.edu/health/downloads/Telemed/AFHCAN.pdf (last
telepsychiatry. J Telemed Telecare 2008;14:277–281. accessed September 9, 2007).
15. Myers KM, Valentine JM, Melzer SM. Child and adolescent 34. Dagroso D, Williams PD, Chesney JD, Lee MM, Theoharis E, Enberg RN.
telepsychiatry: Utilization and satisfaction. Telemed J E Health 2008; Implementation of an obstetrics EMR module: Overcoming user dissatisfaction.
14:131–137. IJ Healthc Inf Manag 2007;21:87–94.
16. Morgan RD, Patrick AR, Magaletta PR. Does the use of telemental health 35. Adler KG, Edsall RL. Electronic health records: A user-satisfaction survey.
alter the treatment experience? Inmates’ perceptions of telemental health Fam Pract Manag 2005;12:45–54.
versus face-to-face treatment modalities. J Consult Clin Psychol 2008;
36. Joos D, Chen Q, Jirjis J, Johnson KB. An electronic medical record in primary
76:158–162.
care: Impact on satisfaction, work efficiency and clinic processes. AMIA Annu
17. Besch CL. Telemedicine improves access to care for HIV-infected prisoners. Symp Proc 2006;2006:394–398.
HIV Clin 2007;19:4–5.
37. Likourezos A, Chalfin DB, Murphy DG, Sommer B, Darcy K, Davidson SJ.
18. Zolfo M, Arnould L, Huyst V, Lynen L. Telemedicine for HIV=AIDS care in low Physician and nurse satisfaction with an electronic medical record system.
resource settings. Stud Health Technol Inform 2005;114:18–22. J Emerg Med 2004;27:419–424.
19. Zolfo M, Lynen L, Dierckx J, Colebunders R. Remote consultations and HIV=AIDS 38. O’Connell RT, Cho C, Shah N, Brown K, Shiffman RN. Take Note(s):
continuing education in low-resource settings. Int J Med Inform 2006;75: Differential EHR satisfaction with two implementations under one roof. J Am
633–637. Med Inform Assoc 2004;11:43–49.
39. Whitten P, Buis L, Mackert M. Factors impacting providers’ perceptions Address correspondence to:
regarding a midwestern university-based EMR. Telemed J E Health Manya Magnus, Ph.D., M.P.H.
2007;13:391–397.
Department of Epidemiology and Biostatistics
40. McDonald WR. External evaluation of the California telemedicine and eHealth
center (CTEC) network development grant: Bi-Annual Progress Report. 2007.
The George Washington University School
of Public Health and Health Services
41. Rose D. Evaluation of the California telehealth and telemedicine center.
In: Dennis & Rose Associates DRA Project Reports, June 2002, Vol. 2. 2100-W Pennsylvania Avenue, NW, Suite 807
http://www.cttconline.org/eval.html (last accessed April 2008). Washington, DC 20037
42. Association AT. Kentucky Telecare, University of Kentucky Chandler
Medical Center. Telemedicine Consultant Evaluation Form. http:// E-mail: [email protected]
www.americantelemed.org/news/forms.html (last accessed April 2008).
Received: November 21, 2009
43. Network MT. Patient questionnaire. 2001.
Accepted: December 24, 2009