Evaluation of Employee Assistance Programs Employee Assistance Quarterly Volume 3, Numbers 3/4 CONTENTS PREFACE EAPs: Assessing How They Work Marvin D. Feit Michael J. Holosko SECTION I: “STATE-OF-THE-ART” OF THE EAP FIELD Economie Change and Social Welfare: Implications for Employees’ Assistance Mark J. Stern ‘Technology and the Changing Composition of the Labor Force Flexible Specialization and the Quality of Work Life ‘The Decline of Labor-Management Cooperation ‘The Casualties of Post-Industrialization Implications for Public and Occupational Welfare EAP in Transition: Purpose and Scope of Services Keith McClellan Richard E. Miller Introduction Sampling and Data CollectionA Framework for Evaluating Employee Assistance Programs N. Joseph Cayer Ronald W. Perry SUMMARY. Particularly in the past decade, Employee Assistance Programs (EAPs) have become an integral part of the management of public sector and private sector organizations. The evolution of APs is briefly traced andthe esi of advocsies regarding their Positive effects are documented. It is also emphasized that, in spite thet ubiquity, ite attention has been aven o devising strategies for evaluating the success or failure of EAPs. A framework is devel oped to guide comprehensive evaluations of such programs. A framework is developed to guide comprehensive evaluations of such Programs, The framework stresses five aspects of EAP performance: effort, program performance (outcome evaluation), adequacy of performance, efficieney, and proces evalation Employee Assistance Programs (EAP) are crucial elements of ‘management in contemporary society. EAPs are designed to help employees whose job performance is affected by theit personal problems. Estimates of the number of employees Who have prob- ems affecting their work range widely, although figures between 5 and 20% seem to be most common. The difficulty in accurately estimating the number stems, in part, from the fact that there is no clear definition or standard by which the problem can be measured, Similarly, there arc no clear standards by which the effectiveness of EAPs can be assessed. As a result, the purpose of this paper is to elaborate a framework for assessing the effectiveness of EAPS. NN. Joseph Cayer, PRD, and Ronald W. Perry, PHD, are both affiliated with the School of Public Affairs, Arizona State University, Tempe, AZ 85287, © 1988 by The Haworth Press, Inc, All rights reserved. ist152 EVALUATION OF EMPLOYEE ASSISTANCE PROGRAMS BACKGROUND Employee Assistance Programs are an extension of alcohol abuse programs which many industries and government jurisdictions set up years ago. In recent years, “substance abuse” has replaced “al- cohol abuse’? as a more descriptive term for people who are im- paired by the use of drugs, including alcohol. Drugs of all kinds have become a major issue in the United States, and numerous pro- posals for dealing with the problems are extant. Employee Assis- tance Programs represent one alternative for dealing with the prob- Jem in the work setting. While BAPs often are associated with substance abuse problems, they also encompass much more, In recent years, they have evolved to deal with all types of personal problems individuals have which may affect performance, and many EAPs include service for the whole family of the employee. Problems other than substance abuse which are covered by some EAPs include marital problems, emo- tional distress, job stress, and sexual dysfunction and anxiety. In short, as long as the problem affects job performance, it may be included in the EAP. EAPs have emerged as part of “'good management’? and out of the humanizing of the workplace which is hallmark of the growth of organization theory in the 1960s and 1970s (Denhardt, 1984; Stewart & Garson, 1983; Harmon & Mayer, 1986). EAPs are justi fied as good management because they are expected to save money and increase organizational productivity. Because of the numbers of employees estimated to have personal problems which affect work performance, estimates of the cost to industry have been immense as well. In 1984, for example, it was claimed that employers in- curred nearly $200 billion in costs associated with personal prob- lems related to work performance (Myers, 1984). Avoidance of those costs through proactive programs to help individual employ- ees before they have to be disciplined is generally touted as good management, Humanizing of the workplace began with the landmark Hawthorne Studies of the 1920s and the development of Human Relations in management, Human Relations focused on the psycho- logical well-being of employees and used social group norms andSection Il: The Context of EAP Evaluations support to complement formal management. Because they became more manipulative in practice, many employees saw through many Human Relations’ efforts. During the 1960s and 1970s, however, individual rights and the personal growth of individuals became im- portant elements of our political and social culture. Employee rights and treating employees with dignity were extensions of those con- cerns. Organizational humanism focuses on the compatibility of the individual and the organization, specifically the work organization (Atgyris 1973a, 1973b; McGregor, 1960). The individual is viewed as a whole person who has needs and a life outside the organization. Thus, it is impossible for the individual to function inside the work organization with no reference to his/her life outside. Training pro- grams dealing with the human side of management burgeoned dur- ing the late 1960s and early 1970s. Managers became aware of the human beings who were their employees and were encouraged to take interest in the whole person. Employee Assistance Programs are consistent with such a perspective. They are much easier to develop in an atmosphere in which the organization is sensitive to the employee as a complex human being. While there has been phenomenal growth in Employee Assis- tance Programs, research on them has been extremely limited. Most research has focused on private sector EAPS and is primarily de- scriptive or promotional without much rigor in evaluation method- ology or design. In the public sector, there is even less written on EAPs, and almost no evaluation materials are available, Employee Assistance Programs are found at all levels of govern- ment, The federal government encourages EAPs through efforts of the Office of Personnel Management. Alcoholism programs are re- quired by Public Law 91-616, and employees must have the oppor- tunity for rehabilitation before they can be terminated for alcohol- ism related job performance deficiencies. While the law applies only to alcoholism, its spirit carries over into development of gen- eral EAs. State and local levels also have a variety of programs although they are not as extensive as those at the national level (Beyer & Trice, 1978; Johnson, 1985; Kemp, 1985). While programs are es- tablished, they do not seem to be evaluated, at least as indicated by published’ research. In periods of retrenchment, it would seem natu-154 EVALUATION OF EMPLOYEE ASSISTANCE PROGRAMS ral that programs which cannot be demonstrated to have tangible benefits would be very vulnerable to cuts. Evaluation of EAPs pro- vides a basis on which to make rational decisions about their viabil- ity. So far, people suggest that they save money by using EAPs, but there is no solid research to support the effectiveness of them (Myers, 1984). Thus, we propose a model for evaluating EAPs. ISSUES IN EVALUATING EAPs There are several possible approaches to evaluating Employee Assistance Programs depending upon the type of assessment de- sired. One tactic is to evaluate an organization's need for an EAP (cf. Burgrabbe & Swift, 1984). Another approach involves evaluat- ing service vendors associated with an existing EAP (cf. Chiabotta, 1985), Still another perspective entails evaluating an EAP as a func- tioning program associated with a host organization (Rostain et al., 1980). Each evaluation approach answers specific questions about the need for or operation of EAPs. Our specific concern is with the zgrowth of EAPs in the public sector. In this context, we expect that decisions to establish or expand EAPs will be based at least in part upon data regarding the performance of EAPs. The conception of performance that is important here encompasses a broad view of the EAP, How does an EAP fit into the host organization? Do EAPS realize their goals? Are such plans cost-effective? We acknowledge these different approaches to evaluation, but will concentrate on the more general task of creating a framework for the comprehensive evaluation of an Employee Assistance Program. At this point, it is useful to briefly discuss two primary assump- tions about EAPs and the structural characteristics of the plans that underly the development of our evaluation model. The first assump- tion is that EAPs should not be conceptualized as a special fringe benefit under the rubric of employee compensation (Hollmann, 1981:40), Instead, one should deal with EAPs in terms of human resources management and fit an Employee Assistance Program into the matrix of other personnel services. For example, one means of identifying employees who need assistance might be the perfor- mance appraisal system, since EAPs typically cite improved perfor- mance as an objective. Also, one might find that a particular workSection Il: The Context of EAP Evaluations 155 unit sends many employees to an EAP “‘stress reduction/coping’” treatment program. This information could be used to either inform the selection process to include stress tolerance in screening, or to begin a process of job redesign in an effort to minimize structural sources of stress. This example shows the potential value of con- necting EAPs with performance appraisal, selection, and job de- sign. A consequence of our assumption that EAPs should be inte grated into the personnel process is that our evaluation model demands attention to the fit of the EAP into the organization’s man- agement structure. One structural characteristic of EAPs that has implications for evaluation is that they provide a variety of specific (but distinct) services. A 1980 survey of ASPA member organizations showed that more than 90% of those with EAPs offered alcohol rehabilita- tion, drug abuse counseling, emotional counseling, and family/mar- ital counseling (Ford & McLaughlin, 1981:32). Other programs commonly offered as employee assistance include financial, legal, and career counseling. The EAP then, may be represented both as, an office in the organization with an employee assistance function and as a collection of diverse services some of which may be ren- dered by organizational staff and some by outside vendors. From an evaluation standpoint, both of these characterizations are important. A primary justification offered for EAPs is that they improve employee performance and consequently improve organi- zational productivity. The specific services are aimed at treating individual’s problems and thereby affecting (presumably improv- ing) job performance. A comprehensive evaluation must, therefore, examine each type of service rendered to determine whether treat- ment objectives are met. In the aggregate, all of these services com- bine to form the Employee Assistance Program which, in itself, has the objective of improving organizational productivity. Conse- quently, an evaluation model should address changes in aggregate measures of productivity as well. Our evaluation framework will reflect this implicit distinction between the EAP and the services offered. The two assumptions discussed above serve to clarify our defini- tion of the phenomenon to be evaluated. We will treat employee assistance as a process that is ongoing within the organization and156 EVALUATION OF EMPLOYEE ASSISTANCE PROGRAMS that is integrated with other types of personnel services. We also acknowledge that EAPs involve an organizational component (at a minimum, an assessment office attached to personnel department). Furthermore, they may also entail an external component or compo- nents in the form of vendors who may control one or more of the specific treatment services offered by the EAP. These assumptions begin to set minimum evaluative performance expectations for the EAP vis-i-vis the operations of the host organization. They also alert the evaluator to the importance of multiple audiences in under- standing the performance of EAPs. These audiences include man- agement, vendors, treated employees, and untreated employees. ‘The following section builds upon these assumptions and lays out the rudiments of a comprehensive evaluation model for EAPs. A COMPREHENSIVE EVALUATION FRAMEWORK From a policy research standpoint, evaluation is often conceived as primarily ouicome or summative evaluation designed to deter- mine if a program meets its goals. Weiss (1972) has pointed out that this is a deceptively simple conception of the evaluation. The de- ception stems from the implication that goals and outcomes may be specified in a straightforward manner. Indeed, to specify the goals of a program one must appreciate its full context and operation; as some important goals may be implicit, unstated, or not subscribed to by the operators of a given program. In addition, a given program may produce a variety of “‘outcomes” along a variety of dimen- sions well beyond the objectives that could be derived from an orga- nizational or legislative goal-type statement, ‘The issues mentioned above commonly arise in the context of policy-related evaluations where a single organization is charged with the delivery of a service or the dissemination of a message. Even here, one must acknowledge the importance of process ot formative evaluation as a means of providing information to help the evaluator understand the outcome evaluation, Process evalua- tion focuses upon how the program functions—not ‘whether’? ‘goals are met but what” is done to attempt to mect them. As part Of process, the evaluator must document what organizational and procedural devices we create to achieve stated objectives. Thus,Section Il: The Context of EAP Evaluations process evaluation answers the question of “why” relative to out- come evaluation—e.g., if the program achieved its objectives, our process evaluation supplies clues regarding why that occurred. Put another way, process evaluation supplies information on the context in which a program operates. We wish to argue that under- standing “process” is critical to the understanding of the perfor- mance of programs associated with public or private sector organi- zations. In connection with EAPs, the context in which the program operates is crucial to determining the success of its operation. In fact, our conception of EAPs as a component of the personnel pro- cess emphasizes that context may vary and principles of evalua- tion—what to identify as-objectives and how to define perfor- mance—depend upon this organizational context. For example, @ completely “‘in-house”” EAP would be evaluated largely in terms of intraorganizational relations, while an EAP involving vendors adds an interorganizational dimension to the evaluation. Thus, it is im- portant that a comprehensive evaluation framework contain ele- ments of both process and outcorhe evaluation, This approach raises the question of exactly what constitutes an evaluation framework. In general research terms, frameworks are ofien seen as precursors to more formal models that constitute sym- bolic constructions designed to represent structures or processes which prevail in the ‘real”” world (Kaplan, 1964:256). An evalua- tion framework, then, may be seen as the specification of the foci and purposes of evaluation. In terms of this paper, an evaluation framework must identify the standards in terms of which perfor- ‘mance will be assessed, define the organizational dimensions of the program, specify the units of analysis, and specify the populations from which evaluative information should be collected. Many of these issues have already been discussed or at least al- luded to in this paper. Our comprehensive framework will take the form of an claboration of performance standards. Then, under each standatd we will identify both specific research questions for evalu- ation and particular issues that might arise in connection with an assessment. In keeping with traditional evaluation practice, and with our concern about the importance of including process dimen- sions for evaluation, we shall build an evaluation framework around158 EVALUATION OF EMPLOYEE ASSISTANCE PROGRAMS five performance standards: Effort, Program Performance (outcome evaluation), Adequacy of Performance, Efficiency, and Process, Effort In general, evaluations of program effort concentrate upon as- sessing the input or energy expended in goal-connected activity (Suchman, 1967:61). One is not concerned with goal achievement here; only’ the level of effort and commitment associated with the program. Ultimately, of course, the results of the “‘effort”” evalua- tion should be connected to program performance especially when evaluators diagnose particular levels of performance. The phase of effort evaluation itself, however, is aimed only at documenting ex- penditures of energy and resources to be used as baseline data for other interpretation, With regard to evaluating EAPs, the routine part of effort evalua- tion involves the production of a resources expenditure chart, Rela- tive to internal organizational resources, this involves specification of the number and levels (rank) of personnel involved in managing the EAP, estimates of levels of effort (personne! hours per week), equipment and space assigned for program use, and budget. If ven: dors are involved in particular treatment services, one would con- struct similar charts for each vendor. In dealing with vendors, how- ever, the evaluative purpose of resource charting is to insure that adequate resources are provided for services relative to the price of service. Our assumptions about the role of EAPs in the personnel. process demand that as part of effort evaluation one also assess the manage- ‘ment priority of such programs. Two elements should be considered here. First, one should examine management’s commitment (o the EAP as a personnel tool. This entails examining, perhaps through a series of interviews or a self-administered questionnaire, attitudes of managers toward the EAP, their estimate of its value as a mecha- nism for dealing with problem employees, and the extent to which they are involved in reviews of the EAP. Second, itis necessary to examine the organizational arrangements associated with the EAP. This involves determining the location of the EAP: is it an office, a branch of the personnel department, a branch of a compensationSection Il: The Context of EAP Evaluations 159 office, or perhaps wholely handled by external vendors? Also, one should document the extent to which the EAP is integrated into the larger matrix of human resources management. The objective of all these analyses is to determine the overall resources and effort that are devoted to employee assistance. EAP Performance This phase of the comprehensive evaluation deals with what is traditionally called output or summative evaluation: determining if the programs under review accomplish-their goals (Rossi & Free- man, 1982:20), Traditional research designs for assessing program performance rely heavily upon experimental or quasiexperimental designs of a before-aficr nature (Campbell & Stanley, 1966:20). While this type of design remains appropriate, our evaluative inter- ests in EAPS require that more than one type of assessment be made. EAPs are actually a collection of what could be called employee “treatment” services. We ate, therefore, interested in all of the dividual programs that make up an EAP (e.g., legal counseling, substance abuse treatment, emotional counseling, and so on). The relevant unit of analysis is the individual and each individual is typically tied to a particular program. Of course in long-standing programs, the evaluator would want to look at the effects of one person being treated in multiple programs. Two types of questions need to be asked about treated individ- uals. The first is: did the program work? Did the substance abuse program, for example, result in the employee becoming detoxed and learning positive coping behaviors? This question may be as- sessed using the traditional before-after control group designs one associates with evaluation. The second, and perhaps more impor- tant question, is: did the treated employee's work performance im- prove? ‘The almost universal claim regarding EAPS is that they are neces- sary to improve performance. There are many ways in which one might operationalize performance, but there seem to be at least three important dimensions of the concept insofar as EAPs are con- cerned. ‘The first dimension is employee job performance. This160 EVALUATION OF EMPLOYEE ASSISTANCE PROGRAMS might be measured by standard performance appraisal techniques. ‘A second dimension relates to their acceptance and socialization of employees into the work group. Indicators of this dimension might include absenteeism, tardiness, grievances filed by employees, grievances filed against employees, work related accidents, and su- pervisor assessments of cooperativene: The thitd dimension of performance, as it is discussed in the EAP literature, involves a change in the unit of analysis and focuses upon organizational measures of performance. For evaluative purposes, organization refers to the units from which EAP treated employees come. In this sense, we are concerned with the collective petfor- ‘mance of the work unit which is normally captured under the rubric of productivity. Indicators of this dimension could include aggr gate absentecism and tardiness, health care payments and insurance Premiums, time spent in grievance and termination proceedings, tumover of experienced employees, aggregate production esti- mates, and recruiting and training costs, The indicators mentioned in connection with all three dimensions are intended to be sugges- tive rather than exhaustive. The important point is that one should choose at least some indicators from eac’ of the three dimensions ef performance. In answering the question of whether the treated employees work performance changed after their experience with the EAP, we would focus upon the indicators listed under the first two dimen- sions of performance just discussed. Two types of comparisons are relevant: looking at differences between treated and untreated em- ployees and at differences between pre- and posttreatment perfor mance of treated employees. ‘The results of such analyses of individual performance should be aggregated by EAP service. Thus, the evaluator compiles informa: ion on the effectiveness of each service relative to treating individ- uals and with regard to the impact of the treatments themselves on subsequent employee performance ‘The previous analyses all deal with the behavior of individuals and theit experience and treatment in EAP service components (which may be in-house or vendor operated). We have pointed out earlier that employee assistance constitutes a personnel function and that consequently it is important to assess the aggrogate perfor-Section Il: The Context of EAP Evaluations 161 mance of that function. With regard to outcome evaluation, this may be accomplished by looking at indicators of the third dimen- sion of productivity discussed earlier. These aggregate measures are designed to examine the unit-wide or organization-wide perfor- ‘mance of the EAP. One may pursue at least three tactics for making such evaluations, First, in organizations where EAPs are new or relatively recent, ‘comparisons on the productivity dimension may be made between periods before and after the EAP began to operate. Second, in orga- nizations with longer standing EAPs, one might seek structurally similar organizations without EAPs for comparative purposes. An- other option in this regard would be to compare productivity in organizational units served and not served by EAPS; this requires of course, a large organization. Finally, one can assume that as an EAP operates over time and more employees are processed, productivity benefits should in- crease. (The appropriateness of this assumption depends upon the accuracy of EAP advocate’s claims that the programs really in- crease productivity.) If this is true, then over a delimited period of time—a year perhaps—one should be able to track changes in the productivity indicators. This third type of design is based upon what psychologists call an ““A-B-A” logic and is somewhat complicated to implement (cf. Bushell & Burgess, 1969:145). Adequacy of EAP Performance The concept of adequacy of performance is a relative matter. It has its roots in the needs assessment literature and refers to the ability of the program being evaluated to address the extent of the Prevailing problem (cf. Burdge, 1983:191). Put another way, we are interested in the power of the EAP to deliver employee assi tance relative to the total need for employee assistance in the oiga- nization. Clearly, estimates of “total need” are difficult to acq sometimes may be somewhat arbitrary. Options for estima need include: (1) estimating need by examining the number of em- ployees seeking assistance in an EAP operated in a comparable or- Banization or unit; (2) estimating need by comparing the number of162 EVALUATION OF EMPLOYEE ASSISTANCE PROGRAMS employees waiting to be served with the number actually being processed through an EAP; (3) estimating need for specific pro- grams by using standard formulae, like —for example ~-those which exist for calculating aggregate levels of alcohol or substance abuse in organizations; or (4) taking a probability sample of all employees and asking for self-reports of “felt”” needs for particular types of assistance, Without regard to the precise logic used for estimating need, ade- quacy assessments involve comparing this need to the current levels of output of an EAP. The results of such evaluations may be used in making recommendations for expanding or reducing levels of effort in connection with specific employees assistance services. Efficiency The concept of efficiency is largely bound up in cost-benefit or cost-effectiveness analyses of EAPs (Thompson, 1980). This in- volves both value judgments and technical judgments. The value judgments focus upon the extent to which the benefits of a program justify the costs. While one may pursue a variety of “objective” techniques for assessing both cost and benefit, the ultimate balanc: ing of one over the other relies both upon the availability of re- sources to sustain the cost and upon the perceived value of offering the service. Cost-benefit analyses may be achieved through a vari- ety of standardized techniques (Rossi & Freeman, 1982:291), ‘Technical judgments are captured in the determination of whether there is a more efficient means of obtaining the observed results, Here interest focuses upon identifying alternative paths to the same level of output and characterizing each in terms of costs. ‘These costs might be measured in terms of budget expended, treat- ment and administrative personnel time, employee work time lost, and levels of personnel involvement. Once again, the objective is to determine if structural or programmatic changes may be made in the EAP that would reduce costs but not seriously alter output. This type of evaluation draws heavily upon the concept of efficiency in the sense that it is used in operations research, where the goal is to determine an optimum course of action relative to decision-maker's objectives (Ackoff, 1956).Section Il: The Context of EAP Evaluations Process Evaluation In large part, the purpose of process evaluation is to aid in inter- preting other phases of the comprehensive evaluation (Weiss, 1972). Most process evaluations begin by creating a case flow chart. Such charts resemble decision trees or fault trees in that they specify all operations (decision and contact points) associated with selecting, evaluating, treating, and reintegrating employees. Pro- cess evaluations of APs should address at least four issues. Attributes of the EAP Here we are concerned with documenting particular aspects of EAP operation. An initial concern is with the paths through which ‘employees come to be involved with the EAP. What are the com- ‘mon modes of selection ~ employee self-selection, supervisor nom- ination, assignment by performance category, etc. — and how many employees enter through each mechanism? This analysis also offers cues to differential use of EAP services and to constellations of treatment programs needed by certain types of employee problems, ‘Another attribute of the EAP which should be explicitly exam- ined is the effect of employee participation in EAP services on the routine production of work units. There are several aspects of this problem. First, one should document the way in which employee participation is handled. Do employees obtain EAP treatment dur- ing work hours and if so, is work made up after regular hours? If an absence from the work group is required, is the employee granted personal leave and replaced or is work simply redistributed? The primary evaluative objective is to determine if unit productivity suf- fers as a function of EAP use. In sophisticated evaluation designs, one might also want to ex- amine the effect of EAP participation on morale in the work group. In this case, interest centers upon documenting hostilities or hard- ships that might prevail in the work group. Such analyses would demand either interviews or self-administered questionnaires for members of the work unit. ‘Two final attributes of EAP performance which demand attention ae the length of time emplayces remain in treatment and chronic EAP use. The average time required to complete any particular164 EVALUATION OF EMPLOYEE ASSISTANCE PROGRAMS EAP service correlates with both the cost of the service and the ‘work time lost by employees. Work time lost may also be a factor in the level of work unit disruption that stems from EAP participation, By documenting time involved, one can identify potential problems stemming from such commitments and potentially develop struc- tural modifications to mitigate them. Chronic EAP use may not be a problem in itself, although it would tend to exacerbate the problem Of time away from the work unit, Since BAP participation is usually contingent upon formal assessment of employee need conducted by trained personnel, it does not seem likely that chronic use would be a manifestation of employee malingering. On the other hand, chronic use may indicate that: (1) EAP treatment services produce transient effects; or (2) that the organization may require a mecha- nism for reviewing employees with serious problems for retention. Differential Treatment Results ‘The evaluative goal here is with identifying the consequences of treatment results. For example, are some types of employees more or less likely to constitute treatment successes? The research tactic involves comparing treatment successes with treatment failures to determine if there are differences relative to sex, age, length of employment, type of work, mode of selection for treatment, spe- cific treatment program, or some other employee attribute. It is also appropriate to determine if a work unit containing EAP treatment successes experiences any changes in productivity, morale, or use of the EAP. Conditions of EAP Service Success In this context, attention is focused upon determining if the tim- ing, location, or treatment arrangements associated with a given EAP service is related to either the number of treatment successes or levels of employee participation. Inthe latter case, data would be generated that could be used for structuring services in such a way that employee participation is facilitated by maximizing personal convenience. Organizational goals could also be promoted by deter- mining which service arrangements involved the lowest levels of work unit disruption, but maintained reasonable treatment successSection i: The Context of EAP Evaluations “fates. In the former case, failure rates of different EAP treatment programs could be monitored to determine the need to possibly con- tract with new vendors or adopt different treatment modalities ~ > which might yield higher levels of treatment success. Specification of Treatment Program Effects In the case of EAPs, the evaluator is interested in a chain of effects, In the first place, one expects that each treatment program will have defined positive effects upon a specific employee prob- lem; that is, alcoholism programs will reduce employee dependence on alcohol. We have given much attention to specifying and analyz ing such effects in the preceding discussions. It is also assumed that EAP participation will have a secondary effect; namely that correc- tion of the treated problem will also result in improved employee productivity. We have already elaborated a variety of performance (productivity) indicators which might be changed as a function of EAP participation, As part of specifying program effects, itis important to document which performance indicators tend to change the most among em- ployees who receive EAP services. If management-defined ‘*key”” Performance indicators change, then it may be assumed that the EAP function is adequately served. On the other hand, if no change is observed or if the changes occur with indicators not deemed im- portant by management, then it is necessary to reconsider and possi- bly reconfigure EAP services. ‘Another aspect of creating an inventory of effects associated with EAP participation is specifying the full range of effects. In compre- hensive evaluations, it may be desirable to document effects other than those upon productivity, For example, EAP patticipation may enhance employee job satisfaction or motivation. Also, there may be identifiable impacts on the work unit from which treatment suc- cesses come; perhaps higher unit morale or productivity. In part, such analyses also begin to establish the extent to which treatment effects are primarily behavioral or attitudinal or both. In some cases, management may feel that a more positive attitude toward work is as useful an outcome as improvements in productivity. Finally, in specifying program effects, it is important to docu-166 EVALUATION OF EMPLOYEE ASSISTANCE PROGRAMS ment both the immediacy and duration of effects. If a particular service renders immediate improvements in the employee which increase with increased participation, the level of disruption to the ‘work unit may be reduced. From the point of view of management, it would seem appropriate to select programs that produce high lev- els of immediacy. The issue of duration of effects raises the ques- tion of recidivism. If results are short-lived, with the employee problem quickly reappearing, it would seem necessary to again de- termine if the problem rests with the treatment program itself— necessitating a program or vendor change—or with the employee — which may demand a retention review, CONCLUSION ‘This paper proposes a comprehensive framework within which one may evaluate Employee Assistance Programs, It is understood that in achieving a degree of comprehensiveness, we have devel- oped a very broad evaluative strategy. The resources required to conduct an evaluation with this many components are admittedly substantial, Indeed, itis likely that none but the largest organiza- tions could afford to undertake an evaluation of the scope proposed here. ‘The framework is useful, however, in at least two senses, First, in its full form we contend that it captures the range of evalua- tive questions that might be asked with respect to any EAP. Second, it is explicit enough to be amenable to selective reductions in scope to accommodate organizations unable to commit the full comple- ment of resources to evaluation. Thus, one could select components of the comprehensive framework for an evaluation effort of smaller scope. Furthermore, the selection can be made on an informed basis by drawing on the types of questions most important to manage- ment. REFERENCES Ackoff, R., The Development of Operations Research as a Science, Operations Research, 1956, 4, 265-295, Argytis, C., Some Limits of Rational Man Organizational Theory. Public Admin istration Review, 19T3a, 33, 253-267,‘Boyer, J. & Trice, H., Implementing Change. New York: The Fre Press, 1978. ‘Butdge, R., Community Needs Assessment and Techniques. pp 191-214 in Kurt “Finsiebusch, Lyon Llewellyn & Charles Wolf, Soci Impact Assessment “Methods. Beverly Hills, CA! Sage, 1983. “ Burgrabbe, J. & Swift, J., Evaluating Your EAP: A Practical Approsch. EAP Digest, 1984, 4, 13-16. “Bushell, D. de Burgess, R., Characteristics of the Experimental Analysis. pp 145- "174 in Robert Burgess & Donald Bushell, Behavioral Sociology. New York Columbia University Press, 1969. - Campbell, D. & Stanley, J., Experimental and Quasi-Experimental Designs for ‘Research. Chicago: Rand McNally, 1966. Chiabota, B., Evaluating EAP Vendors. Personnel Administrator, 1985, 30, 39- 43, Denhardt, R., Theories of Public Organization. Monterey, CA: BrooksiCole, 1988, Fitz-Gibbon, C. & Mortis, L., How to Design a Program Evaluation. Beverly Hill, CA: Sage, 1978. Ford, R. & Mclaughlin, F., Employee Assistance Programs: A Descriptive Sur- vey of ASPA Members. Personnel Administrator, 981, 26, 29-3. Harmon, M. & Mayer, R., Organization Theory for Public Administration. Bos- ton: Little, Brown, 1985. Hollman, R., Beyond Contemporary Employee Assistance Programs. Personnel Administracor, 1981, 26, 31-41. Johnson, A., Municipal Employee Assistance Programs: Maneging Troubled Em ployees. Public Administraton/Review, 1985, 45, 383-390. Kaplan, A., The Conduct of Inquiry. San Francisco: Chandler Publishing Com- pany, 196. Kemp, D., Sate Employee Assistance Programs: Organization and Services Public Administration Review, 1985, 43, 378-382. McClellan, K., The Changing Nature of EAP Practice, Personnel Administrator, 1985, 30, 29.37, McGregor, D., The Human Side of Enterprise. New York: McGraw-Hill, 1960. Myers, D., Measuring EAP Cost Effectiveness: Rests and Recommendations. EAP Digest, 1984, 4, 23-25. Roman, P., Employee Assistance Programs in Australia and the United States. Journal of Applied Behavioral Science, 1983, 19, 367-379. Rossi P. & Freeman, H., Bvaluation: A Systematic Approach. Beverly Hills CA: Sage, 1982. Rosiain, H, Allan, P. & Rosenberg, S., New York City's Approach to Problem- Employee Counseling. Personnel Journal, 980, 21, 308-325 Sonnenstubl, W. & O'Donnel, J., EAPS: The Why's and How's of Planning ‘Them, Personnel Administrator, 1980, 25, 35:38168 EVALUATION OF EMPLOYEE ASSISTANCE PROGRAMS Sur, A. & Byram, G., CostBenefit Analysis for Employee Assistance Pro- siams. Personnel Adminisirators, 1985, 30, 5562. Stewart, D. & Gatson, G., Organizational Behavior and Public Management ‘Now York: Marcel Deke, 1983. Suchman, E., Evaluative Research. New York: Russell Sage Foundation, 1967. ‘Thompson, M., Benefi-Cost Analysis for Program Evaluation. Beverly Hills, CA: Sage, 1980, Weiss, C., Evaluation Research. Englewood Clits, NJ Prentice-Hall, 1972.