Cognitive Predictors

Cognitive Predictors of Students Success in the Medical School

Yuxiang Liu Director of Institutional Assessment Office of Institutional Research St. Johns University 8000 Utopia Parkway Jamaica, NY 11439 [email protected]

Handan Hizmetli Assistant Director of Institutional Research Bronx Community College City University of New York University Avenue at West 181 Street Bronx, NY 10453 [email protected]

2003 Annual Forum Association for Institutional Research Tampa, Florida

Cognitive Predictors of Students Success in the Medical School

Cognitive Predictors

Abstract This study was designed to identify cognitive predictors of students success in the medical school. Selected as sample were 203 medical school students (162 succeeded and 41 failed). Student status (succeeded or failed) was used as the dependent variable. Logistic Regression identified two (ACT Science and Regents Biology) out of 14 independent variables as the most significant predictors. These two, however, are good predictors in predicting who would succeed, not who would fail. The results imply that students with top scores on the admissions tests are more likely to succeed, but those with lower scores are not necessarily bound to fail. Introduction What is the relationship between students high school and college performances, and what are the predictors of their success at college? Educators and researchers have been trying to find answers to such questions. The related literature indicates that, apart from the demographic factors such as gender and ethnicity, studies in this field have concentrated on two broad categories: cognitive predictors and non-cognitive predictors (Blumberg, 1984; Endres, 1997; Fan, Li, & Niess, 1998; Roessler et al., 1978; Schwartz & Washington, 2002; Ting, 1998). Cognitive predictors cover such areas as high school academic performance and college entrance test scores, while non-cognitive predictors include such variables as motivation, career aspiration, and leadership potentials. In 1983, Arnold, Calkins, and Willoughby studied the relationship between students high school academic achievement and their performance in a six-year combined baccalaureate and doctor of medicine program in a school of medicine in Missouri. Used as dependent variables in their study were: the GPA for the baccalaureate degree, scores on the National Board of Medical Examiners Examination Parts I and II, and a clinical performance evaluation of a Year Six medicine rotation. They found that the most consistent predictors of students performance in the combined baccalaureate and doctor of medicine program were high school science GPA, scores on the college aptitude test, the number of social science College Level Examination Program hours earned, and ratings received from references. Touron (1987) did a similar study at University of Navarra, Spain. His study included a multiple regression analysis with students first-year grades in science courses (in a six-year combined baccalaureate and doctor of medicine program) as the dependent variable. The independent variables in his study were students grades in high school science courses and their scores on college admissions science tests. The results of his study indicated: (a) The overall GPA of high school sciences was a better predictor of students success in first-year college science courses than the individual high school science grades, and (b) The same is true of the scores on college admissions science tests: the average score predicts better than the individual scores. Murden et al. (1978) investigated into both the academic and non-academic predictors of medical school students clinical success. Used in their study as the dependent variable was the internship evaluation (with grades A, B, C, or D) by a 10-member Internship Advisory Committee. The academic predictors used as independent variables were college science GPA and Medical College Admission Test, and the non-academic independent variables included maturity, rapport, nonacademic achievement, and motivation, as evaluated during

Cognitive Predictors

the interview. The results of their study revealed that the college science GPA had a smaller but nevertheless significant relationship with clinical success, and that students with high levels of maturity, nonacademic achievement, motivation, or rapport, were about two to three times as likely to receive outstanding internship recommendations as those without such characteristics. To understand the relationship between students high school and college performances and to identify predictors of students success at college are important not only for the college admissions office but also for students, parents, high school counselors, and the society as a whole. With such knowledge, parents and counselors are able to give students betterinformed advice on choosing their college programs, and students are able to better understand their potentials and make wiser decisions for their future. The society as a whole would certainly benefit if the limited educational recourses are used effectively and students potentials are maximized. Problem Statement and Research Questions This study was designed to investigate into the cognitive predictors of students success in the medical school. Success was defined as earning their baccalaureate degree and passing Step 1 of the U.S. Medical Licensure Examination in the fifth year of a seven-year combined baccalaureate and doctor of medicine program. The predictor variables tested in this study were high school GPA, ACT English, ACT Math, ACT Reading, ACT Science, ACT Composite, SAT Verbal, SAT Math, SAT total, Regents Biology, Regents Chemistry, Regents English, Regents Math, and Regents Physics. Specifically, this study was intended to seek answers to the following questions: (a) What were the significant predictors of students success in the medical school? and (b) How accurately did these predictors predict students success in the medical school? Method Selected as the sample in this study were 203 students who entered the medical school of a metropolitan university in New York from 1993 to 1997. These students were directly from high school, and the program they entered was a seven-year combined baccalaureate and doctor of medicine program. In their fifth year, if they successfully fulfilled the requirements, they earned their baccalaureate degree and took Step 1 of the U.S. Medical Licensure Examination (USMLE1), and then after another two years of study, they were expected to earn their doctorate in medicine. Of the 203 students selected, 162 earned their baccalaureate degree and passed USMLE1, and 41 were dismissed or dropped out because of academic difficulties. Students who succeeded and those who failed were dichotomously coded as 1 and 0, respectively, and this student status (succeeded or failed) was used as the dependent variable. Used as the independent variables were high school GPA, and the scores on the following tests: ACT English, ACT Math, ACT Reading, ACT Science, ACT Composite, SAT Verbal, SAT Math, SAT total, Regents Biology, Regents Chemistry, Regents English, Regents Math, and Regents Physics. Logistic Regression (Forward Stepwise) in SPSS was used to test which of the independent variables are significant predictors of students success in the medical school.

Cognitive Predictors

Results Results from Step 0 (when only the constant was used and all the independent variables were excluded) of Stepwise Logistic Regression indicate: of the 14 independent variables, seven could potentially make significant contributions to the model (p=.05). The seven variables were ACT English, ACT Science, ACT Composite, SAT total, Regents Biology, Regents English, and Regents Physics. ACT Science was entered in Step 1 of the Logistic Regression and it was identified as a significant predictor of students success in the medical school. After this variable was entered, the classification table read as follows: The model could predict 96% of who would succeed, but only 34% of who would fail. After ACT Science was entered into the model in Step 2, Regents Biology became the only independent variable (of the 13 variables excluded from the model) that could make a significant contribution to the model. The independent variable of Regents Biology was entered in Step 2 and identified as another significant predictor of students success in the medical school. The classification table indicated that the model could predict 95% of who would succeed and still only 54% of who would fail. After ACT Science and Regents Biology were entered into the model and identified as significant predictors of students success in the medical school, no additional variables met the .05 significance level for entry into the model. Therefore, there were no further steps for the Logistic Regression. The value for the initial 2 Log Likelihood is 204.437, and value for the model Chi-square is 82.831. The model, therefore, accounts for 41% (the model Chi-square divided by the initial 2LL) of students success in the medical school. The parameter estimates for the equation of the model is as follows: Log(odds)=-20.498 + .152(Regents Biology) + .374(ACT Science) According to the above equation, for a student with the Regents Biology score of 75 and ACT Science score of 15, the probability of success is only 3.0%; when ACT science increases to 25 and Regents Biology remains the same, the probability of success increases to 56.3%. Table 1 below presents the students success rate by admissions test scores (identified in this study as potential significant predictors of students success in the medical school). As stated above, success in this study is defined as earning the baccalaureate degree and passing Step 1 of the U.S. Medical Licensure Examination in the fifth year of a seven-year combined baccalaureate and doctor of medicine program. For each of the seven admissions tests, students are grouped into lower scorers and higher scorers, and the success rates for both subgroups are presented. Table 1 Students Success Rate by Admissions Test Scores
Admissions Test Test Score Number of Students Number Succeeded Success Rate

Cognitive Predictors

25 or below ACT English Above 25 25 or below ACT Science Above 25 25 or below ACT Composite Above 25 1200 or below SAT Total Above 1200 90 or below Regents Biology Above 90 85 or below Regents English Above 85 85 or below Regents Physics Above 85

119 84 113 90 96 107 98 105 101 102 99 104 102 101

97 73 87 83 73 98 80 92 79 94 80 93 88 94

81% 87% 77% 92% 76% 91% 81% 88% 78% 92% 81% 90% 86% 93%

Table 1 indicates that it is true for each test that the success rate of students with higher scores is higher than those of students with lower scores. For ACT Composite, ACT Science, and Regents Biology, the differences in success rates between two groups range from 14% to 15%. Conclusion The results of this study indicate that scores on ACT Science and Regents Biology are the most significant predictors of students success in the medical school. The contents of the two tests are actually related since science covers the area of biology. Chemistry (Regents), another area that ACT Science covers, however, was identified as an insignificant predictor of student success in the medical school. The two most significant predictors of students success in the medical school, ACT Science and Regents Biology, however, are only good at predicting who would succeed, not who would fail. The results imply that students with top scores on the tests are more likely to succeed, but those with lower scores are not necessarily bound to fail.

Cognitive Predictors

Medical studies require hard working and perseverance. Students who are not well prepared may finally succeed if they are motivated and work hard, or they may fail if they dont have any motivation and perseverance. This study concentrates on cognitive predictors only. Investigation into non-cognitive predicators is needed in order to fully explain students success in the medical school.

References Arnold, L, Calkins, E. V., & Willoughby, T. L. (1983). Can achievement in high school predict performance in college? College and University, 59, 95-101. Blumberg, P. (1984). Predicting student success from non-cognitive variables. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans. Fan, T, Li, Y, & Niess, M. (1998). Predicting academic achievement of college computer science majors. Journal of Research on Computing in Education, 31(2), 155-172. Murden, R, Galloway, G, Reid, J, & Colwill J. (1978). Academic and personal predictors of clinical success in medical school. Journal of Medical Education, 53, 711-719.

Cognitive Predictors

Roessler, R, Lester, J, Butler, W, Rankin, B, & Collins F. (1978). Cognitive and noncognitive variables in the prediction of preclinical performance. Journal of Medical Education, 53, 679-681. Schwartz, R, & Washington, C. (2002). Predicting academic performance and retention among African American Freshmen. NASPA Journal, 39(4), 354-370. Ting, S. (1998). Predicting first-year grades and academic progress of first-generation and low-income families. Journal of College Admission, 158, 14-23. Touron, J. (1987). High school ranks and admission tests as predictors of first year medical students performance. Higher Education, 16(3), 257-266. Elliott, P. R. (1975). The selection of primary care physicians. Speech presented at the Workshop for Undergraduate Education in Family Medicine hosted by the American Academy of Family Physicians, Kansas Ciry. Jones, B. J., & Borges, N. J. (2000). Evaluation of psychological factors in medical school admissions decisions. Paper presented at the Mid-Western Educational Research Association Annual Meeting, Chicago, IL. Owen, J. A. (2000). The multivariable relationship between student characteristics at medical school interview, interviewer predictions of career places, and student career plans at graduation. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans. Tutton, P. M., & Wigg, S. J. (1990) The influence of last secondary school attended, subjects taken in last year of secondary education and gender on preclinical performance of medical students. Australian Journal of Education, 34, 168-173.