Showing posts with label IQ tests. Show all posts
Showing posts with label IQ tests. Show all posts

Tuesday, January 06, 2026

Interesting, reasonably accurate recent video on IQ/intelligence testing

I just stumbled across a relatively new video covering the history and several major issues regarding intelligence testing and IQ scores.  Two scholars that I respect (Dr. Cecil Reynolds; Dr. Stuart Ritchie) are featured in the video.  I did see some spelling errors in the subtitles (Dr. Ian Dearie instead of Dr. Ian Deary; Benet instead of Binet; using capital G when referencing Spearman's concept of general intelligence, which is always noted with an italic font small g; etc) and heard several statements that made me cringe slightly.  

Also, it left the impression that fluid and crystallized intelligence (and a lessor extent quantitative ability) are the primary recognized broad cognitive abilities measured by intelligence tests.  It did not acknowledge contemporary CHC theory as the consensus taxonomy of human cognitive abilities.  Also, it left the impression that IQ tests are "bubble in" multiple choice tests.  This may be true for group tests, but it is not the case with individually administered intelligence tests.

Overall, it is a reasonable video to share with others as an introduction, possibly in college courses where the concept of intelligence and IQ testing is being introduced.  It did a good job of covering the historical bad uses of IQ tests (e.g., discrimination; cultural bias, eugenics movement, etc.) 

The complete video is approximately 35 minutes.  It did freeze up for me at the 17 minute mark when it was going to display an ad....but I simply restarted the video and quickly moved to that point and then it continued.


 

 

 

Monday, August 25, 2025

IQs Corner: What is (and what is not) clinical judgment in intelligence test interpretation? - #IQ #intelligence #ID #intellectualdisability #schoolpsychologists #schoolpsychology #diagnosis

What is clinical judgment in intelligence testing?  

This term is frequently invoked when psychologists explain or defend their intelligence test interpretations.  Below is a brief explanation I’ve used to describe what it is…and what it is not, based on several sources.  Schalock and Luckasson’s AAIDD Clinical Judgment book (now in a 2014 revised version) is the best single source I have found that addresses this slippery concept in intelligence testing, particularly in the context of a potential diagnosis of intellectual disability (ID)—it is a recommended reading.

—————

Clinical judgment is a process based on solid scientific knowledge and is characterized as being “systematic (i.e., organized, sequential, and logical), formal (i.e., explicit and reasoned), and transparent (i.e., apparent and communicated clearly)” (Schalock & Luckasson, 2005, p.1). The application of clinical judgment in the evaluation of IQ scores in the diagnosis of intellectual disability includes consideration of multiple factors that might influence the accuracy of an assessment of general intellectual ability (APA: DSM-5, 2013).  The “unanimous professional consensus that the diagnosis of intellectual disability requires comprehensive assessment and the application of clinical judgment” (Brief of Amici Curiae American Psychological Association, American Psychiatric Association, American Academy of Psychiatry and the Law, Florida Psychological Association, National Association of Social Workers, and National Association of Social Workers Florida Chapter, in Support of Petitioner; Hall v. Florida; S.Ct., No. 12-10882; 2014; p. 8).

The misuse of clinical judgment in the interpretation of scores from intelligence test batteries should not be used as the basis for “gut instinct” or “seat-of-the-pants” impressions and conclusions of the assessment professional (Macvaugh & Cunningham, 2009), or justification for shortened evaluations, a means to convey stereotypes or prejudices, a substitute for insufficiently explored questions, or an excuse for incomplete testing and missing data (Schalock & Luckasson, 2005). Idiosyncratic methods and intuitive conclusions are not scientifically based and have unknown reliability and validity. 

If clinical judgement interpretations and opinions regarding an individual’s level of general intelligence are based on novel or emerging research-based principles, the assessment professional must document the bases for these new interpretations as well as the limitations of these principles and methods. This requirement is consistent with the Standards for Educational and Psychological Testing Standard 9.4 which states:

When a test is to be used for a purpose for which little or no validity evidence is available, the user is responsible for documenting the rationale for the selection of the test and obtaining evidence of the reliability/precision of the test scores and the validity of the interpretations supporting the use of the scores for this purpose (p. 143).


American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2014).  Standards for educational and psychological testing.  Washington, DC:  Author. 

American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders— Fifth Edition. Washington D.C.:  Author. 

Brief of Amici Curiae American Psychological Association, American Psychiatric Association, American Academy of Psychiatry and the Law, Florida Psychological Association, National Association of Social Workers, and National Association of Social Workers Florida Chapter, in Support of Petitioner; Hall v. Florida; S.Ct., No. 12-10882; 2014; p. 8.

MacVaugh, G. S. & Cunningham, M. D. (2009). Atkins v. Virginia: Implications and recommendations for forensic practice.  The Journal of Psychiatry and Law, 37, 131-187.

Schalock, R. L. & Luckasson, R. (2005). Clinical judgment. Washington, DC: American Association on Intellectual and Developmental Disabilities. 

—————

Kevin S. McGrew, PhD.

Educational Psychologist

Director 

Institute for Applied Psychometrics (IAP)

www.theMindHub.com


Saturday, April 24, 2021

The evolution of the Woodcock-Johnson (WJ--WJ IV) global IQ or g scores - The WJ is the Elon Musk of IQ testing

 Across the various editions of the WJ Tests of Cognitive Abilities (WJ, WJ-R, WJ III, WJ IV), the authors continually have sought to improve the measurement of intelligence via following contemporary research and theory.  As a result, in contrast to many other IQ tests, the WJ has been known for global IQ scores (original called Broad Cognitive Ability, later changed to General Intellectual Ability) that changed rather dramatically from one revision to the next.  We the WJ authors might be considered to be the Elon Musk of IQ test development.

I was recently asked to explain the changing nature of the BAC/GIA scores.  The result is the table inserted below (double click to enlarge).  I believe the table is self-explanatory.  A nice PDF copy can be downloaded here.  Enjoy.




Wednesday, February 25, 2015

Intelligence Testing in Proper Perspective: The Big Picture - a MindHub video-PPT show

I am pleased to announce the availability of my second video-PPT at IQ's Corner YouTube Channel.  The first video can be found here.  The current video is 20 minutes.  You can skip the first three minutes if you don't want background "front matter" material regarding me (the narrator), etc.

The new video-PPT  is called "Intelligence Testing in Proper Perspective:  The Big Picture"

This presentation places the power and value of intelligence testing into a big picture perspective which recognizes the strengths and limitations of intelligence testing.  The goal is to encourage users and consumers of intelligence tests to better understand what these measures can and cannot do, and, more importantly, recognize the other personal and environmental characteristics that influence an individual's learning and development.

Be gentle.  I am not a professional video producer and I do not have the time to edit out pauses, minor mistakes, etc.---- hey...this is FREE quality information. :)

Update 02-27-15.  Thanks to Rueben Lopez for making the suggestion that I reduce the 3 minutes of the introductory "front matter."  I have taken his advice (which I will incorporate into future videos) and have now posted the identical video with the very brief introduction.  It can be accessed here.





Saturday, June 07, 2014

More research on the validity of the C-LIM framework in intelligence testing

Another article adding to the Cultural-Linquistic Interpretive Matrix research literature. Click on images to enlarge. A copy of the article can be found here.

"Conclusion

The primary conclusion drawn from this study and previous research is that linguistic demand is an important consideration when selecting and interpreting tests of cognitive abilities. The implications of this study go beyond a re-classification of the C-LIM to emphasizing one of the underlying motivations of the C-LIM's initial inception—the importance of considering a student's linguistic background and abilities prior to selecting, administering, and interpreting tests of cognitive abilities. A comprehensive evaluation that takes a student's linguistic ability into consideration should consider that a student's language ability (i.e., conversational proficiency) might not be an accurate representation of a student's academic language abilities (Cummins, 2008). Thus, it would be beneficial to gather information on a student's academic language abil-ity, due to the relationship between education and IQ (Matarazzo & Herman, 1984). A student's receptive and expressive language abilities may be a worthwhile pursuit in future research, as student's level of conversational proficiency in the classroom may mislead educators and psy-chologists to assume that the student has been exposed to English with the same frequency and depth as his or her peers (Cummins, 2008). Moreover, as suggested by the results of this study, considering the influence of linguistic ability when assessing cognitive abilities should continue to be supported by empirical evidence, instead of school psychologists continuing to rely on informal measures of linguistic ability through language samples and student interviews to gain information on language ability (Ochoa, Galarza, & Gonzalez, 1996).

A second conclusion is that it is unclear how cultural loading can be represented quantitatively in a way that is meaningful both theoretically and practically. An important, albeit unanswered question is, "What variables do practitioners take into account when making decisions about the cultural influences that may affect the selection and interpretation of tests from cognitive batteries?" Flanagan and Ortiz (2001) define cultural loading as "the degree to which a given test requires specific knowledge of or experience with mainstream culture" (p. 243). However, this broad definition does not identify specific variables that practitioners may consider in practice to make these decisions about whether a student's experiences are significantly different from mainstream culture. Given these unanswered questions, it is possible that the underlying reasoning that led to the creation of the C-LIM and its categorization system needs to be re-thought (as also suggested by Styck & Watkins, 2013), particularly with respect to cultural loading. Specifically, it would be important to consider what is occurring and possible in practice, as this is the intended use of the C-LIM."

 

 

Tuesday, November 15, 2011

Thinking..fast and slow: Dual process models of cognition/intelligence--hot topic

Dual cognitive process (sometimes called Type I/II processing) have increased in prominence the past five years.  Within the past few weeks the long anticipated book "Thinking, Fast and Slow" by Daniel Kaneham was released, and it is already near the top of most non-fiction best selling books.  I can't wait to get my copy, as it will put Malcom Gladwell's "Blink" in it's proper place.  This will give the layperson, and many professionals, a better understanding of these two general classes of cognitive processes.

My thinking about applied intelligence test development and interpretation has been incorporated this general dichotomy in the form of a working (evolving) test development/interpretation framework (see summary figure below).
[Double click on images to enlarge]

The most recent journal to devote a special issue to dual process models is Developmental Review.  Below are the key articles and a few intriguing model figures.















Generated by: Tag Generator

Friday, August 19, 2011

One of the best texts on intellectual assessment has been revised: Flanagan & Harrison's Contemporary Intellectual Assessment

Just in time for your XMAS shopping!!!! One of the best texts on intellectual assessment and theories related to the practice of intellectual assessment.

The publisher has given me permission to post this information. The text below does not show all the formatting in the original document sent to me, so if you want a nicer PDF version to share with others, click here.


Conflict of interest disclosure: I have coauthored a chapter in the book and will be splitting an honorarium check (not big, trust me) and will be receiving a free copy. But, I get no royalties (I wish I did).


Kudos to Dr. Flanagan and Harrison for revising what I consider one of the best texts on intellectual assessment.





NEW FROM THE GUILFORD PRESS (​Revised and Expanded!)

Contemporary Intellectual Assessment, Third Edition
Theories, Tests, and Issues
Edited by Dawn P. Flanagan, PhD, Department of Psychology, St. John's University; and
Patti L. Harrison, PhD, Department of Educational Studies in Psychology, Research Methodology, and Counseling, University of Alabama

Available to Ship: December 2011
Copyright: 2012
Pages: 926
Size: 7" x 10"
Hardcover: ISBN 978-1-60918-995-2
Hardcover Price: $95.00 tentative/short discount
Prior edition copyright: 2005
Prior edition cloth ISBN: 978-1-59385-125-5
Website Category: EDUCATION: Educational Psychology; School Psychology. PSYCHOLOGY: School Psychology; Neuropsychology & Neuroscience; Child/Adolescent Clinical Psychology & Psychiatry.
Subject Areas/Keywords: adults, assessment instruments, batteries, children, cognitive assessments, developmental, educational psychology, evaluations, intellectual assessments, intellectual disabilities, intelligence testing, learning disabilities, response to intervention, RTI, tests
Internal Code: F
Date Issued: August 15, 2011

CRITICAL ACCLAIM
"This is the most comprehensive, authoritative, and up-to-date text on intellectual assessment available. It covers current theories of intelligence, methods of intelligence testing, and their applications to special populations. The chapters are all written by leading scholars and combine clear research summaries with practical advice, making this a valuable book for graduate students and professionals interested in both research and practice."—Paul J. Frick, PhD, University Distinguished Professor and Chair, Department of Psychology, University of New Orleans

"A single source for essential, detailed information on the past, present, and future of intellectual and cognitive assessment practices. The table of contents provides an objective sweep of all major theories, tests, and evaluation procedures at a glance; the expertise of the chapter authors results in a work that is consistently outstanding. The third edition reflects the latest efforts in clinical inquiry that explore cognitive abilities and processes as they relate to the real world. It emphasizes cross-cultural issues in assessment and incorporates new approaches and instruments related to learning and developmental problems. This is an essential text for graduate-level assessment courses."
—Elaine Fletcher-Janzen, EdD, Department of School Psychology, Chicago School of Professional Psychology

"This updated volume is a valuable addition to the field of intellectual and psychological assessment. The editors have assembled the leaders in the field to present the most up-to-date information available. Many of the chapters are written by the test authors and theory creators themselves; readers will benefit from the firsthand approach to test and theory interpretation. The book is written in a way that will appeal to both experienced practitioners and graduate students just starting out in the field."—Andrew S. Davis, PhD, Department of Educational Psychology, Ball State University

"A superb theoretical and clinical overview....The standards of theoretical and methodological rigor, comprehensive topical coverage, balanced and objective critical analysis, life span cognitive evaluation, and advocacy for special populations...are beautifully balanced. One could not ask for more from a review and critical evaluation of this extensive, rich, and complex literature."—PsycCRITIQUES

"Should be required reading in all intelligence testing courses and by anyone involved in the assessment of human and cognitive abilities."—Psychotherapy in Private Practice

DESCRIPTION
In one volume, this authoritative reference presents a current, comprehensive overview of intellectual and cognitive assessment, with a focus on practical applications. Leaders in the field describe major theories of intelligence and provide the knowledge needed to use the latest measures of cognitive abilities with individuals of all ages, from toddlers to adults. Evidence-based approaches to test interpretation, and their relevance for intervention, are described. The book addresses critical issues in assessing particular populations—including culturally and linguistically diverse students, gifted students, and those with learning difficulties and disabilities—in today's educational settings.

New to This Edition
*Incorporates major research advances and legislative and policy changes.
*Covers recent test revisions plus additional tests: the NEPSY-II and the WNV.
*Expanded coverage of specific populations: chapters on autism spectrum disorders, attention-deficit/hyperactivity disorder, sensory and physical disabilities and traumatic brain injury, and intellectual disabilities.
*Chapters on neuropsychological approaches, assessment of executive functions, and multi-tiered service delivery models in schools.

KEY POINTS
> An authoritative reference, revised and expanded: features 11 new chapters.
> Comprehensive and current: covers all major tests and how they relate to educational services and policy.
> Chapters are written by the test developers themselves.
> A successful practitioner handbook and graduate-level text.

AUDIENCE
Practitioners, students, and researchers in school, educational, cognitive, and child clinical psychology.

COURSE USE
Serves as a primary text in graduate-level intellectual/cognitive assessment courses.

CONTENTS
I. The Origins of Intellectual Assessment
1. A History of Intelligence Assessment: The Unfinished Tapestry, John D. Wasserman
2. A History of Intelligence Test Interpretation, Randy W. Kamphaus, Anne Pierce Winsor, Ellen W. Rowe, and Sangwon Kim
II. Contemporary Theoretical Perspectives
3. Foundations for Better Understanding of Cognitive Abilities, John L. Horn and Nayena Blankson
4. The Cattell–Horn–Carroll (CHC) Model of Intelligence, W. Joel Schneider and Kevin S. McGrew
5. Assessment of Intellectual Profile: A Perspective from Multiple-Intelligences Theory, Jie-Qi Chen and Howard Gardner
6. The Triarchic Theory of Successful Intelligence, Robert J. Sternberg
7. Planning, Attention, Simultaneous, Successive (PASS): A Cognitive Processing–Based Theory of Intelligence, Jack A. Naglieri, J. P. Das, and Sam Goldstein
III. Contemporary Intelligence, Cognitive, and Neuropsychological Batteries (and Associated Achievement Tests)
8. The Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV) and the Wechsler Memory Scale–Fourth Edition (WMS-IV), Lisa Whipple Drozdick, Dustin Wahlstrom, Jianjun Zhu, and Lawrence G. Weiss
9. The Wechsler Preschool and Primary Scale of Intelligence–Third Edition (WPPSI–III), the Wechsler Intelligence Scale for Children–Fourth Edition (WISC–IV), and the Wechsler Individual Achievement Test–Third Edition (WIAT–III), Dustin Wahlstrom, Kristina C. Breaux, Jianjun Zhu, and Lawrence G. Weiss
10. The Stanford–Binet Intelligence Scales, Fifth Edition (SB5), Gale H. Roid and Mark Pomplun
11. The Kaufman Assessment Battery for Children–Second Edition (KABC-II) and the Kaufman Test of Educational Achievement–Second Edition (KTEA-II), Jennie Kaufman Singer, Elizabeth O. Lichtenberger, James C. Kaufman, Alan S. Kaufman, and Nadeen L. Kaufman
12. The Woodcock–Johnson III Normative Update (WJ III NU): Tests of Cognitive Abilities and Tests of Achievement, Fredrick A. Schrank and Barbara J. Wendling
13. The Differential Ability Scales–Second Edition (DAS-II), Colin D. Elliott
14. The Universal Nonverbal Intelligence Test (UNIT): A Multidimensional Nonverbal Alternative for Cognitive Assessment, R. Steve McCallum and Bruce A. Bracken
15. The Cognitive Assessment System (CAS): From Theory to Practice, Jack A. Naglieri and Tulio M. Otero
16. The Reynolds Intellectual Assessment Scales (RIAS) and the Reynolds Intellectual Screening Test (RIST), Cecil R. Reynolds, Randy W. Kamphaus, and Tara C. Raines
17. The NEPSY-II, Robb N. Matthews, Cynthia A. Riccio, and John L. Davis
18. The Wechsler Nonverbal Scale of Ability (WNV): Assessment of Diverse Populations, Jack A. Naglieri and Tulio M. Otero
IV. Contemporary Interpretive Approaches and Their Relevance for Intervention
19. The Cross-Battery Assessment (XBA) Approach: An Overview, Historical Perspective, and Current Directions, Dawn P. Flanagan, Vincent C. Alfonso, and Samuel O. Ortiz
20. Cognitive Hypothesis Testing (CHT): Linking Test Results to the Real World, Catherine A. Fiorello, James B. Hale, and Kirby L. Wycoff
21. Processing Approaches to Interpreting Information from Cognitive Ability Tests: A Critical Review, Randy G. Floyd and John H. Kranzler
22. Testing with Culturally and Linguistically Diverse Populations: Moving beyond the Verbal–Performance Dichotomy into Evidence-Based Practice, Samuel O. Ortiz, Salvador Hector Ochoa, and Agnieszka M. Dynda
23. Linking Cognitive Abilities to Academic Interventions for Students with Specific Learning Disabilities (SLD), Nancy Mather and Barbara J. Wendling
V. Assessment of Intelligence and Cognitive Functioning in Different Populations
24. Cognitive Assessment in Early Childhood: Theoretical and Practical Perspectives, Laurie Ford, Michelle L. Kozey, and Juliana Negreiros
25. Use of Intelligence Tests in the Identification of Giftedness, David E. McIntosh, Felicia A. Dixon, and Eric E. Pierson
26. Use of Ability Tests in the Identification of Specific Learning Disabilities (SLD) within the Context of an Operational Definition, Dawn P. Flanagan, Vincent C. Alfonso, Jennifer T. Mascolo, and Marlene Sotelo-Dynega
27. Assessment of Intellectual Functioning in Autism Spectrum Disorder (ASD), Laura Grofer Klinger, Sarah E. O’Kelley, Joanna L. Mussey, Sam Goldstein, and Melissa DeVries
28. Cognitive and Neuropsychological Assessment of ADHD: Redefining a Disruptive Behavior Disorder, James B. Hale, Megan Yim, Andrea N. Schneider, Gabrielle Wilcox, Julie N. Henzel, and Shauna G. Dixon
29. Intellectual and Neuropsychological Assessment of Individuals with Sensory and Physical Disabilities and Traumatic Brain Injury, Scott L. Decker, Julia A. Englund, and Alycia M. Roberts
30. Use of Intelligence Tests in the Identification of Children with Intellectual and Developmental Disabilities (IDD), Kathleen Armstrong, Jason Hangauer, and Joshua Nadeau
VI. Contemporary and Emerging Issues in Intellectual Assessment
31. Using Joint Test Standards to Evaluate the Validity Evidence for Intelligence Tests, Jeffery P. Braden and Bradley C. Niebling
32. Using Confirmatory Factor Analysis (CFA) to Aid in Understanding the Constructs Measured by Intelligence Tests, Timothy Z. Keith and Matthew R. Reynolds
33: The Emergence of Neuropsychological Constructs into Tests of Intelligence and Cognitive Abilities, Daniel C. Miller and Denise E. Maricle
34. The Role of Cognitive and Intelligence Tests in the Assessment of Executive Functions, Denise E. Maricle and Erin Avirett
35. Intelligence Tests in the Context of Emerging Assessment Practices: Problem-Solving Applications, Rachel Brown-Chidsey and Kristina J. Andren
36. Intellectual, Cognitive, and Neuropsychological Assessment in Three-Tier Service Delivery Practices in Schools, George McCloskey, James Whitaker, Ryan Murphy, and Jane Rogers
Appendix. The Three-Stratum Theory of Cognitive Abilities, John B. Carroll

CONTRIBUTORS
Dynda Agnieszka, PsyD, Department of Psychology, St. John’s University, Jamaica, New York
Vincent C. Alfonso, PhD, Graduate School of Education, Fordham University, New York, New York
Kristina J. Andren, PsyD, School Psychology Program, University of Southern Maine, Gorham, Maine
Kathleen Armstrong, PhD, Department of Pediatrics, University of South Florida, Tampa, Florida
Erin Avirett, BA, Department of Psychology and Philosophy, Texas Women’s University, Denton, Texas
Nayena Blankson, PhD, Department of Psychology, Spelman College, Atlanta, Georgia
Bruce A. Bracken, PhD, School of Education, The College of William and Mary, Williamsburg, Virginia
Jeffery P. Braden, PhD, Department of Psychology, North Carolina State University, Raleigh, North Carolina
Kristina C. Breaux, PhD, The Psychological Corporation, San Antonio, Texas
Rachel Brown-Chidsey, PhD, School Psychology Program, University of Southern Maine, Gorham, Maine
John B. Carroll, PhD, Emeritus Professor of Psychology, University of North Carolina, Chapel Hill, North Carolina
Jie-Qi Chen, PhD, Erikson Institute, Chicago, Illinois
J. P. Das, PhD, Department of Educational Psychology, University of Alberta, Edmonton, Alberta, Canada
John L. Davis, MA, Department of Educational Psychology, Texas A&M University, College Station, Texas
Scott L. Decker, PhD, Department of Psychology, Barnwell College, University of South Carolina, Columbia, South Carolina
Melissa DeVries, PhD, Neurology, Learning, and Behavior Center, Salt Lake City, Utah
Felicia A. Dixon, PhD, Department of Educational Psychology, Ball State University, Muncie, Indiana
Shauna G. Dixon, MS, Graduate School of Education, Harvard University, Cambridge, Massachusetts
Lisa Whipple Drozdick, PhD, The Psychological Corporation, San Antonio, Texas
Agnieszka M. Dynda, PsyD, Department of Psychology, St. John’s University, Jamaica, New York
Colin D. Elliot, PhD, The Gervitz School of Education, University of California, Santa Barbara, California
Julia A. Englund, BA, Department of Psychology, University of South Carolina, Columbia, South Carolina
Catherine A. Fiorello, PhD, NCSP, School Psychology Program and Department of Psychological Studies in Education, College of Education, Temple University, Philadelphia, Philadelphia
Dawn P. Flanagan, PhD, Department of Psychology, St. John’s University, Jamaica, New York
Randy G. Floyd, PhD, Department of Psychology, University of Memphis, Memphis, Tennessee
Laurie Ford, PhD, Department of Educational and Counseling Psychology, University of British Columbia, Vancouver, British Columbia, Canada
Howard Gardner, PhD, Graduate School of Education, Harvard University, Cambridge, Massachusetts
Sam Goldstein, PhD, Neurology, Learning, and Behavior Center, Salt Lake City, Utah
James B. Hale, PhD, School Psychology Program, Philadelphia College of Osteopathic Medicine, Philadelphia, Philadelphia
Jason Hangauer, EdS, Department of Pediatrics, University of South Florida, Tampa, Florida
Julie N. Henzel, PsyD, The Nisonger Center, Ohio State University, Columbus, Ohio
John L. Horn, PhD (deceased), Department of Psychology, University of Southern California, Los Angeles, California
Randy W. Kamphaus, PhD, College of Education, Georgia State University, Atlanta, Georgia
Alan S. Kaufman, PhD, Child Study Center, Yale University, New Haven, Connecticut
James C. Kaufman, PhD, Department of Psychology, California State University, San Bernardino, California
Nadeen L. Kaufman, PhD, Child Study Center, Yale University, New Haven, Connecticut
Timothy Z. Keith, PhD, Department of Educational Psychology, University of Texas at Austin, Austin, Texas
Sangwon Kim, PhD, Graduate School of Education, Fordham University, New York, New York
Laura Grofer Klinger, PhD, Department of Psychology, University of Alabama, Tuscaloosa, Alabama
Michelle L. Kozey, MA, Department of Educational and Counseling Psychology, University of British Columbia, Vancouver, British Columbia, Canada
John H. Kranzler, PhD, Special Education Program, College of Education, University of Florida, Gainesville, Florida
Elizabeth O. Lichtenberger, PhD, private practice, Carlsbad, California
Denise E. Maricle, PhD, Department of Psychology and Philosophy, Texas Women’s University, Denton, Texas
Jennifer T. Mascolo, PsyD, Department of Psychology, St. John’s University, Jamaica, New York
Nancy Mather, PhD, Department of Disability and Psychoeducational Studies, College of Education, University of Arizona, Tucson, Arizona
Robb N. Matthews, MA, Department of Educational Psychology, Texas A&M University, College Station, Texas
R. Steve McCallum, PhD, Department of Educational Psychology and Counseling, University of Tennessee, Knoxville, Tennessee
George McCloskey, PhD, Department of Psychology, Philadelphia College of Osteopathic Medicine, Philadelphia, Pennsylvania
Kevin S. McGrew, PhD, Institute for Applied Psychometrics, St. Cloud, Minnesota
David E. McIntosh, PhD, Department of Educational Psychology, Ball State University, Muncie, Indiana
Daniel C. Miller, PhD, Department of Psychology and Philosophy, Texas Women’s University, Denton, Texas
Ryan Murphy, EdS, Department of School Psychology, Philadelphia College of Osteopathic Medicine, Philadelphia, Pennsylvania
Joanna L. Mussey, MA, Department of Psychology, University of Alabama, Tuscaloosa, Alabama
Joshua Nadeau, MS, Department of Pediatrics, University of South Florida, Tampa, Florida
Jack A. Naglieri, PhD, ABAP, Department of Psychology, George Mason University, Fairfax, Virginia
Juliana Negreiros, MA, Department of Educational and Counseling Psychology, University of British Columbia, Vancouver, British Columbia, Canada
Bradley C. Niebling, PhD, Midwest Instructional Leadership Council, Urbandale, Iowa
Salvador Hector Ochoa, PhD, Department of Educational Psychology, University of Texas–Pan American, Edinburg, Texas
Sarah E. O’Kelly, PhD, Department of Psychiatry and Behavioral Neurobiology, University of Alabama, Tuscaloosa, Alabama
Samuel O. Ortiz, PhD, Department of Psychology, St. John’s University, Jamaica, New York
Tulio M. Otero, PhD, School Psychology Program, Chicago School of Professional Psychology, Chicago, Illinois
Eric E. Pierson, PhD, NCSP, HSPP, Department of Educational Psychology, Ball State University, Muncie, Indiana
Mark Pomplun, PhD, Riverside Publishing, Itasca, Illinois
Tara C. Raines, PsyS, Gwinnett County Public Schools, Gwinnett County, Georgia
Cecil R. Reynolds, PhD, Department of Educational Psychology, Texas A&M University, College Station, Texas
Matthew R. Reynolds, PhD, Department of Psychology and Research in Education, University of Kansas, Lawrence, Kansas
Cynthia A. Riccio, PhD, Department of Educational Psychology, Texas A&M University, College Station, Texas
Alycia M. Roberts, BA, Department of Psychology, University of South Carolina, Columbia, South Carolina
Jane Rogers, PsyD, Department of Psychology, Philadelphia College of Osteopathic Medicine, Philadelphia, Pennsylvania
Gale H. Roid, PhD, Department of Institutional Research, Warner Pacific College, Portland, Oregon
Ellen W. Rowe, PhD, Center for Psychological Services, George Mason University, Fairfax, Virginia
Andrea N. Schneider, BA, Department of Psychology, University of Victoria, Victoria, British Columbia, Canada
W. Joel Schneider, PhD, Department of Psychology, Illinois State University, Normal, Illinois
Fredrick A. Schrank, PhD, Woodcock–Munoz Foundation, Olympia, Washington
Jennie Kaufman Singer, PhD, College of Health and Human Services, Sacramento State University, Sacramento, California
Marlene Sotelo-Dynega, PhD, Department of Psychology, St. John’s University, Jamaica, New York
Robert J. Sternberg, PhD, Provost, Oklahoma State University, Stillwater, Oklahoma
Dustin Wahlstrom, PhD, The Psychological Corporation, San Antonio, Texas
John D. Wasserman, PhD, Department of Psychology, George Mason University, Fairfax, Virginia
Lawrence G. Weiss, PhD, The Psychological Corporation, San Antonio, Texas
Barbara J. Wendling, MA, Consulting Services, Dallas, Texas
James Whitaker, PsyD, Department of Psychology, Philadelphia College of Osteopathic Medicine, Philadelphia, Pennsylvania
Gabrielle Wilcox, PsyD, Department of School Psychology, Philadelphia College of Osteopathic Medicine, Philadelphia, Pennsylvania
Anne Pierce Winsor, PhD, Department of Educational Psychology, University of Georgia, Athens, Georgia
Kirby L. Wycoff, EdM, NCSP, Graduate School of Applied and Professional Psychology, Rutgers, The State University of New Jersey, Piscataway, New Jersey
Megan Yim, BA, Department of Psychology, Victoria Island University, Victoria, British Columbia, Canada
Jianjun Zhu, PhD, The Psychological Corporation, San Antonio, Texas

GUILFORD PUBLICATIONS, INC.
72 Spring Street, New York, NY 10012
Tel: (212) 431-9800​Toll Free: (800) 365-7006
Fax: (212) 966-6708​E-mail: info@guilford.com
Visit our website: www.guilford.com


- iPost using BlogPress from Kevin McGrew's iPad


Generated by: Tag Generator

Sunday, March 27, 2011

IAP Applied Psychometrics 101 Report #10: "Just say no" to averaging IQ subtest scores

Should psychologists engage in the practice of calculating simple arithmetic averages of two or more scaled or standard scores from different subtests (pseudo-composites) within or across different IQ batteries? Dr. Joel Schneider and I, Dr. Kevin McGrew say "no."

Do psychologists who include simple pseudo-composite scores in their reports, or make interpretations and recommendations based on such scores, have a professional responsibility to alert recipients of psychological reports (e.g., lawyers, the courts, parents, special education staff, other mental health practitioners, etc.) of the potential amount of error in their statements when simple pseudo-composite scores are the foundation of some of their statements? We believe "yes."

Simple pseudo-composite scores, in contrast to norm-based scores (i.e., composite scores with norms provided by test publishers/authors--e.g., Wechsler Verbal Comprehension Index), contain significant sources of error. Although they have intuitive appeal, this appeal cloaks hidden sources of error in the scores---with the amount of error being a function of a combination of psychometric variables.

IAP Applied Psychometrics 101 Report #10 addresses the psychometric issues involved in pseudo-composite scores.

In the report we offer recommendations and resources that allow users to calculate psychometrically sound pseudo-composites when they are deemed important and relevant to the interpretation of a person's assessment results.

Finally, understanding the sources of error in simple pseudo-composite scores provides an opportunity for practitioners to understand the paradoxical phenomenon frequently observed in practice where norm-based or psychometrically sound pseudo-composite scores are often higher (or lower) than the subtest scores that comprise the composite. The "total does not equal the average of the parts" phenomenon is explained conceptually, statistically, and via an interesting visual explanation based on trigonometry.



Abstract

The publishers and authors of intelligence test batteries provide norm-based composite scores based on two or more individual subtests. In practice, clinicians frequently form hypotheses based on combinations of tests for which norm-based composite scores are not available. In addition, with the emergence of Cattell-Horn-Carroll (CHC) theory as the consensus psychometric theory of intelligence, clinicians are now more frequently “crossing batteries” to form composites intended to represent broad or narrow CHC abilities. Beyond simple “eye-balling” of groups of subtests, clinicians at times compute the arithmetic average of subtest scaled or standard scores (pseudo-composites). This practice suffers from serious psychometric flaws and can lead to incorrect diagnoses and decisions. The problems with pseudo-composite scores are explained and recommendations made for the proper calculation of special composite scores.


- iPost using BlogPress from my Kevin McGrew's iPad

Generated by: Tag Generator





Sunday, January 02, 2011

Hot and cold CHC intelligence abilities--Gf,Gc,Gv hot--Ga,Glr cold

Interesting article in the journal Intelligence reviewing the state-of-the-art of factor analysis practices for identifying the g (general intelligence) factors. Abstract is below. Of interest is the use of the CHC framework to classify the type of broad CHC factor indicators found in the research synthesis.

Not unexpectedly, Gf, Gc, and Gv were found most often in IQ factor analysis research, followed by Gq, Gs and Gsm. Abilities that appear underrepresented in IQ factor analysis g research are the domains of Glr and Ga.

However, a couple of major caveats. The literature review was primarily adult samples. There has been considerable factor analysis activity with tests in childhood and adolescent samples that might increase the proportion of Glr and Ga indicators. Also, the authors did not include journals favored by those doing research in school psychology, special education, and speech and language---fields of study that most likely have published more studies in the under-represented CHC domains.

Never-the-less......the general trends are not surprising.

Clicking on images should take you toe larger versions.






- iPost using BlogPress from my Kevin McGrew's iPad


Thursday, December 02, 2010

IQ test battery publication timeline: Atkins MR/ID Flynn Effect cheat sheet

As I've become involved in consulting on Atkins MR/ID death penalty cases, a frequent topic raised is that of norm obsolescence (aka, the Flynn Effect). When talking with others I often have trouble spitting out the exact date of publication of the various revisions of tests, as I keep track of more than just the Wechsler batteries (which are the primary IQ tests in Atkins reports). I often wonder if others question my expertise...but most don't realize that there are more IQ batteries out there than just the Wechsler adult battery....and, in particular, a large number of child normed batteries and other batteries spanning childhood and adulthood. Thus, I decided to put together a cheat sheet for myself..one that I could print and have in my files. I put it together in the form of a simple IQ battery publication timeline. Below is an image of the figure. Double click on it to enlarge.

An important point to understand is that when serious discussions start focusing on the Flynn effect in trial's, most often the test publication date is NOT used in the calculation of how obsolete a set of test norms are. Instead, the best estimate of the year the test was normed/standardized is used, which is not included in this figure (you will need to locate this information). For example, the WAIS-R was published in 1981...but the manual states that the norming occurred from May 1976 to May 1980. Thus, in most Flynn effect discussions in court cases, the date of 1978 (middle of the norming period) is typically used. This makes recall of this information difficult for experts who track all the major individually administered IQ batteries.

Hope this helpful...if nothing else...you must admit that it is pretty :)  Click on image to view





- iPost using BlogPress from my Kevin McGrew's iPad

Thursday, November 18, 2010

Why the ASVAB should not be used in Dx of MR/ID: IAP Applied Psychometrics 101 # 9 report




I'm pleased to announce the posting of IAP Applied Psychometrics 101 Report No: 9 is now available. In addition to the written text, an appendix includes a dozen figures intended to help readers understand the fundamental and important difference between intelligence and aptitude batteries.

The Armed Services Vocational Aptitude Battery (ASVAB): Why it should not be used to in the determination of a diagnosis of mental retardation / intellectual disability Click here to access (due to the figures it is relatively large...weighing in at 7+ MB)

Abstract

ASVAB scores are often incorrectly interpreted as a measure of general intellectual functioning in the context of determining if a person is an individual with (or without) mental retardation (MR) / intellectual disability (ID). The ASVAB is an aptitude battery and not an intelligence test battery. Although the differentiation between aptitude and intelligence test batteries measures may (at times) sound fuzzy, the distinction between the two is critically important, particularly regarding how the different respective batteries are designed, the abilities they each measure, and how the resultant scores should be validly interpreted. The aptitude -intelligence test battery distinction is clearly defined in psychological measurement fields. Although aptitude and intelligence batteries often measure some overlapping abilities, the ASVAB-as-an-aptitude measure is often confused with the incorrect interpretation of the ASVAB-as-IQ (general intelligence) measure. This report explains the distinction.

- iPost using BlogPress from my Kevin McGrew's iPad



- iPost using BlogPress from my Kevin McGrew's iPad

Wednesday, October 13, 2010

Dr. Detterman's intelligence bytes: Study of intelligence a major social science success

One of the leading scholars and role models in the field of intelligence research is Dr. Doug Detterman. Recently I was honored to review a number of chapters of a major book on intelligence he is writing. I was extremely impressed with the readability of the draft. It will serve as an excellent and comprehensive introduction to the wide-ranging field of human intelligence.

I recently asked Dr. Detterman if I could make a series of posts of select quotes from his draft manuscript. Being the obvious educator he is, he was pleased to grant such permission. Thus, starting today I plan to make regular (as regular as I can....hopefully at least once a week) posts with intriguing or informative direct (brief) quotes from the draft chapters I have in my possession. This is the first in the series.

I have no financial interest in Dr. Detterman's book, but in the spirit of any possible "conflict of interest disclosure" I must mention that I received a very small honorarium to review sections of his book. My goal is simple - to allow readers to learn little tidbits in advance of this major work. From what I have read, most novices and experts in the field of intelligence will want to own this book once it reaches publication with a yet to be named publisher. Kudos to Dr. Detterman.

Below is the first paragraph in the draft "introduction" section.

Intelligence is the best-understood and most powerful variable in the social sciences. Sophisticated psychometric methods, developed largely for intelligence tests, are used to construct and assess modern tests. A substantial commercial industry has grown up around the development and sale of tests. Intelligence can be measured with better reliability than any other social science variable. Huge amounts of data have been collected using the tests and these data span the last century providing a database unavailable in most other areas in the social sciences. Data from individuals show strong relationships to many important social outcome measures including educational achievement, occupational success, income, and death, to name a few. The data collected also provide important information about the genetic, biological, and environmental origins of intelligence. Relationships at the country level have also been investigated showing that countries with higher mean IQs show a greater gross domestic product per person, are less religious, and show higher levels of democracy. In short, by any objective standard, intelligence is the social science success story of unrivaled proportions




- iPost using BlogPress from my Kevin McGrew's iPad

Tuesday, January 05, 2010

The Wechsler-like IQ subtest scaled score metric: The potential for misuse, misinterpretation and impact on critical life decisions---draft report in search of feedback




The following are the first three paragraphs (and a critical figure) of a draft of an IAP Applied Psychometrics 101 Brief Report (#5).  The complete report can be download in PDF format by clicking here.  A web-page version of the complete report can be found by clicking here (note - the web page verision may NOT display two embedded figures....viewing the PDF copy may be necessary)

I'm providing this initial draft report with the expressed intent of soliciting feedback and comments regarding the accuracy and soundness of my analyses and logic.  I'm looking for critical feedback to improve the report.  This is a draft report that will be revised if comments suggest important changes.  Please read it in the spirit of "tossing out some critical ideas" for reflective analysis and feedback.  Feedback can be sent directly to me (iap@earthlink.net) or could be provided in the form of listserv thread discussions at the NASP and/or CHC listservs.


I've recently been skimming James Flynn's new book (What is Intelligence:  Beyond the Flynn Effect) to better understand the methodology and interpretation of the Flynn effect. Of particular interest to me (as an applied measurement person) is his analysis of the individual subtest scores from the various Wechsler scales across time. As most psychologists know, Wechsler subtest scaled scores (ss) are on a scale with a mean (M) = 10 and a standard deviation (SD) = 3. The subtest ss range from 1 to 19.  In Appendix 1 of his book, Flynn states "it is customary to score subtests on a scale in which the SD is 3, as opposed to IQ scores which are scaled with SD set at 15. To convert to IQ, just multiply subtest gains by five, as was done to get the IQ gains in the last column."  At first glance, this statement makes it sound as if the transformation of subtest ss to IQ SS is an easy (“just multiply….”; emphasis added by me) and mathematically acceptable procedure without problems. However, on close inspection this transformation has the potential to introduce unknown sources of error into the precision of the transformed SS scores.  It is the goal of this brief technical post to explain the issues involved when making this ss-to- IQ SS conversion.

The ss 1-19 scale has a long history in the Wechsler batteries. For sample, in Appendix 1 of Measurement of Adult Intelligence (Wechsler, 1944), Wechsler described the steps used to translate subtest raw scores to the new ss metric. The Wechsler batteries have continued this tradition in each new revision, although the methodology and procedures to calculate the ss 1-19 values have become more sophisticated over time.   Although the methods used to develop the Wechsler ss 1-19 scale may have become more sophisticated, the resultant underlying scale for each subtest has not…scores still range from 1-19 (M=10; SD=3).  Also, the most recent Stanford-Binet—5th Edition (SB5; Roid, 2003) and Kaufman Assessment Battery for Children-2nd Edition (KABC-II) have both adopted the same ss 1-19 scale for their respective individual subtests.

Why is this relatively crude (to be defined below) scale metric still used in some intelligence batteries when other contemporary intelligence batteries provide subtest scale metrics with finer measurement resolution?  For example, the DAS-II (Elliott, 2007) places individual test scores on the T-scale (M=50; SD=10), with scores that range from 10-90.  The WJ III (McGrew & Woodcock, 2001) places all test and composite scores on the standard score (SS) metric associated with full scale and composite scores (M=100; SD=15).  The critical question to be asked is “are there advantages or disadvantages to retaining the historical ss 1-19 scale or, are their real advantages to having individual test scales with finer measurement resolution (DAS-II; WJ III)?”

......continued............
(complete report available at links in first paragraph of this post)

[Double click on image to enlarge]





Technorati Tags: , ,, , , , , , , , , , , , , , , , , , , , , , , , , ,


Thursday, November 19, 2009

Can a person with mild MR/ID fail to be Dx'd before age 18: Do Forrest Gumps exist?

Readers interested in the issue of IQ testing in Atkins MR/ID death penalty cases may find the above titled post of interest at IQs Corner sister blog--ICDP.

Sunday, November 08, 2009

What does the WAIS-IV measure ? Applied Psychometrics 101 Report 2


What does the WAIS-IV measure?:  CHC analysis and beyond.

IAP AP101 # 2 report is now available (click here for other AP101 reports and briefs).  "IAP AP101 IQ TEST SCORE DIFFERENCE SERIES #2: What does the WAIS-IV measure? CHC analysis and beyond" can be viewed  or downloaded by clicking here.

The PPT files are also viewable and downloadable via SlideShare.

Abstract
The WAIS-IV (2008) is the latest revision of the adult Wechsler battery. The addition of new, and deletion of old tests, plus a more-factor based foundation for the composite indexes, requires psychologists to be familiar with the best possible interpretative structure of the venerable battery. In this PowerPoint based report, the available published and unpublished confirmatory factor studies of the WAIS-IV subtests are summarized. They are then augmented via a series of new exploratory data analysis of the WAIS-IV. It is concluded that the currently available structural research argues for a CHC-based organization of WAIS subtest scores that differs from the suggested structure provided by the test publisher. In addition, exploratory methods, when combined with similar analysis of the WJ III battery, provide support for possible intermediate level CHC dimensions (between g and the Gf-Gc broad abilities) in the contemporary CHC theory of intelligence.

Technorati Tags: , , , , , , , , , , , , , , , , , , , , , , , ,


Friday, November 06, 2009

The complex issues in MR/ID Atkins death penalty cases: APA Div 33 Ad Hoc Committee on Mental Retardation 2006 list

As I continue to manage this blog, it has become increasingly clear that the psychological and legal issues involved in Atkins MR/ID death penalty cases are very lenghty and complex.  Last night I skimmed a Div 33 Newsletter article (Vol 31, Issue 2, 2006 - click here to find way to obtain copy) that had an article that described the role and function of the Ad Hoc Committee on Mental Retardation and the Death Penalty.  Although the list of issues was generated in 2006, most all still seem pertinent today.  I've reproduced the list of issues below.  Of course, the 1992 AAMR manual mentioned has since been replaced by the 2002 10th and just recently the 2009 11th edition.

Clearly there is much to address, discuss, research, resolve.  I get tired and overwhelmed just reading the list.  I can only hope that the ICDP blog contributes in some small way to the resolution of some of these issues:
 
Qualifications of experts
• Qualifications of psychologists (Licensure? Experience in mental retardation? Forensic training?)
• Qualifications of other experts (In the Atkins trial in Virginia, only a psychologist could testify regarding diagnosis, although other experts testified about other issues.)
• Use of lay experts (How do psychologists obtain and use information from lay persons who know the defendant?) with a background in mental retardation and little forensic experience.)
• Protection of raw test data and test protocols (Ethical principles 9.04 and 9.11)
Issues related to definition of mental retardation
• The Supreme Court used the 1992 American Association on Mental Retardation definition. Should a psychologist offer best interpretation and application of that definition or make suggestions to states for new definitions that are congruent with Atkins?
• Since the 1992 definition is widely used in Atkins hearings, is there consensus on the meaning of the 10 areas of adaptive behavior as they apply to forensic circumstances? (They are only briefly described in the 1992 AAMR manual.) Is a new manual needed that addresses forensic procedures?
• Use of terms: mild, moderate, severe, borderline
• Use of the mental age concept
• Times at which mental retardation must be determined: childhood, time of the crime, present (This question will be determined differently by different courts.)
• Issue of taking an isolated example of high skill to show no mental retardation (e.g., language use, driving a car)
Issues Related to Retrospective Diagnosis
• Role of archival information
• School records; missing or incomplete records; problems in interpretation of old school records
• Lack of Special Education documentation in some records (It is policy for some school systems to delete this information.)
• Interpretation of past testing reports
• Standards for interpretation of old tests (Are they interpreted by the norms of their time, or can they be re-interpreted based on more contemporary psychometric methods?)
• Proper methods to obtain information from informants
• Possible biases in the memories of informants Ethical/Professional Issues
• Independence of the evaluation vs. advocacy role (This is not unique to Atkins cases, but it may be a confusing issue for psychologists Measurement of Intelligence
• Choosing a proper test
• Qualifications of tester
• Firm IQ cutoffs vs. ranges (Law may make no mention of range or variability of scores.)
• Global IQ vs. subtest scores (Are subtest scores or Verbal or Performance IQs relevant?)
• Interpretation of group tests and short forms. Should they be considered at all?
• Attempts to alter or reinterpret IQ scores after the fact (e.g., He had higher potential. He could have done better. His “true” IQ is 5 points higher.). Invalid score vs. altered or reinterpreted score.
• Interpretation of variability of scores with repeated testing.
• Flynn effect (Does it apply to individual scores?)
• Practice effect Adaptive Behavior
• Is adaptive behavior measured solely by functioning, or can potential be considered?
• Interpretation of cultural factors
• Value of multiple respondents
• Value of multiple sources and types of information (e.g., archival, anecdotal, standardized instruments, direct observation)
• Is it valid to use the events of the crime to show adaptive behavior?
• Problems assessing adaptive behavior in prison.
• Role of clinical interview and observation in prison
• Is it valid to use maladaptive behavior as evidence of adequate adaptive behavior?
• Is it valid to argue against a diagnosis of mental retardation by interpreting low adaptive behavior as a behavior disorder, not retardation? (“It’s not mental retardation; it’s a conduct disorder.”)
• Is reliance on others or need for support a key characteristic of mental retardation in adults? (This factor is only indirectly assessed in tests of adaptive behavior.)
• Is naïveté or gullibility or susceptibility to undue influence of others a critical characteristic of mental retardation? This factor is emphasized in the literature on mental retardation (e.g., Greenspan, 1999) but not in tests of adaptive behavior.
• Role of standardized tests of adaptive behavior
• Can standardized tests of adaptive behavior (e.g., SIB-R, ABAS-II) be given retrospectively? What adaptations are acceptable?
• Is client self-report of adaptive behavior valid (either anecdotally or in response to the ABAS)?
• Is the Street Survival Skills Questionnaire a valid instrument for diagnosing mental retardation?
• Role of anecdotal information from informants
• Bias in information from informants (possible bias in either direction)
• Lack of visible stigmata
• Association with conditions of poverty
• Heritable component Responsibility of psychologist in educating the court about characteristics of mild mental retardation
• Variability in skills
• “Cloak of Competence” - influence of trying to look good
• Mental retardation vs. mental illness
• Mental retardation not defined by etiology but by functioning Assessing malingering
• Appropriate methods and instruments


Technorati Tags: , , , , , , , , , , , , , , , , , , , ,