Stanovich Notes

6.14

John Snow (Cholera) – found a naturally occurring situation that allowed him to eliminate alternative
explanations.

Elements of scientific Thinking =

 Comparison
 Control
 Manipulation

Experimental Design:

1. Isolates a variable by directly controlling the experiment’s situation

2. Isolates a variable by observing the kinds of naturally occurring situations (to test all
explanations)

Comparing conditions = to achieve a more fundamental understanding of a phenomenon.

Variable manipulation = essential when scientists need to recreate situations w/ eliminated

explanations (confounding variables).

Independent Variable ----> Manipulated Variable of Interest

Dependent Variable ------> The Effect posited by the I.V.

The best experimental design when I.V is manipulated while controlling for all other extraneous
variables affecting the situation.

Example: Goldberger manipulated a variable of interest & controlled others in his studies of
Pellagra. (He manipulated other variables too).

Random Assignment in conjunction w/manipulation defines the true Experiment

Direct manipulation creates stronger inferences

Example: snow’s design is flawed; it cannot completely rule out extraneous variables. “Why did
people sign up with one company over the other?”

Random Assignment ensures that there’s no participant Bias and participants are equal in:

 Characteristics / differences
 Behavioural
 Biological
 As well as other variables not thought about

Random Assignment (RA) = a method of assigning participants to the conditions / groups so that each
participant has an equal chance of being assigned to any of groups/conditions
  Flipping a Coin
 Computer generated table of random numbers

R.A. depends on N in experiments (the more the better) --- works well w/ (20-25) in each group

R.A. eliminates systematic bias in assignment (allows us to be confident in any conclusion about the
cause as long as the study can be replicated).

R.A. helps to replicate the experiment

As N increases, R.A ensures that the two groups are relatively matched on all extraneous variables.

Control Groups

A control group = gives value in determining the effectiveness of a treatment program, as a comparison
benchmark for non-treatment.

Controlled conditions reveal the true explanations of particular observations.

Parsimony = when two theories have the same explanatory power, the simpler theory is preferred.

Scientific experimentation breaks apart the natural correlations in the world to isolate the influence
of a single variable.

Creating special conditions to test for actual causal relationships is a key tool we can use to prevent
pseudoscientific claims.

An experiment allows stronger causal inferences than a correlational study.

Chapter 7

Prying variables apart to get to the dominant variable and manipulate it after randomization to
measure its effect in treatment/control groups.

 Naturally occurring conditions = Snow & Cholera

 Artificially occurring conditions = Goldberger & Pellagra

Prying apart variables sometimes is necessary to be brought into a lab to be observed and to control
for extraneous naturally occurring variables.

The random sample confusion = we can’t draw random samples from the population if we want to
study a treatment’s effect on depression or maladaptive behaviour.

A random sample (R.S) is not necessary in psychological research.

R.S = drawing a sample from the population in a manner that ensures that each member of the
population has an equal chance of being chosen for the sample.

Psychological testing involves:

  Theory Testing
 A convenience sample
 Random Assignment (True Experiments)

Most theory driven research seeks to test theories of psychological processes rather than to generalize
the findings to a particular real-world situation (Basic Research can be applied to real-world situations
in due time)

Directly applied research --------> Real-world application (fast) (findings of a study will be applied
directly).

Basic Research findings are applied to theory & with other findings, the theory could apply to a
particular problem (indirect application through theory)

Examples of Applications of Psychological Theory:

1. Classical & Operant Conditioning (Pavlov (dogs) + Skinner (rats)) = both in the laboratory
 Treatment of autistic children
 Treatment of maladaptive behaviour
 Treatment of obesity and alcoholism
 Management of residents in psych hospitals
 Treatment of Phobias
2. Perceptual Processes
 Radar monitoring
 Street lighting
 Cockpit design
 Cognitive effects of aging
3. Judgement & Decision Making
 Medical decision making
 Economic decision making
 Obedience to authority (officer training / military).

The college sophomore problem (CSP) = psychologists run many experiments using college sophomores
because they are convenient to study, however, they’re not representative of the college/university
population or society at large. (Results cannot be generalized)

1. Doesn’t invalidate results, but calls for more findings

2. It’s not a problem, because the processes investigated are basic
3. Replication uses diversity from a variety of universities serving all SES

This problem has been addressed by:

1. Developmental psychologists
2. Educational psychologists
3. Cognitive psychologists
Using internet college sophomores gives better results for generalization

Replication Failures are often due to subtle differences in experimental stimuli and methods (rare).

Most psychological research is of the basic type NOT the applied type.

Basic research tests theories.

Chapter 8

Exact replication = the researcher reproduces the exact circumstances of the original research

Conceptual Replications = the researcher attempts to replicate the basic idea of the research.

Multiple Analysis uses the principle of convergent evidence (stronger conclusions)

Case Studies ----> Correlational Studies -----> Experiments with manipulated variables

The complex correlational technique of partial correlation makes it possible to test whether a
particular 3rd variable is accounting for a relationship.

We should expect many different methods to be used in all psychological research, the various classes of
research techniques have different strengths and weaknesses (example: The Unresponsive Bystander
Phenomenon).

Scientists are dependent on the combined wisdom of thousands of other scientists.

Science is public. Scientific Research builds on other previous research to expand theories and
understand the world better.

Science doesn’t make breakthroughs by Einsteinian Leaps, but by building on research flaws and much
theoretical explanation, speculation, and criticism.

 No experiment in science is perfectly designed.

 Many sciences have progressed even though they were without an Einstein.

The principle of Converging Evidence “Converging Operations”

Flawed Experiment Theory Testing

To examine the patterns of flaws running through When a conceptual hypothesis survives many
the research literature, the nature of the pattern potential falsifications based on different sets of
will either support or undermine the conclusions assumptions, we have a Robust Effect.
we wish to draw.
If Flaws are the same Research is highly convergent when a series of
= experiments consistently support a given theory
Undermine Confidence while collectively eliminating the most important
competing explanations.
 If flaws are different
=
Increased Confidence
(The evidence has converged)

Converging Evidence in Psychology

Conclusions in psychology are often based on the principle of Converging Evidence.

Low diagnosticity = data that supports a given theory usually rule out only a small set of alternative
explanations, leaving many additional theories as viable candidates.

Convergence allows us to reach many reasonable strong conclusions despite the flaws in all
psychological research (Example: research on violent T.V programing & Aggression in Children).

 Epidemiological studies are ALWAYS correlational and possibility of fake links between
variables is high.
 Laboratory studies can be highly controlled, but the subjects are often animals (not humans)
 Clinical trials in hospitals use human subjects but face the Placebo Effect in real treatment
context. Expectation also creates some form of experimenter bias.

Variables that cannot be manipulated “Categorical”: Age, sex, birth order, disability, malnutrition,
disorders – are inherently correlational.

Knowledge can be useful even when it’s not certain.

Science progresses by convergence upon conclusions. The outcomes of one study can only be
interpreted in the context of the present state of the convergence on that particular issue in questions.

Gradual-Synthesis Model is best in Psychology.

Chapter 9

Multiple Causation

The fact that an outcome is determined by many different variables doesn’t reduce the importance of
any one variable that is causally related to the outcome even if the variable accounts for only a small
portion of the outcome.

Multiple Causation ------> Concept of interaction = the magnitude of the effect that one variable has
may depend on the level of another variable.
Only at the very strongest end of the continuum does a variable act in isolation.

The strongest form of causal influence is one where a variable is necessary and sufficient for producing
an effect on a dependent variable.

Chapter 11

Probabilistic Trend = it is more likely than not but does not hold true in all cases.

Example: People seem to prefer to believe that authorities are engaged in monstrous lying than
to simply admit that is uncertainty in science.

People often use a single-case of a ‘person-who’ went against a probabilistic trend in an attempt to
invalidate it (especially if the facts go against their opinions).

People use this fallacious technique in debates because they experience great difficulty in dealing
with probabilistic information. (Human thinking).

Probabilistic reasoning and the misunderstanding of psychology:

 Knowledge doesn’t have to be certain to be useful

 Forecasting group trends is very informative

Aggregate/Actuarial Prediction = the prediction of outcomes based on group characteristics.

Psychology suffers the most from the general public’s inability to think statistically.

Psychological research on probabilistic reasoning:

 To understand the answers to social/economical questions concerning societies, one must think
statistically.
 It’s important to become aware of the most common fallacies that arise when people reason
statistically.

Cognitive Illusions = even when people know the correct answer, they may be drawn to an incorrect
conclusion by the structure of the problem.

Case evidence is always probabilistic.

Fallacious reasoning = case evidence (lab test results) seem tangible and concrete to most people.

Failure to use sample size information:

 A small N is always more likely to deviate from the mean.

 A larger N always more accurately estimates a population value.
 A basic principle of Evidence Evaluation is an appreciation of the influence of Sample Size on the
Reliability of Information.

The Gambler’s Fallacy = the tendency for people to see links between events in the past and events in
the future when the two are really independent.

 Truly random sequences will often not seem random to people because of our tendency to use
patterns everywhere.
Chapter 10 = The Role of Chance

When designing, analysing, or reading about research, you should be able to form reasoned
expectations about, and draw conclusions concerning, the possible role of sampling
variability (i.e. chance) for:

 a single study
 a collection of studies which examine the same research question.

Sampling Variability = Sampling Error

CHANCE

The natural (random) variation in sample statistics when we take several samples from
a population.

The tendency to try to explain chance events:

A coin toss is a chance event because there is no easy way to measure all the variables in
the event. ------> The outcome is just currently indeterminable [not in principle
indeterminate]

Explaining Chance: Illusory Correlation + Illusion of Control

Illusory Correlations = the phenomenon of perceiving a relationship between variables

even then such a relationship doesn’t exist. (A false association maybe formed due to
rare/novel occurrences that are more salient)

 Belief in the Rorschach Test.

The Rorschach Test = a type of projective test used in psychoanalysis, in which a standard set of
symmetrical ink blots of different shapes and colours is presented one by one to the subject, who is
asked to describe what they suggest or resemble.

People tend to see their expected correlation even in random events (they see structure
where there is NONE).
The Illusion of Control <Langer> = the tendency to believe that personal skill can affect
outcomes determined by chance.

Examples:

 State Lottery
 Books that show how to beat the lottery.

Just-World Hypothesis = the tendency to believe that people live in a world in which
everyone gets what they deserve.

Chance and Psychology

Unfalsifiable psychological theories are the result from the tendency to try to explain
every bit of variability instead of leaving it to chance.

Example: Practitioners of psychohistories commit many of these errors.

 An understanding of the role of chance is critical to the lay consumer of psychological

information.
 Legitimate psychologists openly acknowledge the chance factor. (True scientists are not
afraid to admit what they do not know).
 Before accepting a complicated explanation of an event consider what part chance may
have played in its occurrence.

Coincidence = is merely an occurrence of related events that is due to chance.

 Rare events to happened purely by chance.

 Coincidences DO NOT NEED an explanation.

Oddmatch = two events whose co-occurrence strikes us as odd or strange.

 Laws of probability guarantee that ↑ events = ↑ probability of Oddmatch

occurring
 Laws of chance
1. Allow oddmatches to happen, and
2. Guarantee them in the long run.

Personal Coincidences = we think personal oddmatches to be special because they

happened to us by they are the result of mere probability!

Accepting Error in order to reduce Error: - Clinical vs. Actuarial Prediction:

The reluctance to acknowledge the role of chance when trying to explain outcomes in
the world CAN reduce our ability to predict Real-world events.
Acknowledging the role of chance in determining outcomes means we can never be
100% accurate.

 Knowing our predictions will be less than 100% = ↑ overall predictive accuracy

Therefore, we must accept error in order to reduce error.

Actuarial Prediction = Refers to predictions based on group trends derived from statistical
records. (Example: predicting the lifespan of 77.5 y.o nonsmokers and 64.3 y.o smokers).

 More accurate predictions can be made if we take more than one group
characteristic into account. (multiple regression).
Example: predicting that 58.2 years of life for overweight smokers who do not exercise
is a more accurate prediction).

Clinical prediction Do NOT WORK --- in contrast to Actuarial Predictions

 The process of examining such factors as signs, symptoms, and case histories to
determine the clinical diagnosis and likely progress of individual patients.

Actuarial Prediction “equation” > Clinical Prediction + Additional Special Information

(because the equations integrates information accurately & consistently).

 Consistency can overcome any informational advantage the clinician has from into
gleaned informally.

Failing to “accept error in order to reduce error” is when clinicians take the actuarial
equation and adjust it based on client info.

 We get more accurate prediction by using the Actuarial Approach.

 The Actuarial Approach has an advantage over clinical predictions.
 Actuarial Equations -----> Public Knowledge
 Clinical Predictions ------> Singular & Idiosyncratic (not subject to public
criticism).

Psychological Predictions = probabilistic predictions of Aggregate Trends.

10.5 Notes

Chapter 5

Why Goldberger’s evidence was better than Snow’s?

Because of controlled manipulation evidence (He ruled out whether associations were
over- or under- estimated).

We should approach Correlational Evidence with a certain skepticism.

 The 3rd Variable Problem

 The correlation between 2 variables may not indicate a direct causal path between
them, BUT may rise because both variables are related to a 3rd variable that has not
even been measured. (why we cannot infer causation from a correlation).
 Zero-Order Correlations = Spurious Correlations

Statistical Techniques that deal with the 3rd Variable Problem:

 Multiple Regression
 Partial Correlation
 Path Analysis

The progression to more powerful methods

Research on a particular problem often proceeds from weaker methods to ones that allow
more powerful conclusion:

Case Studies -----> Correlational Studies ------> Experiments w/ manipulated variables

Case studies suggest hypotheses for further study with more powerful techniques and
to motivate scientists to apply more rigorous methods to research problems.

Correlational Studies can contribute to knowledge by:

 Scientific Hypotheses are stated in terms of correlation or lack of correlation

 Correlation ≠ Causation BUT Causation = Correlation
 Partial Correlation makes it possible to test whether a 3rd Variable is accounting for a
relationship.

Other notes from Stanovich

Falsifiability = allowing for the possibility that a theory is wrong where the H0 can be
accepted.
General principles of the scientific method:

1. Science employs methods of systematic empiricism.

2. It aims for knowledge that is publicly verifiable
3. It seeks problems that are empirically solvable and that yield testable theories.

Structured and controlled observations define systematic empiricism.

Operationism Essentialism
‘ultimates’ = unanswerable Obsess & argue about meanings of words
Eliminates errors Distracts from matters of real substance
Skepticism Unfalsifiable
Relies on Intuitive Claims resulting in
Free and open pursuit of knowledge
conflicts of power struggles
Science defines concepts after extensive
investigation
Refinement of conceptual terms

= data + theory
Explanation of Phenomena is Goal
Falsifiable
Link concepts to observable events

The key progress in all sciences is to abandon essentialism and adopt Operationism.

Operationism = the idea that concepts in scientific theories must in some way be grounded
in, linked to, observable events that can be measures.

 Crucial to the construction of theory

 Important for evaluating theoretical claims in psychology.

Operationalizing concepts has to be accessible by other observers.

Example:

 Hunger via blood sugar (not feeling).

 Anxiety via Questionnaires & physiologic measurements (heartbeat, sweating).

Concepts are defined by a set of operations, not just by a single Behavioural even/task
 Example: Reading Ability ----> performance on standardized scale that
contains a whole set of tasks.

Operational Definitions force us to think carefully and empirically about how we want
to define a concepts. (Example: Typing ability ----> Speed ----> requires measuring: time,
errors, percent correct, etc).

Inter-rater Reliability = the consistency of a measuring instrument no matter who is using

it.

Test-retest Reliability = the consistency of a measuring instrument when measuring the

same concept repeatedly.

Reliability and Validity

For an operational definition of a concept to be useful, it must display both reliability

and validity.

Reliability = the consistency of a measuring instrument with replication. (Producing the

same results).

Reliability is necessary but not sufficient on its own. The operations assessed must also
be a valid measure of that concept.

Construct Validity = whether a measuring instrument’s operational definition is measuring

what it is supposed to be measuring.

The aims of a measuring instrument:

Yes No
Reliability (no enough alone) Valid but not reliable
Validity ↑ Reliability + ↓ Validity
↑ Reliability + ↑ Validity ↑ Validity + ↓ Reliability

Direct & Indirect Operational Definitions

The link between concepts & observable operations varies greatly in its degree of
directness or indirectness.

 Few scientific concepts are defined almost entirely by observable operations in the
real world.
 Most concepts are defined more indirectly
Example: the use of some concepts is determined by:
1. A set of operations, and
2. The concepts’ relationship to other theoretical constructs.
  Latent Constructs = some concepts are only linked to other concepts that are
defined by observable operations. (Common in psychology).
Example: Type-A behaviour pattern is linked to the incidence of coronary
heart disease (CHD).
Type-A Behaviour Pattern is actually defined by a set of subordinate
concepts:
(Monica Gellar – Friends)
 A strong desire to compete
 A potential for Hostility
 Time Urgent Behaviour
 Intense drive to accomplish goals
 Each one of these defining features is in itself a concept in need of
operational definition.
 Type-A Behaviour Pattern is a complex concept that is not directly
defined by operations.

Theoretical concepts differ in how closely they are linked to observations, all concepts
acquire their meaning partially through their link to such observations.

Scientific Concepts Evolve

The definition of a scientific concept is NOT FIXED, but constantly changing as the
observations that apply to the concept are enriched.

Example:

1. Intelligence was what is measured by tests of mental functioning.

But, with what empirical evidence has accumulated, the concept was
enriched and refined. So, now, Intelligence is a higher-order construct
defined by sever more specific info-processing operations.

2. Theories of human memory:

 Short-term memory
 Iconic storage
 Semantic and episodic memory
 Working memory

The use of theoretical terms evolves from scientific activity rather from debates about
the meaning of words.

 Concepts in science derive meaning not from language definitions, but from
observations and other concepts to which they are related.
As scientific concepts evolve, they often become enmeshed in several different
theoretical systems & acquire alternative operational definitions. Not unique to
psychology (Common occurrence in other sciences).

Example:

 Heat is conceptualized in : Thermodynamics Theory + Kinetic Theory

 The Electron --------------> A wave + A particle
 Pluto ----------------------> no longer defined as a planet (2006) (dwarf planet)

The different operational definitions do not reflect badly on science, they merely
reflect differing ways of triangulating concepts in the discipline.

Example:

 Genes ----> Concept = A unit of DNA that codes for a protein. BUT in recent
years however, this concept has been complexified.

It’s the normal process of doing science. You start off simple and you develop complexity.

In psychology = concepts complexify and their operational definitions shift.

Operational definitions in Psychology:

Problem 1: Pre-existing Bias Problem = people come into psychology with emotionally
held beliefs about the nature of people, making it harder for them to accept
another definition.

One of the greatest sources of misunderstanding of the accurate presentation of

psychology findings in the media is the fact that many technical concepts in psychology
are designed by words used in everyday language ( & other sciences)

Example:

1. Intelligence
When intelligence or anxiety are used in psychological theories, their
direct/indirect operational definitions determine their correct usage:
 Highly technical definitions
 Usually fairly specific
 Very different from layperson usage.
“The first principle component of the factor analysis of a
large sampling of cognitive tasks.” (part of the
operational definition of “intelligence”).
 2. Depression
Layperson -------> feeling down in the dumps
Technical --------> over a dozen pages in the DSM
3. Physics – Relativity
Layperson --------> There are no absolutes because everything is
relative!
Einstein -----------> “Although the measurements any observer makes
depends on his coordinates and reference frame, the physical
phenomena he measures have an invariant description that transcends
the observer’s particular coordinates.” (The theory is really about
finding an invariant description of Physical Phenomena).

How to fix the misunderstanding between the scientific psychology & Layperson?

Operationism is not unique to psychology. It is a humanizing force.

In science, the truth of a knowledge claim is not determined by the strength of believe of
the individual putting forth the claim. (It is made public to be replicated).

By operationally defining concepts, we put them in the public realm, where they can
be criticized, tested, improved, or rejected.

Psychology rejects all personal definitions of concepts. Just as physics rejects all personal
claims of energy, and meteorology rejects personal definitions of a cloud is.

Publicly accessible knowledge is available to solve human problems only when

concepts have become grounded in operational definitions & are not the focus of
essentialist arguments about meanings of words.

Example: Wound Shock (WWII), was identified based on:

 Abnormally high concentrations or RBC

 Low BP
 Skin Pallor
 Rapid Pulse

However, operational definitions were inconsistent, so Dr. Grant

abandoned the concept as it was doing more harm than good by
confusing physicians, and made it so observations would be made
without using the term.

Sometimes the changes in the meaning of concepts in science will put scientific
understanding of a concept in conflict with the none-specialists understanding.
 Example: Fire

 Burning carbon-stuffs
 Activity on the sun/stars
 Nuclear Fusion -----> lightning, electrically-induced
incandescence.
 Northern lights -----> spectral emission
 Flash of fireflies -----> Phosphorescence

Problem 2: Another reason many people seem to abandon the idea of Operationism
when they approach psychology is that they seek essentialist answers to
certain human problems.

The public becomes confused when uninformed critics claim that there
has been no progress in psychology.

Modern science doesn’t claim to find out what things really are.

Psychology is not a religion, it is a broad field that seeks a scientific understanding of

all aspects of behaviour.

 Psychology’s current explanations are temporary theoretical constructs that account

for behaviour better than alternative explanations.

The idea of an operational definition can be a very useful tool in evaluating the
falsifiability of a psychological theory.

 The presence of concepts that are not directly or indirectly grounded in observable
operations in an important clue to recognizing a non-falsifiable theory.
[ no observable operations = no way to replicate it ]

 The presence of loose concepts should be viewed with suspicion.

Parsimony = When two theories have the same explanatory power, the simpler theory
(fewer concepts & conceptual relationships) is preferred. (using the simplest
theory/definition).

Because, a theory with fewer conceptual relationships will likely be the more
falsifiable (testable) of the two in future tests.

A strong grasp of the principle of Operationism will also aid in the recognition of problems
or questions that are scientifically meaningless. (a concept without an operational
definition is meaningless).