EPIDEMIOLOGICAL

DATA COLLECTION METHODS

Presented by: Sahar Soomro

 DATA
• Strictly speaking, data is the plural of datum, a single piece of
information.
• Data are distinct pieces of information, usually formatted in a
special way.
• In database management systems, data files are the files
that store the database information.
• Research data is data in any format or medium that is collected,
observed, or created, for purposes of analysis to produce original
research results.
• Data vs. information
o Data are plain facts. When data are processed,
organized, structured or presented in a given context
so as to make them useful, they are called
 DATA COLLECTION
- SELECT DEFINITIONS-

• The process of acquiring subjects and/or gathering information –so-called

“research data”-needed for a study.
• The process of gathering and measuring information on targeted
variables in an established systematic fashion, which then enables one to
answer relevant questions and evaluate outcomes.
• While methods vary by discipline and study design, the emphasis on
ensuring accurate and honest collection remains the same.
• The goal for all data collection is to capture quality evidence that then
translates to rich data analysis and allows the building of a convincing and
credible answer to questions that have been posed.
• The ongoing systematic collection, analysis, and interpretation of health
data necessary for designing, implementing, and evaluating public health
prevention programs. (WHO)
RESEARCH METHODOLOGY

The methodology section is an outline of the

broader strategy of your research approach

Methodology does not equal methods

3 key components: selecting the overall research

and analysis methods, selecting the appropriate
data collection approach (es), designing the
sampling strategy.
 RESEARCH METHODS
Quantitative Qualitative research Mixed methods
research
• Positivism; reality can • Interpretive framework, • Combines and
be measured and multiple
integrates elements
observed objectively realities shaped by personal
viewpoints, contexts of both quantitative
• Deductive appraoches and meanings and qualitative
• Minimize confounding
• rich descriptions of views,
beliefs and meanings and
approaches in one
experiences study (e.g.,
• Large N 🡪 • Acknowledges the role of the viewpoints, data
generalizability researcher when shaping collection methods,
and producing data
• Dominant research analysis and
• Inductive nature; hypotheses
paradigm in health emerge from data; robust inference techniques)
research but difficult to theory for the purpose of
grasp: • Interpretative and breadth and depth of
less standardized:
o Practices of actors o small N understanding and
o Individual meaning o Not representative corroboration
o Context of action (in a statistical
sense)
o New, o Difficult to test
hypotheses
 RESEARCH METHODS
Quantitative Qualitative research Mixed methods
research

• Experimental vs. • Case study • Many possible

observational designs designs based on
• Ethnography timing, weighting,
• Randomized-
• Grounded theory and mixing of
controlled trials
methods used
(RCTs) • Narrative o Concurrent/
• Non-randomized approaches parallel;
analytical studies sequential
• Phenomenology o Equal weight or
(case- control, cohort
• Qualitative QUANT- qual/
or cross- sectional QUAL-quant
studies) description o Merge, embed or
• Descriptive studies connect
DEDUCTIVE VS. INDUCTIVE
APPROACHES

UMEÅ
UNIVERSITY
 SELECTING RESEARCH AND
DATA COLLECTION METHODS

Quality Which factors to consider?

✔ Quality of data and its overall
applicability to meet research
objectives
✔ Time, e.g., funding project
and timeline
✔ Resources available
Time Resource
✔ Material resources
s
✔ Financial resources
✔ Human resources
 UNIT OF ANALYSIS
• The unit that will be used to
record, measure and analyze the
collected data
• The choice of unit will impact the
time, resources needed to collect
and analyse data

• Unit will define the depth of

information and the scope of analysis
• Examples:
⮚ Individual?
⮚ Family? Household?
⮚ Community?
⮚ Village?
⮚ Facilities? (schools, hospitals)
⮚ National?
⮚ Environment? UMEÅ Ref: Ahrens, Handbook of Epidemiology
UNIVERSITY (2007)
 UNIT OF ANALYSIS

Family/ Household Level

Individual Level

Village level
information

Community, Group, Facility Level

Depth of

National level

Time / Cost / Ac cess

Source:
 PRIMARY VS. SECONDARY DATA

Primary data Secondary data (“desk

research”)
(“field research”)
• Data which has been collected for another
• Tailored data collected by the purpose by someone other than the user.
user of that data. • It plays an important role in the
• Original and raw. exploratory phase, helping to define key
issues.
• Advantages: targeted; precise,
• Advantages: ease of access; cost and
current; known methodology;
time-efficient; often longitudinal; variety of
• Disadvantages: expensive and sources; often large N
time consuming • Disadvantages: specific to your purpose?
Outdated? No control over data quality;
biased towards data collectors?

UMEÅ
UNIVERSITY
HIERARCHY OF EVIDENCE
 Quantitative Qualitative

1. Type of Confirmatory, Exploratory,

paradigm and deductive, objective, inductive, subjective,
analysis numeric-based narrative- based

2. Type of Structured, close- Semi-structured open-

data ended data
ended data collection
collection collection tools
tools

3. Type of Can use both Non-probability sampling

sampling strategy probability or non-  generalization to the
probability wider population not
sampling
possible (in a
 generalization to
statistical
the wider
UMEÅ
sense)
UNIVERSITY
 Quantitative Research Qualitative Research
Using E.g., E.g., text analysis from
available vital/civil/population policy or program
statistics; medical documentation
information
records or
patient/hospital
registers; etc.

Observations E.g., checklists, taking E.g., ethnographic

measurements or observation methods
samples

Interviews E.g., structured E.g., In-depth or key

interviews using informant interviews;
questionnaires focus group discussions
(FGDs)

Self- E.g., postal or (Open-ended questions)

administered electronic
UMEÅ
surveys
questionnaires UNIVERSITY
 QUANTITATIVE DATA COLLECTION

Basic distinction:
• Collected through ‘conversation’ or
observation
o primary vs. secondary data
o Probability or non-probability sampling
techniques

1. Structured closed-ended survey

approaches
2. Structured close-ended observational
 STRUCTURED SURVEY APPROACHES

• Collects quantifiable cross-sectional or longitudinal

data through a ‘conversation’
• Examples:
o Individual surveys
o Household surveys
o [Key informant interviews or group discussions
(e.g. community or group level)]
 INTERVIEWS
Modes: Be aware:
✔ Literacy independent
• PAPI: verbal interviews, face-to-face, using ✔ Allows clarification of
traditional paper and pencil interview
questions
• CAPI: verbal interviews, face-to-face using ✔ Higher response rate
computer-assisted personal interviewing ✔ More in-depth information
methods via personal computer or lap top ✔ Reporting bias (“social
questionnaire programs
desirability”)
• CATI: verbal interviews, by telephone, o Interviewer effect  importance of
using training
paper or electronic computer assisted o Not good for sensitive questions
questionnaires o Time-consuming field work
• Remote data collection techniques
 SELF-ADMINISTERED QUESTIONNAIRES

• Modes of administration:
o PAPI: traditional paper and pencil self-administration “interview”
methods by post, or handing paper questionnaires to people in
person and asking them to complete them by hand and return
them to the researcher
o CASI: computer-assisted (electronic) self-administration
“interview” methods by automated electronic, including audio
computer-assisted methods
o ACASI: self-administration via interactive voice response methods
with automated computer-assisted
Advantages: telephone program.
Disadvantages
• Less expensive and • Literacy dependent
quick • Low response rate (follow
• Good for long ups)
questions • Misunderstanding of
• No effect of the questions Less bias? Who
interviewer answers?
• Can show pictures, maps,
 REMOTE DATA COLLECTION

• Gathering data without • When is it useful?

a physical presence o Disease outbreaks (e.g.,
in data collection Covid-19)
location and without o Time and/or resource
direct, in- person constraints
contact with study o Access constraints
population ▪ Security concerns
▪ Travel restrictions
▪ Lack of infrastructure Etc.
 STRUCTURED OBSERVATIONS

• Data collected through observations using structured, close-ended checklists to

collect quantifiable data
o E.g., specific object, behavior or event
▪ ‘medicine cabinets’; use of soap; attendance; equipment or facilities

• Provides detailed additional contextual information needed to frame for example

an evaluation and helps making sense of data collected using other methods
• Be aware of:
o Confidentiality and ethical issues!
o Observer bias (unconscious assumptions or preconceptions harbored by the
researcher)
o Hawthorne effect (observer effect) is a type of reactivity in which
individuals modify or improve an aspect of their behavior in response to
their awareness of being observed)
o Time consuming: Need for intensive staff training
 ‘TOTAL SURVEY ERROR’ PERSPECTIVE

Groves RM, Fowler FJ, Couper MP, Lepkowski JM, Singer E, Tourangeau R. (2009). Survey
 QUESTIONNAIRE DESIGN

• Main objectives in designing a questionnaire:

o To maximize the proportion of subjects answering our questionnaire—that
is, the response rate.
o To obtain accurate relevant information for our survey.
• To maximize our response rate, we have to consider carefully how we
administer the questionnaire, establish rapport, explain the purpose of the
survey, and remind those who have not responded.
• In order to obtain accurate relevant information, we have to give some thought to
what questions we ask, how we ask them, the order we ask them in, and the
general layout of the questionnaire.
🡪 Questionnaire design and development must be supported by a
logical, systematic, and structured approach!
 WHAT TO ASK?

• Information we are primarily interested in —that is,

dependent variables.
• Information which might explain the dependent variables—
that is,
independent variables.
• Other factors related to both dependent and independent
factors which may distort the results and have to be adjusted
for—that is, confounding variables.
 QUESTIONS
-TYPES AND RESPONSE OPTIONS-

• Simple Yes/No question • The questionnaire must meet

the needs of…
• Running prompt questions
o Researcher
• Closed – one choice
o Respondent
• Closed – “mark all that applies”
o Interviewer
• Multiple response – “mark all
that applies” o Data entry staff (think

• Filter questions (“If YES, about the coding of data

continue with question XX) beforehand!)
• Open (-ended) questions 🡪 Importance of a priori study
(“Please tell us what you like protocol and pilot testing
best about…”)
 A CATALOGUE OF BIASES IN
QUESTIONNAIRES
(TAKEN FROM CHOI, 2005)

IN QUESTION DESIGN
o Wording (ambiguity, complexity, double-barreled questions,
uncommon words, vague words)
o Missing or inadequate data for intended purpose (belief vs.
behavior, starting time, degraded data)
o Faulty scale (forced choice, missing or overlapping interval, scale
format)
o Leading questions (framing, leading question, mind-set)
o Intrusiveness (reporting, sensitive questions)
o Inconsistencies (case definition, change of scale, change of
wording, diagnostic vogue)
 A CATALOGUE OF BIASES IN
QUESTIONNAIRES
(TAKEN FROM CHOI, 2005)

IN QUESTIONNAIRE DESIGN
• Poor formatting (horizontal format, juxtaposed scales,
alignment)

• Too long (Nay/Yea saying, open questions, response fatigue,

flawed structure, skipping questions)
 A CATALOGUE OF BIASES IN
QUESTIONNAIRES
(TAKEN FROM CHOI, 2005)

IN THE ADMINISTRATION OF THE

QUESTIONNAIRE
• Interviewer is not objective ( inter-interviewer or intra-interviewer error;
non-blinding)

• Respondent’s subconscious reaction (end aversion, positive satisfaction)

• Respondent’s conscious reaction (faking bad, faking good (= social

desirability), unacceptable disease or exposure, unacceptability, underlying
cause)

• Respondent’s learning (learning, hypothesis guessing)

• Respondent’s inaccurate recall (primacy vs recency, proxy

respondent/surrogate data, recall, telescope)

• Cultural differences!
 GOOD QUESTIONS?
1)How often do you smoke? 4) How is your health in general?
❑ Regularly ❑ Very good
❑ Occasionally
❑ Never ❑ Good
❑ Bad
2) Which age group do you belong ❑ Very bad
to?
❑ Under 18 5) What is your income?
❑ Over 18
❑ Over 50 6) Now we move on to question number
512 regarding the health of your
3) Being healthy and helping people goldfish…
is important in life.
❑ Yes
❑ No
 BASIC PSYCHOMETRICAL CONCEPTS
VALIDITY
ERROR
• ‘measuring what you think you’re
o Random measurement error
measuring’.
(variability, “noise in the system”)
• Content, criterion and factorial validity
o Systematic (non-random)
measurement error (=Bias)
RELIABILITY
• the ability of the questionnaire to produce
the same results under the same conditions
• test-retest reliability

DISRCIMINATION
• people with identical numerical scores are
identical in the construct being measured
• the difference in score is proportional to the
 CROSS-CULTURAL ISSUES
• How can we do survey research if different respondents (perhaps from different
cultures, countries, or ethnic groups) understand questions in completely different
ways, or if investigators mean one thing and respondents think they mean something
else?
• One way: Anchoring Vignettes
o “Thick Description”
o Anchoring vignettes are brief texts describing a hypothetical character who
exemplifies a certain fixed level of the trait of interest. The respondent is
asked to rate the level of the trait for the vignette character as she/he would do
for his/her own.
o The vignette ratings are used to identify the problem of reporting heterogeneity
and then
adjust the self-rating response by removing its systematic variation using
either a parametric or non-parametric approach.
o Anchoring vignettes’ method has increasingly been used to improve interpersonal
and cross- cultural comparability of survey questions in areas of political
efficacy, work disability,
VIGNETTE EXAMPLE FROM WHO-
SAGE
 TAKE HOME MESSAGES
• Choice of data collection method influenced by many factors
• Questionnaire design – art or science?
• Essential epidemiological tool
• Questionnaire data are a prerequisite for drawing valid
conclusions
• Design and usage to maximize validity and reliability and minimize
error and bias
• Requires good knowledge of theory and language skills
• Consider where things can go wrong!
• Just the beginning of the journey…
 EXAMPLES OF AVAILABLE DATASETS

www.indepth-network.org
50+ HDSS in 20 countries in
Asia and Africa
www.share-project.
Ca. 400 surveys in Data collections org
90 countries https://
29 European
www.dhsprogram.c www.who.int/data/
om countries
collections

www.oecd.o www.healthdata.
rg org

Statistics National Board of www.pubmed.ncbi.nlm.nih.

Sweden Health and Welfare org
www.scb.se www.socialstyrelsen.s
e