Quantitative data collection: Survey

Design and Implementation

Research Methods in PH
Spring 2022
 Survey Research
• Survey research is the systematic method of gathering
information with the use of questionnaires to draw
quantitative conclusions about the respondents’ attitudes,
beliefs, opinions, behaviors, and practices.

• The survey is a non-experimental, descriptive research

method. Surveys can be useful when a researcher wants to
collect data on phenomena that cannot be directly
observed.
• The major issues related to survey research are sampling
issues and questionnaire design. These affect the accuracy,
reliability, and representativeness of the research findings.
 Survey research methods
Advantages
• Efficient for collecting large amount of information
• Statistical techniques to determine validity, reliability,
and statistical significance
• Flexible to collect wide range of information
– attitudes, values, beliefs, and past behaviours.
• Standardized - relatively free from several types of
errors
• Relatively easy to administer
• Economy in data collection
 Survey research methods

Disadvantages
• Subjects’ motivation, memory, and ability to
respond
• Not appropriate for studying complex social
phenomena
• Structured surveys, particularly those with closed
ended questions, may have low validity when
researching affective variables.
• Risk of selection and information bias
• Participants may not answer honestly
 Developing And Conducting Surveys
 Planning the Survey
• Conducting Formative Qualitative Research
 Qualitative research can provide valuable formative
information for the development of surveys or an
adjunct to surveys.
 Qualitative methods may contribute to;
*Developing study’s method (sampling &
recruitment)
*Exploring characteristics of the target population
which can be helpful for recruitment approach and
response rate.
 Developing And Conducting Surveys
Surveys may be self administered or
interviewer administered.

1. Self administered may include the following:

 mail survey and respondent returns in paper
 Telephone surveys (Touch Data entry)
 Computer assisted self interview (CASI)
 Telephone ACASI (audio computer assisted self
interview)
 Developing And Conducting Surveys

Interviewer administered interview:

Paper and pencil, face to face interview
Computer assisted personal interview (CAPI),
computer displays the question and interviewer
reads the question and records the response on
the computer
Telephone interview- interviewer calls the
respondent and records the response.
Developing a survey tool/questionnaire
Decision making process for content and survey items.
• Deciding on survey (questionnaire) content
and design can be challenging
• Ideally, research objectives, published
literature and local needs and context guide
survey development
• Researcher have the option to use existing
survey items or develop their own
– advantages and disadvantages
Questionnaires in Clinical Research
• Much of the data in clinical research is gathered using
questionnaires or interviews.
• The validity of the results depends on the quality of these
instruments.
– Good questionnaires are difficult to construct; bad questionnaires are
difficult to analyze.
• Difficult to design for several reasons:
– Each question must provide a valid and reliable measure.
– The questions must clearly communicate the research intention to the
survey respondent.
– The questions must be assembled into a logical, clear instrument that
flows naturally and will keep the respondent sufficiently interested to 9
 Quality aims in survey research
Goal is to collect information that is:

• Valid: measures the quantity or concept that is supposed to

be measured
• Reliable: measures the quantity or concept in a consistent or
reproducible manner
• Unbiased: measures the quantity or concept in a way that
does not systematically under- or overestimate the true value
• Discriminating: can distinguish adequately between
respondents for whom the underlying level of the quantity or
concept is different

10
 Developing new survey items
• Two concept of measurement are Reliability and Validity
• Reliability assess the consistency of a measure
– A weight scale reports same weight of a child when repeated 5 times
• Validity refers to having the true value as it is in fact.
– A weight scale is valid if it gives actual weight of a child

A survey question should have high reliability and high validity. There
are tests to check reliability and validity
 Developing new survey items
• The survey questions could be closed or open-
ended or both.
• Closed-ended responses are use previous
knowledge that defined a range of responses
that accurately capture necessary information
• Open-ended responses are allowed when;
– Previous research identified many responses
 Example of questions
Open-ended: When you think about public transportation in your city,
what comes in your mind?

Close-ended: In the past 24 hrs, have you used any form of public
transportation?
No… [0] (skip to #?)
Yes…. [1]
Close-ended- Multiple choice possible: What form of public
transportation you used in last 24 hrs?
Taxi… [1]
Bus…. [2]
Subway… [3]
Train… [4]
Others… [5]
None…. [6]
Close-ended-Likert scale: This type of question ask about feeling,
perception opinion etc.. Having convenient public transportation is
important to me when I decide where to live?
 Deasigning questionnaire: Wording
With closed questions, include all reasonable
possibilities as explicit response options

What is your marital

staus:
• Married • Married
• Divorced • Single
• Widowed
• Separated
• Never married
 Attention to the wording

Make the question as specific as possible

(about who it covers, what time period, which
behaviours…)

Over the last month, In a tipical

that is ….. how often week, how
do you read a often do you
newspaper in a tipical read a
week? newspaper?
 Pay attention to the wording

Use words that respondents understand better, avoid

jargons

Have you ever

Have you ever
had a heart
had a
attack?
miocardial
infarction?
 Provide memory help

Please look cerefully at

the following list of
To which
volountary volountary
organisations: which, if organisation
any, do you belong to? do you belong
to?

A Religious organisations
B Cultural organisations
C Political groups
D Other
 Attitude questions*

Measure the strength of the attitute

by using a response scale
Do you agree or disagree with the following
statement?
Government is spending too little on higher education-
1 Agree strongly
2 Agree
3 Neither agree nor disagree
4 Disagree
5 Disagree strongly

*note that an attitude have generally a direction (agree or disagree)

and an intensity (strongly disagree….strongly agree)
 Data Preparation for analysis

• Data management/preparation
– The steps necessary to convert data from a raw
form to a form which allows for analysis
• Steps in the Process
– Check Questionnaires
– Edit data
– Code Data
– Transcribe data
– Clean data
– Adjust data
 Using existing survey items
• Finding out an exiting survey questionnaire
applicable to research objectives can be
advantageous for many reason
– It save time
– Reliability and validity may been tested
Disadvantages may include:
- Questionnaire may not be fully applicable to local
context and population, which needs further
adaptation
- There might be restriction to use
 Preparing for interview

• Interviewer training
• Interview monitoring
• Automated interviewing monitoring
• Data quality monitoring
 Data Management and Analysis
• Data management and analysis plans:
– Data management includes checking consistency,
missing data, skipping pattern, data entry,
maintaining confidentiality, storage, back-up, etc..
– Prior data analysis plans help the investigator how
study objectives will be met.

• Piloting study procedures and conducting

practice interviews
 Current Issues and Future Directions
• Resource Conservation
• Data Quality
• Scope
• Periodicity
• Technology- Biology Connection
• Confidentiality
 Resource Conservation
• Public health practice including surveys are often
limited by resource.
• Resource conservation can be achieved in many ways:
- Example of resource conservation act: US government
enacted Paperwork Reduction Action in 1995 to reduce
paperwork burden imposed on private business and
citizens
• Technology plays a great role in data collection. But it
also needs initial investment to, purchase device,
programs, software etc..
 Data Quality
• Data quality in international research may be influenced
by: limited skills and and training for data collector, data
management, practice of integrity
• Technology based surveys introduce better validity and
reduce bias. For example: People who take surveys on
computers may be willing to respond more accurately
than face to face conversation specially for sensitive
questions
• Then framing questions need to clearer because there is
no one to explain
• Extra care is also needed for impaired, illiterate people
 Scope
• Scope of public health survey is limited with
resources
• Increased geographic coverage could be leveraged
using technology, which is otherwise difficult
• For example, conducting multi-site survey could
be done through internet, text messaging etc..
• Now issue becomes relevance, standardization
and quality of information
 Periodicity
• Survey could be conducted one time or
multiple times, technology made it easy
• Population could be exposed to multiple
surveys in same period of time
• This could result in survey fatigues such as
reduced response rates, selection bias and
response bias.
 Technology-Biology Connection
• The application of these newest techniques
might cause the respondents and survey team
to lose interest in person to person
interaction.
• Survey may be done remotely using
technology, however, having biological
specimen to link the survey may be an issue
 Confidentiality
• Technology based survey has introduced to
unauthorized access to data reports
• Lack of information security
Survey data: Examples
 Online survey providers:
SurveyMonkey: https://www.surveymonkey.com/
 Google forms:
https://docs.google.com/forms/u/0/?tgif=d
 KOBOtoolbox:
https://www.kobotoolbox.org/