Deploying the Proper

Methodology
KARIM SAFIEDDINE

DEPARTMENT OF SOCIOLOGY | UNIVERSITY OF PITTSBURGH

SOCIAL RESEARCH METHODS

WEEK 3
Overview
Writing a Methods Section for:
◦ Quantitative Methodologies
◦ Qualitative Methodologies
◦ Mixed Methodologies

Discussion – outlining methodologies. Preview of methods section

Quantitative Methodologies
Survey Design – provides a quantitative description of trends, attitudes, and opinions about a
population. Or it tests for associations among variables of a population (Example: General Social
Survey)
◦ Surveys help answer three types of Questions
1. Descriptive questions (ex: Cross-sectional surveys)
2. Questions about relationships between variables
3. Questions about predictive relationships between variables over time (ex: Longitudinal surveys)

Experimental Design – Systematically manipulates one or more variables to evaluate how this
manipulation affects an outcome of interest (Example: schools affected by number of students
/teacher)
Survey Design: Broad Questions
Identify the purpose or rationale of the survey
Indicate why a survey is the appropriate method.
Is the survey cross-sectional (from different individuals at a single point in time) or longitudinal
(the same group of individuals over a period of time)?
How will you collect the data? Goal: randomness (each data point has an equal and independent
chance of being chosen) OR representativeness (quota sampling… maybe?)
Survey Design: Intricate Questions
Population and Sample
◦ Identify the Population (What are you studying as a whole)
◦ Construct a sampling design – single stage or multistage (clustered) sampling
◦ What type of sampling – ideally a simple random sample (SRS). But a systematic can approximate the probabilistic characteristics of
an SRS. Though these are required for statistical procedures, they are difficult to construct.
◦ Non-probabilistic: Convenience sample (Chosen based on availability)
◦ Sample Size: A larger sample is typically desired, but at the cost of time and resources
◦ Power Analysis: If you need to detect significant associations between variables, a power analysis can help determine necessary
sample size. Many journals require a power analysis for sample size determination in their methods sections (to determine
minimum sample size)
Guide: preventivemedicine.northwestern.edu/docs/applied-statistics-presentation-materials/sample-size-and-power-presentation.pdf
Survey Design: Instrumentation
Are you using your own survey instrument or someone else's?
Validity Scores
◦ Content Validity – Do the items in the survey measure what they say to measure? (ensuring that all
aspects of the measurement are incorporated)
◦ Predictive Validity – Do scores accurately predict outcomes? (can you get a strong regression out of it?)
◦ Construct Validity – Does the instrument measure hypothetical concepts? (does it measure the
theoretical “idea” across the board)

Reliability Scores – are the they consistent?

◦ Internal consistency – the degree to which items on an instrument behave in the same way.

Administering the survey – how will people actually take it? Mail, phone, in-person, online?
Survey Design: Instrumentation
With these ideas in mind: we need to word our questions and responses carefully. Avoid:
◦ Leading Questions
◦ Does the phrasing of the question lead people to a certain response?
◦ Loaded Questions
◦ Does the question make assumptions?
◦ Double Barreled Questions
◦ Are you asking two questions at once?
◦ Jargon
◦ Double negatives

Be sure your responses are clear and exhaustive – if not they can produce bias in the results
Qualitative Methodologies
Characteristics of Qualitative Research:
◦“Natural Setting” (vs. experimental design) – ‘in the field’
◦Researcher as the instrument
◦Multiple sources of data
◦Inductive and deductive data analysis
◦Participants’ meanings
◦Emergent design
◦Reflexivity
◦Holistic account
Qualitative Research: Researcher Role
and Reflexivity
Because the researcher is involved in sustained interaction with the
participants, the researchers needs to consider how their subjective
experience influences what is observed.
◦ Social Statues: Race, class, gender, personal history shape how you interpret
experiences
Qualitative researchers need to consider Reflexivity, or the
awareness of how past experience shape interpretation, when
writing their methods sections
Qualitative Research: Role of the
Researcher and Reflexivity
How do establish reflexivity? – write personal notes in Memos
◦ Drafting Memos is an important process in qualitative work. It allows you to reflect on the
process and that shape the development of the project and identify emergent themes.

Ethical issues and considerations must be taken to account when considering

your role in the field. Consider gatekeepers and access. Consider the following
questions:
◦ Why did you select the site?
◦ What activities will occur there?
◦ Will the study be disruptive?
◦ How will you report results?
◦ What will gatekeepers and participants gain?
Qualitative Research: Data collection
procedures
Identify and purposefully select participants/research sites
◦ You don’t need a “large n” but you should identify:
◦ The setting (where will the research take place?)
◦ The actors (who will be observed or interviewed?)
◦ The events (what will the actors be doing?)
◦ The process (How do the events change over time?)

Discuss strategies for recruitment – how do you inform participants?

Comment on the number of participants and sites that are involved
Indicate types of data used (typically multiple)
Qualitative Research: Data Recording
Procedures
Observational protocol:
◦Jottings and Fieldnotes
◦How will you systematize your observations?
Interview Protocol:
◦Script for interviews – how rigid will you be?
◦Most interviews are “semi-structured” but they can be informal or
structured as well
Qualitative Research: Data Analysis
Procedures
Simultaneous analysis: insights written in the field as side notes.
Winnowing the data: Because you cannot capture everything – researchers must think about what to
focus on.
Using Qualitative Analysis software (MaxQDA; Nvivo)
◦ “coding” – reading line-by-line and assessing how the data fit into certain ideas. These are aggregated into
themes over time.
◦ Qualitative researchers typically do multiple rounds of coding. First line-by-line, then more broadly.

Goal of Qualitative analysis is to identify themes and interpret meaning from those themes.
Interpretation
◦ Interpretation is facilitated through writing memos: What were the lessons learned?
◦ You can compare what you found to relevant literature
Qualitative Research: Validity
Validity:
◦ One of the strengths of qualitative research. Determine if the findings are accurate from the
researcher’s standpoint. Are the findings (and how they are reported) trustworthy, authentic, and
credible?
◦ Validity Strategies:
◦ Thick description
◦ Reflexivity
◦ Look for negative information (that run counter to the theme)
◦ Prolonged time in the field
◦ Peer debriefing
◦ External auditor
Qualitative Research: Reliability
How can we make sure our findings and our interpretation of them
are reliable?
◦ Clearly document procedures and steps taken
◦ Check transcripts for errors
◦ Limit code drift
Goal is not generalization, but in particularity. We can get new
theoretical insights from this particularity
Mixed Methodologies
Combine insights from Qual and Quant designs
The question is:
◦ in what order do you do the methodologies and methods and why?
 Quantitative Approach Qualitative Approach Mixed Approach
Activity:
Identify the purpose or What is the purpose or What is the purpose of
Considering rationale of the survey rationale of your using mixed
observations/interviews methodologies?
methodologies Indicate why a survey is How will you establish In what order will you
For your approach, begin outlining the appropriate reflexivity? deploy these
your methods section. method. methodologies?
◦ What do you need to consider and
how do you think you will address Is the survey Identify site/actors/events Identify rationale for
it? cross-sectional or and processes quantitative approach
longitudinal?

Population and Sample What types of data will you Identify rationale for
collect qualitative approach?
Describe How will you ensure Describe
Instrumentation and Validity? population/sample and
Administration sites/actors