ACCELERATED MASTERS & DOCTORAL ONLINE SUPPORT PROGRAMME –

SELECTING A METHODOLOGY AND RESEARCH DESIGN :

Mixed Methods Research

Facilitator – Professor
Tennyson Mgutshini
Learning Outcomes
At the end of this session, you should

• Appreciate the strengths and weaknesses of Quantitative and Qualitative research designs.

• Recognise the various cases when Qualitative and Quantitative research can be ‘integrated’ or
‘combined’ to better answer research question(s) within a Mixed Methods design approach.

• Have the capacity to select from a range of Mixed Methods designs based on prioritisation, timing
and mixing procedures.

• Appreciate the strengths and weaknesses of Mixed Methods Research.

• Understand ethical and epistemological issues that can arise in the presentation and
dissemination of Mixed Methods research.
Qualitative vs Quantitative Methods
1. Fundamentally different, have strengths and weaknesses

QUANTITATIVE APPROACH QUALITATIVE APPROACH

• Objective conclusions (objectivist) • Subjective conclusions (constructivist)

• Structured data • Unstructured data
• Surveys, experiments • Summaries, themes, coding
• Reliable • Interviews, focus groups, observations
• Generalisable • Approaches –narrative,
• Statistical analysis phenomenology, grounded theory,
• Approaches -Descriptive, experimental, ethnography, case studies
correlational, quasi-experimental

• No specifics, reasons, examples • Not generalisable, typically narrow

• No explanations, just ranged, small samples
generalisations for large samples – • Focused on meaning and context
nomothetic –lawlike findings that • Biased
hold irrespective of time • Ideographic- depth and intensity focus
 Mixed Methods Research: What IT IS
“as a method, mixed methods focuses on collecting, analysing, and mixing both quantitative and
qualitative data in a single study or series of studies. Its central premise is that the use of quantitative and
qualitative approaches, in combination, provides a better understanding of research problems than either
approach alone.”

Beware!!
• NOT triangulating data collection methods where all of them are either qualitative or quantitative

• NOT qualitative and quantitative data, reported separately and qualitative and quantitative data
answering separate research questions.
Mixed Methods : a research methodology
As a research methodology, Mixed Methods Research involves the following elements

• Philosophical Stance –pragmatism, multiple worldviews, changing worldview, worldview shaped by

community, worldview shaped by context

• answers Research Question(s)

• Data Collection components (both qualitative and quantitative)

• Data Analysis components (qualitative and quantitative)

• Data Interpretation components (qualitative and quantitative)

• Data Reporting components

Why Mixed Methods?
Using Mixed methods we are able to

• Generalise from a sample

quantitative research)
to a population (as in

• Gain a richer, contextual understanding of the

phenomenon being studied (as in qualitative research)
Mixing Methods is great but……

• It can be resource intensive – Costs can be high

• Quantitative questions may end up measuring different
constructs to qualitative

• Problems in managing to synthesise quantitative and

qualitative findings

• Very often lack of integration

When to use Mixed Methods Design
Some primary considerations before choosing Mixed Methods design - a checklist

1. Mixed Methods design matches the research problem

2. Research team possesses both qualitative and quantitative data collection and analysis skills

3. Resources are available to fund the component qualitative and quantitative studies

4. There is enough time to pursue design with both qualitative and quantitative strands

5. The sponsors of the study and the target audience are open to mixing of methods
 Mixed Methods Design Selection
Three additional decisions relating to procedures;
• Timing Concurrent?
Sequential ?

• Weighting Equal weight

Unequal weight

Merge results during interpretation

Merge the data Merge data during analysis
Embed qualitative data in quantitative design
Embed the data
• Mixing Embed quantitative data in qualitative design
Embed qualitative data in quantitative design
Connect the data
Embed quantitative data in qualitative design
TRIANGULATION DESIGN(Mixed Methods Type 1)
• also called the ‘Concurrent’ Design

• Use :obtaining complementary data on same topic

• Procedures : Single phase, separate Qualitative and Quantitative data collection and analysis within
the same timeframe (timing) , equal priority , data sets merged at analysis or at interpretation

• Strengths : intuitive, common, easy to comprehend, efficient data collection, team oriented

• Weaknesses: effort and expertise heavy, dilemma when qual and quan do not converge

• Variants: convergence, data transformation, validating quantitative and multilevel models depending
on specific intent

• Schematically,

QUAL QUAN
Equal weight as quan RESULTS Equal weight as qual
 Convergence Model (variant of Triangulation Design)
• Use: to substantiate conclusions by comparing, validating , confirming or corroborating quantitative
results with qualitative findings

• Procedures : Single phase, separate Qualitative and Quantitative data collection and analysis within
the same timeframe (timing) , equal priority , data sets merged during interpretation

• Weaknesses :Qual and Quan mismatched sample sizes, converging different data types not easy

• (Examples : Andersen, Newell and Kilcoyne(1999))

• Schematically,

QUAN QUAN QUAN

Data collection Data analysis results

Compare QUAN +QUAL

Contrast Interpret results

QUAL QUAL QUAL

Data collection Data analysis results
Data Transformation Model (variant of Triangulation Design)
• Use: interrelating different data types about a research problem

• Procedures: Single phase, separate Qualitative and Quantitative data collection and
analysis within the same timeframe (timing) , equal priority , data sets merged by
transforming one data type to the other during analysis, comparing, interrelating

• Weaknesses: transforming one data type to another not always easy

• Example: Pagano et.al (2002)

• Schematically:

QUAN QUAN Data analysis

Data collection

Compare and QUAN +QUAL

Interrelate two Interpret results
QUAN data sets
QUAL data
QUAL QUAL transformed to
Data collection Data analysis quan
Validating Quantitative Data Model (variant Triangulation Design)

• Use: for validating and expanding on quant findings from survey by enriching with qual input

• Procedures : qual and quant data collected within one survey instrument, qual open-ended
questions whose responses are used to embellish quant findings

• Examples: Webb et.al (2002)

• Schematically,

QUAN QUAN Data QUAN results

Data collection: analysis
Survey
Validate QUAN QUAN +qual
results with qual Interpret results
results
qual
Data collection qual
qual results
open ended Data analysis
surveys
Multilevel Model (variant of Triangulation Design)
• Use: overall interpretation of multilevel system e.g.organisation - different data types
deemed appropriate to address different levels

• Procedures : Separate data collection and analysis for each level, merged at interpretation

• Example: Elliot and Williams (2002)

• Schematically,

Level 1: QUAN
Data collection and analysis, results

Level 2: QUAL
Data collection and analysis, results Overall interpretation

Level 3: QUAN
Data collection and analysis , results
EMBEDDED DESIGN(Mixed Methods Type 2)
• Use: single data set deemed insufficient, different questions requiring different types of data required
to answer research question fully.

• Procedures: Second. data type embedded @design level & less priority, Case 1 : quant data
embedded in qual methodology (phenomenology design) , Case 2-qual data in quan
dataset(experiment or quasi-experiment)

• Strengths : secondary data requires less time, logistics less cumbersome, attractive to funders
because of traditional quant focus

• Weaknesses: clarity on purpose of embedding, integration difficulties, not so widely researched

• Variants: several, depend on setting, experimental or correlational setting discussed here

•
• RESULTS OR RESULTS
QUAN QUAL
embedded qual embedded quan
Embedded Experimental Model (variant of Embedded Design)
• Use: shaping interventions, developing instruments, participant selection and follow-up

• Procedures: one or two-phase, quant methodology priority, qual subservient, single phase - qual
data embedded during intervention (concurrent nested mixed method design), two-phase-qual
comes before (shape intervention, develop instrument, choose participants) or after (follow-up)

• Weaknesses :timing of embedment, pre-intervention planning, bias for during intervention, post
intervention decisions on further exploration

• Example: Messer et.al (1999)

EXPERIMENTAL METHODOLOGY
qual
QUAN QUAN qual after
pre intervention intervention
premeasure postmeasure intervention
Interpretation
Based on QUAN (qual)
results

qual
during intervention
Correlational Model (variant of Embedded Design)
• Use: to explain how mechanisms work in correlational model

• Procedures: Qual data is collected as part of larger correlational study based on

quan methodology to overlay and explain mechanisms

• Examples: Krupa, E (2010)

QUAN
predictors

QUAN QUAN outcome Interpretation

predictors based on
QUAN(qual)
results
QUAN
predictors

qual
process
EXPLANATORY DESIGN(Mixed Methods Type 3)
• Use: to explain outliers, significant & non-significant results, ‘unexpected’ results, identify groups
needing further understanding through qual. used before qual to direct sample based on quan
participant characteristics

• Procedures: two phases ; an initial quant analysis then qual to build up on initial quant results,
greater weight placed on initial quant

• Strengths: data collection and analysis -one type data per phase, 2 phase report easy, flexibility in
phases, user-friendly for quant researchers

• Weaknesses: time, participant selection for phases, difficult to plan phase 2 which depends on
phase 1 outcomes

• Variants: follow-up explanations design and participant selection design

• Schematically ,:

Interpretation based on
QUAN
qual QUAN qual results
Follow-up Explanations Model (variant of Explanatory Design)
• Use: use qual data to explain anomalies, outliers, significant or insignificant results from quan
results.

• Procedures: quan data has priority, collect and analyse quan data, identify anomalies &
unexpected results, seek explanations from target group via qual.

• Challenges : second phase contingent on results of quan means difficult to plan.

• Examples : Ivankova (2004)

• Schematically

QUAN QUAN QUAN Identify qual qual qual Interpretation

Data Data results results for Data Data results QUAN qual
collection analysis Follow-up collection analysis
 Participant Selection Model(variant of Explanatory Design)

• Use: Ascertain participant selection for in-depth qual study

• Procedures : initial quan inquiry to finesse the participant selection for a major qual, quan
subservient to qual

• Challenges : clear specification of criteria used for participant selection

• Example: May and Etkina (2002)

• Schematically

quan quan quan QUAL to QUAL QUAL QUAL Interpretation

Data Data results Identify Data Data results quan QUAL
collection analysis participants collection analysis
EXPLORATORY DESIGN(Mixed Methods Type 4)
• Also called Exploratory Sequential Design

• Use: develop, test new instrument, develop new framework, to establish variables governing
some behaviour or phenomenon, measure their prevalence, interrogate aspects of new theory or
classification

• Procedures: two sequential phases, initial dominant qual strand followed by quan

• Strengths : 2 phase implementation easy to describe, implement and report, quant-based

audiences welcome inclusion of quant to dominant qual, amenable to single &multiphase research

• Challenges: time intense 2 phase, difficult to plan i.e. qual outcomes unknown, participant choices

• Variants: several, include ‘instrument development model’ &‘taxonomy development model’

• Schematically:

Interpretation based on
QUAL
quan QUAL qual results
Instrument Development Model : variant, Exploratory Design

• Use: use findings from qual data to inform subsequent quan data to ascertain variables,
instruments, measures, or guiding framework or theory
• Procedures : 2 phase, initial qual data with few participants to explore research topic, results of
initial qual inform instrument design for subsequent quan. 2nd phase quan to implement and
validate instrument . Quan strand given greater priority

• Challenges : expertise required to choose qual data for instrument development and how to
generate quan measurements and measures reqd to ensure credibility of scores

• Example :Mak and Marshall(2004))

qual qual qual Develop QUAN QUAN QUAN Interpretation

Data Data results instrument Data Data results qual QUAN
collection analysis collection analysis
Taxonomy Development Model : variant of Exploratory Design

• Use: to use findings from qual data to identify important variables, taxonomy or classification system
or to develop emergent theory and to test them, examine emergent categories and their prevalence
• Procedures: The initial qual to produce categories/relationships used to inform research questions and
data techniques for subsequent quan phase to address questions/hypotheses from phase 1

• Challenges : Expertise is required in deciding what qual findings from phase 1 to use for the quan
phase

• Examples :Goldenberg et. Al (2005)

Develop Interpretation
QUAL QUAL QUAL quan quan quan
Taxonomy QUAL qual
Data Data results Data Data results
Or theory
collection analysis collection analysis
For testing
Ethical issues associated with mixed methods
• participants cannot be anonymous, may be required for later phases

• Generally more types of information than would ordinarily be the case with other
methods

• Generally studies more difficult to explain to and to comprehend by participants

• Consent forms can be complex - detailed

• Questions cannot be anticipated and they may become uncomfortable

• voice recognition etc concerns

• Follow-up participation– inform them at beginning.

• Well described safeguards for participant protections

• Mixed methods research can be cumbersome to participants, repeated interviews

draining. Reassure participants about benefits
 Hints on Mixed Methods Research Proposal
The following is a typical structure of a research proposal for a study based on Mixed Methods Research
Extracted from Creswell

• Title Page – foreshadow the use of mixed methods in the title

• Abstract – Identify the type of mixed method design, the quantitative and qualitative methods and
the integration intent
• Introduction - in describing the gap that the study is filling, emphasize the value of using both
qualitative and quantitative strands and the enhancement of results from the integration.
• Purpose and Questions – the purpose statement must be clear and should explicitly articulate the
purpose of the mixed methods. For each of the qualitative strand, the quantitative strand and the
mixed methods integration, a research question or aim should be articulated in the correct order as it
is implemented during the study.
• Philosophical and Theoretical Foundations – state your worldview and positionality and demonstrate
the relationship between it and your planned mixed methods. Include a diagram of the theoretical
model or conceptual framework demonstrating your understanding what you expect to learn from
your study.
• Methods –Define and provide and provide rationale for using mixed methods design
 Hints on Mixed Methods Research Proposal
• Literature Review– review quantitative, qualitative and mixed methods studies relevant to your
research. Conclude by highlighting the need for a study based on Mixed Methods to add valuable
insights to the literature and explain how the literature reviewed inform the current study.
• Methods – Define and provide a rationale for your use of mixed methods. State the mixed methods
design you plan to use and why and cite examples of its use.
- As an appendix, present and discuss a procedural diagram for your mixed methods design
-Discuss both qualitative and quantitative sampling, data collection, analysis and data transformation
and validity . Order the topics to match the design
-Explain how data will be integrated, show the integrated results in a joint display and demonstrate
the additional insights that the integration will achieve
- Mention mixed methods validity threats that match the design and discuss how they will be
anticipated and mitigated
• Potential Ethical Issues – mention any risks or benefits associated with the use of mixed methods
• Researchers resources and skills – materials, software and training required
• Timelines for study – report all phases, approvals data collection, analysis, integration
 Resources and Reading List
• Creswell, J.W (2015). A concise introduction to mixed methods

• Bergman, M.N. (2008). Advances in mixed methods research: theories and applications.

• Hesse-Biber, S.N. (2010). Mixed methods research: merging theory with practice.

• Edmonds, W.A. (2013). An applied reference guide to research designs: quantitative, qualitative and

mixed methods
Thank you