SHORT ANSWER HIGHLIGHT IN BLUE
MULTIPLE CHOICE HIGHLIGHT IN GREEN
What are the general guidelines for constructing survey items we discussed in lecture?
- Make items clear: avoid ambiguous questions; do not ask “double-barreled” questions
- “The Department of Corrections should stop releasing inmates for weekend
release and concentrate on rehabilitating criminals”
- Short items are best - respondents like to read and answer a question quickly
- Avoid negative items: leads to misinterpretation
- Avoid biased items and terms: phrasing can encourage a certain answer
Be familiar with the relationship between sampling strategies and generalizability.
- How it is we collected our participants that relates to how confident we can be to those
results. We can’t generalize those that weren’t included in the study. If we do probability
sampling it gives us a better confidence to generalize the people that weren’t included in
sample because it was random selection.
Be able to name and describe the key features of the probability and non-probability
sampling strategies we discussed in lecture.
Probability Sampling:
Random Sampling
- Probability sampling uses statistical probability to generate a sample that is
representative of the population
- What is the importance of random selection?
- A sampling frame is necessary in probability sampling
- This is an exact (or quasi-) list of all the elements in a population
- How to calculate the ideal size of a sample
Simple Random Sampling
- Process
- Each element in a sampling frame is assigned a number
- A necessary sample size is determined
- A random number generation is then used to select which elements will be
included in your sample
Systematic Sampling
- Process is the same as simple random sampling with a slight variation hb
- There is a pattern to the overall selection of elements
- Every “n”th element is selected
- The first element is selected randomly, with every “n” element selected in the list from
that first element
- If your list categories elements in an organized way you should not use systematic
sampling
- Your sample will be biased
Stratified Sampling
- Is not an alternative to random or systematic sampling
- It’s a modification to their use
- Stratified sampling is a method for obtaining a greater degree of representatives in your
sample
- The variables you choose to depend on
- What variables you have information on
- Which are particularly important for the research questions
Nonprobability Sampling:
Purposive Sampling
- Selecting a sample on the basis of your judgement and the purpose of the study
- Sometimes selecting a sample requires our own knowledge of the population, its
elements, and the nature of our research aims
- Purposive sampling is ideal if members of a population are easily identified, but
compiling a list of all of them would be nearly impossible
Convenience Sampling
- Relying on persons available to complete your study with no or minimal limitations on the
characteristics of the types of persons participating
- A convenience sample is made up of people who are easy to reach
- Online participants
- Student participants
- Convenience sampling is a matter of taking what you can get and is the least desirable
type of sample
Snowball Sampling
- Involves…
- Identifying a single subject or small number of subjects
- Asking the subjects to identify others like them who might be willing to participate
in the study
- Snowball samples are essentially variations on purposive samples and samples of
available subjects
Be familiar with how the examples of Rosenbaum’s and Ferrell’s research differ in these
regards.
- Ferrell’s was inductive, had no idea what he was looking for before he composed a
theory. Pulling pictures out of dumpster.
- Rosenbaum’s study was deductive because he had a theory he was looking for in his
research. Shoplifters.
What are the four field researcher roles we discussed in lecture?
- Complete Participant- Participate fully; true identity and purpose are not known to
subjects
- Complete Observer- Observe without engaging with that being observed or becoming a
participant
- Participant as Observer- Make known your position as researcher and participate in the
observed activity
- Observer as Participant- Make known your position as a researcher; do not actually
participate in the activity
Be able to name and describe the three general types of interview schedules.
- There are 3 general types of interview schedules…
- Structured
- Semi-structured
- Unstructured
- Structured interviews:
- Completely standardize the questions being asked across all participants
- Semi structured interviews:
- Also use standardized questions but include unscheduled probes to enhance the
depth of the response
- Unstructured interviews
- Make more flexible, no specific questions but general guidelines wanted to focus
on
Be familiar with the method details of how Douglas and Cuskelly (2012) employed the
focus group method.
- Aim was to investigate how police in Queensland, Australia, determined that an
individual has an intellectual disability
- In Australia, if “it becomes apparent” that a person is intellectually disabled,
police must suspend questioning until support is available
- Interested in the capacity of police to recognize than an individual requires additional
supports:
- Correct identification ensures that the appropriate protections are immediately
put in place
- Participants were seated in a circle to:
- Maximize face-to-face contact
- Emphasize equality
- Allow interaction to occur in a conversational manner
- Two main questions were used to frame the discussion:
- “When you had an occasion to deal with a person with an intellectual disability
during your work, how did you recognize that the person had an intellectual
disability?”
- “If you came across a person with an intellectual disability how would you
recognize, if no one told you, that this was a person with a disability?”
- The process of writing our interviews verbatim is called transcription
- But you want more than just a written record of what your participants said
- Memoing is a technique that involves writing about your research process from
beginning to end
What are the four steps in the policy creation process? Be able to describe each step.
- Policy demands
- Support for some new course of action of opposition to some existing policy
- Policy agenda
- Consideration of ultimate goals and strategies for achievement
- Resource allocation
- Draft standard course of action
- Policy outputs
- Are the means to achieve desired policy goals
- Policy impacts
- Assess if the policy action is achieving the goal it was intended to achieve
Be familiar with the core aspects of Becker’s (1966) article (e.g., argument and
conclusion).
- A recurrent theme in this course has been the importance of objective research
- Sampling techniques
- Material (survey items and interview questions) construction
- Reflexivity
- Social scientists confront a false dilemma
- Do we strive to do neutral research that is value free?
- Do we express our values and adopt a commitment to a position
- Why is this dilemma false?
- The question is not whether we should take sides, because we inevitably will!
- The question is: whose side are we on?
- Becker considers two types of research situations where researchers will inevitably take
a side
- Political
- There is acknowledged tension between subordinates and authorities
- Subordinates have some degree of organization
- Apolitical
- There is not a defined power struggle between subordinates and
authorities
- Whatever point of view we take, our task is to ensure that our research meets the
standards of good scientific work
- We do this by using sound methods
- Impartial techniques that improve the reliability of and validity of our research
Technical term for questions and statements used in a survey?
- Items is the Technical Term
What are the two basic ways of presenting items in surveys?
- Close-ended
- Respondent selects an answer from a list of choices
- (ex; do you think crime is a problem in your community? YES NO)
- Open-ended
- Respondent is asked to provide their own answer
- (Ex; what sort of things would make you think that crime is a problem in a
community? __________ )
Be familiar with contingency and matrix formatting options.
Contingency Formatting
- Contingency items:
- Relevant only to some respondents
- Are answered only if necessary based on the previous response
- Ex; “have you ever used illegal narcotics?”
- Yes
- No
- If yes, proceed to next item
- If no, skip next item
- “Please list the illegal narcotics you have used.”
Matrix Formatting
- Same set of attributes used for multiple items (eg. Well, Pretty Well, Extremely Well)
- Key advantages
- Uses space efficiently
- Easier for respondents
- Increasing response compatibility across options
- Response rate refers to the number of people participating in a survey divided by the
number selected for the sample
What does a response rate refer to?
- The number of people who answered the survey/The number of people asked to
complete survey
How does the survey method fair in relation to validity and reliability concerns?
- Validity- Low due to the survey being a superficial overview.
- Reliability- High since it can be given to a large population.
- Surveys tend to be high on reliability
- Standardization is respondent responses;
- Data collection benefit
- Data analysis benefit
- One of the biggest strengths of surveys is the ability to tabulate experiences
- Artificial understanding of social understanding (Not Sure What I Meant Here)
Sampling
What is the difference between a sample and a population?
- Sample element:
- Who or what we are studying (Ex student)
- Population
- Whole group we seek to generalize tob (ex criminology program students)
Be able to name and define the two fundamental approaches to sampling.
- 2 fundamental approaches to sampling
- Probability sampling
- Non-probability sampling
The Logic of Probability Sampling
- Probability sampling allows researchers to generalize from observed to unobserved
cases
- Gives all the cases in the population an equal chance of being selected
What is the logic behind selecting a sample?
- To represent a larger population of cases
- To generalize from observed sample cases to unobserved cases from the
population
Nonprobability Sampling
- There are situations when it is impossible to select a probability sample
- Non Probability sampling is that in which the probability that an element will be included
in the sample is not known
- Findings cannot be reliably generalized to larger population
What is the basic principle of probability sampling?
- Gives all the cases in the population an equal chance of being selected
Be familiar with the four key concepts associated with probability theory: sample
element, population, population parameter and sample statistic.
- Sample element:
- Who or what we are studying (Ex student)
- Population
- Whole group we seek to generalize to (ex criminology program students)
- Sample statistic:
- The summary description of a given variable in the sample (ex average of
program satisfaction for sampled criminology students)
- Population Parameter:
- The actual or assumed value for a given variable in the whole population (Ex
program satisfaction for all criminology students)
What is a sampling frame?
- A sampling frame is necessary in probability sampling
- This is an exact (or quasi-) list of all the elements in a population
-
-
Field Research
Be familiar with the use of field observations for both quantitative and qualitative
research.
- Quantitative approach- Structured idea of what you are looking for.
- Qualitative approach- It is used to gain an understanding of underlying reasons,
opinions, and motivations. It provides insights into the problem or helps to develop ideas
or hypotheses for potential quantitative research.
- Field research encompasses two different methods of obtaining data…
- Direct observation
- Asking questions
- Most commonly associated with qualitative research but can yield quantitative data
- Provides the opportunity to produce highly detailed, “rich” descriptions
- Qualitative field research is often a theory or hypothesis generating activity
Be able to identify when an inductive or deductive approach is being adopted in field
research.
- Deductive- taking hypothesis and finding data that supports the idea.
- Inductive- Collecting data then determining a theory.
What does reaching saturation mean in the context of qualitative research?
- Saturation- When sampling more data does not lead to more information or results.
What is the name for exceptionally in-depth field observation studies?
- Ethnography- Focused on detailed and accurate description rather than explanation
What does it mean to “go native” in the context of field research?
- Go Native- Get too attached and involved, which blurs the lines of the research.
What is reactivity in the context of field research?
- Reactivity - Research subjects change their behaviour in reaction to being studied.
How does field research tend to fair with regards to validity, reliability and
generalizability?
- Validity
- Whether recorded observations are representative of the true phenomenon under
study
- Reliability
- Whether observations are dependable and consistent
- Generalizability
- Whether specific research findings from the sample apply to a broader population
from which the sample was drawn
Interviews
What is the difference between thin and thick description?
- Thin- Quantifiable facts (Measurable data that is able to be expressed)
- Thick- Add some verbal context to the elements (winking example- someone could be
winking to be flirting, or can be a joke or can be a eye twitch)
Be familiar with the relationship between these different types of interview schedules and
adopting an inductive or deductive approach to research.
- Most research is deductive, but can be both.
What is the critical realist perspective and why is it relevant to qualitative interviewing?
- Critical realist perspective
- A philosophical view that reality exists, but knowledge is constructed through
multiple meanings
Be familiar with the details of the method used by Burgess, Hartman, Ressler and
Douglas (1986).
- Method
- Study focused on analyzing crime scene evidence in conjunction with the details
gleaned about these murderers
- Semi-structured interviews with 36 sexual killers
- Was not designed to examine motivation, but yielded rich descriptive data
about what moved these men to kill
- Interviews included questions about the offender, victim, offense, and
crime scene
- Deductive Approach- Designed to examine data about what moved men to kill
What is an interview guide?
- Interview guide- Includes a list of topical areas that you want to cover in the conversation
What is the role of the sponsor in research?
- The basic principle of using a sponsor to gain initial access operates in much the same
way
- People whose job involves working with offenders
- Offenders who are connected with other criminals
What is reflexivity in the context of qualitative research?
- Reflexivity
- Refers generally to your subjectivity in the research process
- Related to the critical realist perspective and the idea that reality is composed of
multiple perspectives
- Interviewers should engage in self-reflexivity to encourage awareness of the
influence we have on the research process
- How do your feelings influence your relationships with participants?
- How do your feelings influence the research process more generally?
Focus Groups
What are the types of probes we discussed in lecture as commonly used in qualitative
interviewing?
- Probe types include:
- Attention probe (ex lean in)
- Continuation probe (ex nod)
- Clarification probe (ex ask the respondent to clarify or ask a follow-up question)
What is the definition of a focus group?
- Involves a directed discussion with a small group of people
- A defining characteristic of the focus group method is:
- Attention to group dynamics to explore how opinions are produced, expressed,
and changed
Be able to name and describe the two basic types of focus groups.
- Natural- When people are comfortable which give more real answers.
- Artificial- Have been selected based on criteria and don’t know except what they are
talking about.
What are the two factors we discussed in lecture as important to keep track of in focus
groups regarding group dynamics?
- Groupthink:
- Occurs when participants minimize conflict by reaching consensus
- Diminishes access to alternative viewpoints
- Dominant group members:
- Persons who take over the group
- Silences participants who are more naturally quiet
What is transcription in the context of qualitative interviewing?
- The process of writing out interviews verbatim is called Transcription…
- But you want more than just a written record of what your participants said
What are the three types of memos used in qualitative research that we discussed in
lecture?
- Memoing is a technique that involves writing about your research process from
beginning to end
- There are three types of memos…
- Operational
- Steps you took in each stage of your research
- Coding
- Documenting the process of coding your data
- Analytic
- Record of how you explored relationships in your data
What techniques did we discuss in lecture as ways to enhance validity and reliability
when using qualitative interviewing as a method?
- Validity and reliability can be enhanced in qualitative research through…
- Member checks…
- When participants read drafts of products to verify its accuracy
- Using multiple coders…
- Allows an inter-rater reliability statistic to be generated
Content Analysis
Be able to identify the research methods we discussed that do not require interaction
with research subjects.
- The methods of focus this week do not require direct interaction with research subjects
- Data from agency records
- Agencies collect a vast amount of crime and criminal justice data
- Secondary analysis
- Analyzing data previously collected
- Content analysis
- Researchers examine a class of social artifacts (Ex written, visual, verbal
materials)
What unit of analysis does content analysis involve working with?
- Content analysis involves the systematic study of messages and the meaning those
messages convey
- Who says what, to whom, why, how, and with what effect?
What is the difference between obtrusive and unobtrusive research?
- Obtrusive- Methods are aware and intruding into life
- Unobtrusive- Unaware we are plucking data from them
Is content analysis a qualitative or quantitative research method?
- Can be both.
What two main stages does coding involve?
- Coding is conducted in multiple stages including…
- Open coding
- Going over content in its totality and looking for overarching themes
- Selective coding
- Involves looking for themes in the reduced versions of your data
Be able to name and define the two forms of content researchers look for when they
perform a content analysis.
- Choices of content to analyze…
- Manifest content
- Visible, surface content that is directly observable
- Latent content
- Underlying content meaning that is not directly observable
What are the two techniques we discussed in lecture as ways to enhance the reliability of
your content analysis?
- Reliability enhanced- test and retest method and inter coder reliability method
Be familiar with the method details of how Cowan and O’Brien (1990) employed their
content analysis method.
- Catalogs from from live local video outlets were examined
- Movies that appeared in 4-5 catalogs in the “horror” category (excluding films
depicting violence by nonhuman forces) formed the population
- Of these, 56 films were selected and assigned to 5 coders
- Material coded included…
- Provocative clothing
- Nudity
- Use of sexual and/or obscene language
- Promiscuity
- Demographics
- All codes were observed on a present/absent basis
- Overall inter-rater reliability, was 87% agreement, with percent agreements
ranging from 75% to 100%
Agency Records and Secondary Analysis
What is a meta-analysis?
- A meta-analysis is a statistical procedure that combines the results of multiple studies to
assess the findings collectively
Be familiar with the advantages of secondary data analysis.
- Advantages
- Cheaper and faster
- Benefit from quality researchers’ skills
- Access international data easily
- Disadvantages
- Operational definitions may vary in ways that affect validity
Be familiar with the style of research method Cao (2014) used as well as the key finding
from his research.
- Data for the research are from the General Social Survey (Secondary Research)
- The results reveal that aboriginal people have a significantly lower level of confidence in
police than other Canadians
- The findings are not surprising given the long history of racism and the strained
relationship between the police and aboriginal persons in Canada
- The findings help identify
- The details of why aboriginal persons are unhappy with the police services they
receive
- Tangible steps that can be taken to address these concerns
Secondary datasets often come with a codebook. What is a codebook?
- Codebook- tells you exactly what they asked and what their answer was and chose from.
Reference to lookback to.
What two categories of agency records did we discuss in lecture?
- Published Statistics
- Most government agencies routinely collect and publish aggregate data to
provide a general overview of the agency and its activities
- A challenge or working with published statistics it that they are typically only
available in summary format
- Data cannot be used to analyze the individuals from which or about whom
information was originally collected
- Summary data is useful in addressing questions about aggregated patterns or
trends
- But, not useful for providing nuanced details researchers might be
interested in
- Nonpublic Agency Records
- Agencies produce a wealth of data which is not routinely released to the public
- Many criminal justice agencies will make non-public data available to researchers
upon an appropriately channeled request
- While potentially valuable, researchers want to have a clear understanding of
how agency records were produced
- What operational definitions were used?
- How was the data collected?
- Who collected the data?
How can researchers use Freedom of Information Act requests in data collection?
- They get access due to freedom of information act. They have to go to research centers
and request the material, which can take a while.
What is the best guard against reliability and validity concerns in research using agency
records?
- Understanding the details of how agency records are produced is the best guard against
reliability and validity problems
- Try to get access to a detailed written description of the method outlining
procedures and instruments used
What are key factors to keep in mind when assessing the reliability and validity of data
from agency records?
- Other key factors to keep in mind
- The extent to which agency records are a social construction
- Agency records are typically not designed for research and are meant to track
people rather than patterns
- Clerical errors increase with volume
Applied Research: Problem Analysis and Evaluation
Be able to name and define the two types of applied studies.
- Two key types of applied studies
- Problem analysis
- Helps policy makers analyze alternative actions, choose among them and
formulated routine practices for implementing policy to achieve desired
goals
- Program evaluation
- Evaluate the actual effects of policies that have been put in place
- Are policies being implemented as planned?
- Are policies achieving their intended goals?
What is the definition of a policy intervention?
- Policy intervention- Action undertaken from the specific results (Eg. Calories on menus
due to too many overweight people in society)
Be able to define the concept of evidence-based policy and explain its relevance to the
criminal justice system.
- Social policies should be founded on evidence
- Further, policies adopted should be those with the most supportive evidence
- When criminal justice agencies do not follow this principle, they are potentially
- Not as effective as they could be
- Wasteful of resources
- Developing evidence based policy requires two basic steps
- An awareness and comprehension of existing research evidence
- An understanding of how to judge the quality and scientific believability of that
evidence
What are the four key steps involved in problem analysis?
- Problem analysis involves 4 key steps
- Define specific problems
- Conduct analysis to understand causes
- Search for multiple solutions to bring about lasting reductions in problems
- Evaluated the success of these activities
Be familiar with how Plouffe and Sampson (2004) used problem analysis to construct a
policy intervention that responded to auto thefts in Chula Vista.
-
What does adopting the scientific realist approach mean in the context of problem
analysis?
- In relation to problem analysis, adopting a scientific realist approach would mean
expecting that
- Similar interventions can naturally be expected to have different outcomes in
different contexts
What are the two key types of evaluations we discussed in lecture? Be able to define
each.
- Process based evaluation
- Geared toward understanding how a program or policy works
- How does it produce the results that it gets?
- Employee training, customer/client interaction, complaints
- Outcome based evaluation
- Geared toward understanding if a program or policy meets the goals it was
intended to meet
- Does it produce the results that it was expected to?
- Outcomes, operationalization
What is a stakeholder in the context of applied research?
- Conducting evaluation research requires operationalizing, observing, and recognizing
what is under study
- General program objectives need to be clarified into empirically testable
statements
- Different stakeholders often have different goals for a policy
Be familiar with how Welsh, Farrington and Taheri’s (2015) meta-analysis of the
effectiveness of CCTVs illustrates the importance of operationalization for the creation of
sound evidence-based policy.
- CCTV has become a highly popular method to prevent crime in public space worldwide
- Despite the dominance of CCTV in policy, questions have been raised about its
effectiveness
- The authors conducted a meta-analysis of 41 studies that assessed the effectiveness of
CCTVs at reducing crime
- Results that averaged the findings from all the studies revealed
- An overall 16% reduction in crime
- But, where the CCTV system was located affected the results
- Reduced crime by 51% in parking lots
- Did not significantly reduce crime in city centers, public housing or public transit
- But the type of crime affected the results
- Did not significantly reduce violent crime (3% overall reduction)
- Significantly reduced vehicle crime by 26%
- Operational definitions matter
What is the definition of treatment fidelity?
- the strategies that monitor and enhance the accuracy and consistency of an
intervention to ensure it is implemented as planned and that each component is
delivered in a comparable manner to all study participants over time
Interpreting Basic Social Science Statistics
Be able to identify key characteristics of quantitative research.
- Key characteristics of quantitative research include
- Clearly defined research questions to which objective answers are sought
- Study is carefully designed before data is collected
- Data is usually gathered using structured research instruments
- Results are based on larger sample sizes that are representative of the
population
- Study can usually be replicated or repeated, given its high reliability
Be able to define descriptive and inferential statistics.
- summarize data - data reduction technique that involves using a single statistic to
represent a distribution (group of scores)
What do the mean, standard deviation, and p value statistics tell us in statistical
analyses?
- P Values- Inferential statistics use a random sample of data taken from a population to
describe and make inferences about the population
- A key objective of inferential statistics is to describe relationships between
variables
- P value less than 0.5 is significant
- between variables is statistically significant
- Mean and SD- Are both specific types of descriptive statistics
- Descriptive statistics summarize data
- They are a data reduction technique that involves using a single statistic
to represent a distribution (group of scores)
- The mean is the arithmetic average of the scores
- Calculated by adding all the scores and dividing by the total number of scores
- All means are accompanied with a standard deviation
- The standard deviation is a single number that tells us the variability, or spread, of a
distribution
What is the purpose of a t-test? Be able to identify research questions that would be best
suited for a t-test.
- Test purpose
- Used when you want to compare the means of two groups for a dependent
variable
- What levels of measurement must variables included in the analysis have?
- IV = dichotomous
- DV = interval/ratio
- What is an example of a question we might analyze using a t-test?
- Do students’ test scores vary by sex (male vs female)?
- Does police foot patrol (practices vc non-practised) affect rates of violent crime?
What is the purpose of an ANOVA? Be able to identify research questions that would be
best suited for an ANOVA.
- Test purpose
- Used when you want to compare the means of three or more groups for a
dependent variable
- What levels of measurement must variables included in the analysis have?
- IV = categorical (3 or more groups, nominal or ordinal)
- DV = interval/ratio
- What is an example of a question we might analyze using an ANOVA?
- Does income vary by level or education completed (less than secondary school;
secondary school; post-secondary school)
- Does victim presence for an apology (ambiguous; surrogate; direct) affect
offender perceptions of accountability for harm caused
What is the purpose of a correlational analysis? Be able to identify research questions
that would be best suited for a correlational analysis.
- Test purpose
- Is a measure of the strength and direction of an association that exists between
two variables
- What is an example of a question we might analyze using correlation
- Is offence severity associated with length of prison sentence?
- Is length of prison sentence associated with offering a guilty pleas?
Be able to identify the distinction between a positive and negative association between
variables.
- Two types of regression
- Linear regression
- The dependent variable is truly measured at the interval/ratio level
- Logistic regression
- The dependent variable is dichotomously coded
Methodology, Methods, and Values
Be able to define the key distinction between qualitative and quantitative approaches to
research.
- The key distinguishing feature is the focus on numerical data
- The numerical descriptions of things and their relationships is the focus of
quantitative research methods
- Qualitative research methods emphasize subjective interpretation, holistic
perspectives, looking at local contexts, and producing a greater depth of
understanding
What are the criteria that should define whether you adopt a quantitative or qualitative
approach to research?
- Selecting a quantitative or qualitative approach should be based on
- Your research question
- Your research purpose
- The data or data sources you have access to
Which approach, quantitative or qualitative, is more common in published criminology
research?
- Quantitative research is typically considered to be the more “scientific” approach to doing
social science while qualitative research has been relegated to the realm of
pseudo-science
- Qualitative criminological research is less commonly published
- Less than 11% of articles in top tier journals in the discipline employ qualitative
methods
Be familiar with Tewksbury’s (2013) assessment of the value of qualitative research to
criminological research.
- Tewksbury (2013) argues that qualitative methods are about gaining more valid
understanding of
- The social aspects of how crime occurs and how the agents, structures and
processes of responding to crime operate in culturally-grounded contexts
- Qualitative methods provide a depth of understanding that is not possible through the
use of quantitative, statistically-based investigations
- While less generalizable, qualitative research offers a more in depth understanding of
social issues and the contexts in which they occur
What is the difference between being critical and engaging in critical thinking in
research?
- Critically thinking- is using information and conceptualize and how to solve it
- Being critical- when someone is just disagreeing with another opinion for the sake of
being wrong (identify problems)
