LOGISTIC and ORDINAL

REGRESSION
StratOne
 • Logistic regression, or logit regression, or logit model is
a regression model where the dependent variable
(DV) is categorical (nominal).
• Case of a binary dependent variable—that is, where it can
What is take only two values, "0" and "1", which represent
outcomes such as pass/fail, win/lose, alive/dead or
Logistic healthy/sick.
• The appropriate regression analysis to conduct when the
Regression dependent variable is dichotomous (binary). Like all
regression analyses, the logistic regression is a predictive
analysis.
• Logistic regression is used to describe data and to explain
the relationship between one dependent binary variable
and one or more nominal, ordinal, interval or ratio-level
independent variables.
 The dependent variable can be at least
dichotomous in nature (e.g., presence vs.
absent).

There should be no outliers in the data, which

can be assessed by converting the continuous
predictors to standardized, or z scores, and
ASSUMPTIONS remove values below -3.29 or greater than 3.29.

There needs to be no multicollinearity in the

data
 Dependent variable and error terms don't have to
be normally distributed

Dependent variable doesn't have to have equal

variance in each group

No linear relationship between the independent

variable and dependent variable has to be assumed.
ADVANTAGES
Can handle nonlinear effect, interaction effect and
power terms

Independent can be categorical variable and

bounded
 • The goal of logistic regression is to find the best
fitting (yet biologically reasonable) model to
describe the relationship between the
dichotomous characteristic of interest
(dependent variable = response or outcome
variable) and a set of independent (predictor or
explanatory) variables. Logistic regression
generates the coefficients (and its standard errors
How to use? and significance levels) of a formula to predict
a logit transformation of the probability of
presence of the characteristic of interest:
𝑒 (𝛽0 + 𝛽1 𝑥1 +𝛽2 𝑥2 +𝛽3 𝑥3 + ⋯+𝛽𝑘𝑥𝑘 )
•𝑃 𝑌 =
1−𝑒 (𝛽0 + 𝛽1 𝑥1 +𝛽2 𝑥2 +𝛽3 𝑥3 + ⋯+𝛽𝑘𝑥𝑘)
 Hosmer-Lemeshow Test - The Hosmer-Lemeshow
tests the null hypothesis that predictions made by
the model fit perfectly with observed group
memberships. A chi-square statistic is computed
comparing the observed frequencies with those
STATISTICS expected under the linear model. A nonsignificant
chisquare indicates that the data fit the model well.
AND
DIAGNOSTICS The Cox & Snell R Square and Nagelkerke R Square
(Pseudo R Square) values are both methods of
calculating the explained variation. These values are
sometimes referred to as pseudo R2 values (and will
have lower values than in multiple regression).
However, they are interpreted in the same manner,
but with more caution.
 • Beta Coefficient - the predicted change in log odd.
It could be negative or positive. Positive mean
increase in likelihood while negative means
decrease in likelihood.
• The Wald test is used to determine statistical
significance for each of the independent variables.
STATISTICS • Sensitivity – the proportion of True- Positive is
AND correctly predicted
• Specificity – the proportion of True- Negative is
DIAGNOSTICS correctly predicted
• Odds ratio (EXP(B))- the odds ratio represents the
constant effect of a predictor X, on the likelihood
that one outcome will occur.
Example, suppose, 1 = “Died”, 0 = “Survived”, and
OR for variable Age is 1.26.
This implies that the probability of surviving is
increase by 26% for older persons.
 ✓ Ordinal logistic regression (often just called 'ordinal
regression') is used to predict an ordinal dependent
variable given one or more independent variables. It
can be considered as either a generalization
of multiple linear regression or as a generalization
of binomial logistic regression.
✓ It is a more suited analysis when the dependent
ORDINAL variable is ordinal level, unlike binomial or
REGRESSION multinomial logistic regression where the dependent
variable is nominal level.
✓ It is the best substitute for multiple linear regression,
especially when normality assumption is violated. The
idea is to convert the dependent variable to ordinal
level, example, use low, moderate, and high values in
converting the data.
 1. Your dependent variable should be measured at
the ordinal level.
2. One or more of the independent variables
are continuous, ordinal or categorical. However, ordinal
independent variables must be treated as being either
continuous or categorical.
ORDINAL 3. No multicollinearity. Multicollinearity occurs when you
have two or more independent variables that are highly
REGRESSION - correlated with each other. This leads to problems with
understanding which variable contributes to the
ASSUMPTIONS explanation of the dependent variable and technical
issues in calculating an ordinal regression. Determining
whether there is multicollinearity is an important step
in ordinal regression.
4. Proportional odds. The assumption of proportional
odds means that each independent variable has an
identical effect at each cumulative split of the ordinal
dependent variable. (https://statistics.laerd.com/)
 • Beta Coefficient - the predicted change in log odd.
It could be negative or positive. Positive mean
STATISTICS increase in likelihood while negative means
decrease in likelihood.
AND
• The Wald test is used to determine statistical
DIAGNOSTICS significance for each of the independent variables.
• Odds ratio (EXP(B))- the odds ratio represents the
constant effect of a predictor X, on the likelihood
that one outcome will occur.