Regression Analysis

Lecture Outline

Regression Analysis

Principles

Estimation

Testing

Assumptions, Violations and

Therapy

Practical Issues

-2-
Basic Idea of Regression Analysis

The key goal of regression analysis is to determine the strength of impact of one
or more independent variables on a dependent variable

• What is the impact of education on income? Does one additional year of

education increase average income and if yes by how much?
INCOME

INCOME
EDUCATION EDUCATION

-3-
Basic Model

x1 β1 e.g. y = sales
x1 = unit price
indep.
β2 y dep. x2 = advertising budget
x2 x3 = salesman‘s visits
β3
var. var.
 = error term
x3
y    1 x1  2 x2  ...   j x j  

 Analyze associative relationships between a metric dependent variable

Y and one or more independent variables X.
 Reproduce the variation in the criterion Y as a linear combination of the
predictors X.
 Estimate parameters , β1, β2, β3 from data by e.g. minimizing the sum
of squared deviations (most frequently used).

-4-
 Applications

Regression analysis can be used in the following ways:

To determine whether a relationship between the dependent

and the independent variable exists.

To determine the strength of the relationship.

To determine the structure or form of the relationship.

Note: the estimated To predict the values of the dependent variable.

relationship a regression
analysis does not
necessarily imply any
causation. To control for other independent variables.

-5-
Bivariate Regression

Derive a mathematical
relationship between a
single metric
dependent variable
and a single metric
independent variable.

-6-
 Bivariate Regression – Basic Concepts

A line can be algebraically represented as: y = a + bx

y = dependent variable
x = independent variable
a = intercept or how much income an individual without education would make
b = how much additional income an extra year of education brings

Income

Change in income
Change in b = Slope =
Income Change in education

Change in Education
Education

-7-
Graphical Representation of Price-Sales Function

Price-sales function

y (quantity)

e3

y3 y3
ŷ 3 Regression function

x3

x (price)
x3 = price of observation no.3
y3 = quantity of observation no.3
ŷ 3 = estimated quantity for no.3 based on the regression function
e3 = estimation error (residual)

-8-
General Linear Regression Model

y    1 x1  2 x2  ...   j x j   y

dependent variable
intercept term
xj independent variable
j unknown parameter
Coefficient j indicate how much y changes (in units)  error term
when xj increases by one unit.

yˆ  ˆ  ˆ1 x1  ˆ2 x2  ...  ˆ j x j  

Empirical data for model variables
provide the following relation:

̂  the intercept term that represents

the value y when xi are zero.

̂ j  estimated values of the

unknown j coefficient

-9-
Lecture Outline

Regression Analysis

Principles

Estimation

Testing

Assumptions, Violations and

Therapy

Practical Issues

- 10 -
Regression Example - Where Is the Regression Line?

INCOME

EDUCATION

- 11 -
Regression Example - Where Is the Regression Line?

INCOME

EDUCATION

- 12 -
Regression Example - Where Is the Regression Line?

INCOME

EDUCATION

- 13 -
Data Matrix

Objective: Explain the variation in y through the regression relation and estimate.

yˆ  ˆ  ˆ1 x1  ...  ˆg xg

 y1 x11 x j1 x g1  y vector of dependent variable

 . . . . 
 
 . . . .  X matrix of values of independent variables
 
 . . . . 
 . . . . 
 
 y , x    yi x1i x ji x gi 
 . . . . 
 
 . . . . 
 . . . . 
 
 . . . . 
y x1N x jN x gN 
 N

- 14 -
Estimation

Model equation: y  Xβ  ε e vector of error terms (e1,...,en)

Estimation: yˆ  Xβˆ  e 𝛽෠ vector of unknown beta parameters (𝛽෠ 1,..., 𝛽෠ n)

Least Squares Estimation (LSE)

Fit a straight line into a scatterplot that is estimate the unknown beta parameters by
minimizing the vertical distances of all the points from the line (i.e minimize the sum of
squared error terms):

e 2
 ee  (y  Xβˆ)(y  Xβˆ)  yy  2 yXβˆ  βˆ X Xβˆ min
β

Minimum obtained by differentiating e´e

Estimated coefficient:
with respect to ˆ and setting it to be equal
to zero:
ee ̂  ( X X ) 1 X y
 2 X y  2 X Xˆ  0


- 15 -
Explanatory Power of Regression

Good fit: small SSE, even Good fit: moderate SSE,

compared with small total but large total spread
spread

Bad fit: big Bad fit: big

SSE, relative SSE, relative
to small total to small total
spread spread

- 16 -
Variation Decomposition – Graphical Representation

𝑦𝑖

𝑦ෝ𝑖

- 17 -
Variation Decomposition - Formula

Explained Unexplained
Total variation = variation + variation

 y i  y     yˆ i  y    yi  yˆ i
2 2
 
2

In regression analysis, the total variation in y can be decomposed into (following

the fundamental theorem of analysis of variance). y is the sample mean of y.

Optimization problem: Solution (Least Squares Method):

̂   X X 1 X Y
Min   y  yˆ  
2
i i
αˆ,βˆ1,...,βˆ j
̂  y    j x j

Note: The Least Squares Method is also referred to as OLS (Ordering least squares)

- 18 -
Regression Analysis – Statistics

Definition of statistical terms

Regression Ordinate axis intercept; predicted value of Y for zero values of the independent
constant  variable.

Regression Slope of the regression function; signifies how much the dependent variable
coefficient i changes for one unit change of the independent variable.

Standardized
Slope obtained out of standardized data; signifies how important an
regression independent variable is in explaining the dependent variable.
coefficient i*
Coefficient of Measures the strength of association R2=[0;1]; signifies the proportion of total
determination R2 variance in Y that is accounted for by X.
Coefficient of determination adjusted for the number of independent variables
Adjusted R2 included in the model and the sample size to account for the diminishing
returns.
Used to test the null hypothesis that no linear relationship exists between Xi
t statistic and Y; H0: βi = 0.
Used to test the null hypothesis that the coefficient of determination is zero; H0:
F statistic β1 = β2 = … = βi = 0

- 19 -
Lecture Outline

Regression Analysis

Principles

Estimation

Testing

Assumptions, Violations and

Therapy

Practical Issues

- 20 -
Test the Regression Equation

1. Determine strength of association

 Coefficient of determination R2
I

 i i
y  ˆ
y 2

explained variance unexplained variance

R2  1  1 i 1
I
total variance total variance

 iy  y 2

i 1

 Adjusted R2 adjusts for the number of predictions:

n -1
R  1  (1  R 2 ) K= number of exploratory variables
n = total number of observation
n-K

- 21 -
Test the Regression Coefficients

2. Test association for significance

 F-test : H0: β1 = β2 = … = βi = 0 At least one exploratory variable

explains the dependant variable.
H1: β1 ≠ β2 ≠ … ≠ βi ≠ 0
R2
 Ftab > Femp  H0 not rejected K
Femp 
 F follows an F distribution with degrees of 1 R 2

n  K 1
freedom K and n-K-1
 Femp is compared to the critical F-value K= number of independant variables
n= total number of observations
(Ftab) at significant level 

3. Test coefficients for significance

 t -test: is the parameter significantly different than zero? bj

H0: βi = 0 T emp 
H1: βi ≠ 0 σbj

- 22 -
Test the Regression Coefficients

4. Determine importance of variables

 Standardize the coefficients to neutralize the impact of different scales and scale
units and to make the coefficients comparable:
− independent from linear transformations
s( x j )
− absolute measure for the importance of influence factors b  bj
*
j
− influence expressed as part of standard deviation in y s( y)
− is similar to a path coefficient in path analysis

How many standard deviations units does the dependent variable change
if the independent variable changes by one standard deviation.

Can only be interpreted with and relative to the other variables in the equation,
will likely change if additional variables are added.

- 23 -
Test the Regression Model – Example

Model Summary

Std. Error of the

Model R R Square Adjusted R Square Estimate
1 .615a .378 .369 2.051

ANOVA
Sum of
Model Squares df Mean Square F Sig.
1 Regression 564.445 3 188.148 44.723 .000a
Residual 929.750 221 4.207
Total 1494.196 224

Coefficientsa,b
Unstandardized Standardized
Coefficients Coefficients
Model B Std. Error Beta t Sig.
1 (Constant) -1.232 .671 -1.836 .068
x3 I have stylish clothes .252 .099 .136 2.542 .012
x5 Life is too short not .928 .089 .566 10.382 .000
to take some gambles.
x7 government is doing .101 .094 .058 1.079 .282
too much to control
pollution.

- 24 -
Confidence Interval around βi

 Like all parameter estimate the estimates of the i population parameters are
point estimates

 The 95% confidence interval is given

by the following equation:

β̂i  SE β̂ i * t α/2 ; β̂i  SEβ̂ i * t α/2 

- 25 -
Confidence Intervals of a Linear Regression - SPSS

- 26 -
Confidence Intervals of a Linear Regression - SPSS

- 27 -
Confidence Intervals of a Linear Regression - SPSS

Tick the case

“Confidence
intervals”
4

5
Type in the confidence
interval you select e.g. for
95% write 95, for 99% put
99

7
- 28 -
Confidence Intervals of a Linear Regression - SPSS

You will find in the Coefficients table two

additional columns: “Lower Bound” and
“Upper Bound”. These are the extreme B
coefficients at a 95% confidence interval.

- 29 -
Testing Subsets of Regression Coefficients

Testing subsets of coefficients

yi    1 x1i   2 x2i   3 x3i   4 x4i  

 Test whether 3 and 4 are jointly equal to zero

 Run model without X3 and X4 in the regression get R2R (R=restricted)
 Run model with X3 and X4 in the equation  get R2UR (R=unrestricted)

R 2
UR  R2R  H0: 3=4=0
F h H1: at least one coefficient is unequal to zero
1  R  2
UR
Note can also test whether i=j
n  K 1

h =equals number of restrictions, in our case 2 (3=4=0)

n= number of observations
K=number of exploratory variables, in our case 4

- 30 -
Lecture Outline

Regression Analysis
Multicollinearity
Principles

Nonlinearity
Estimation

Heteroscedasticity
Testing

Assumptions, Violations and Autocorrelation

Therapy

Practical Issues

- 31 -
Assumptions of Regression Analysis

A1 Linearity and Additivity of the Relationship between dependent and independent

Variables
1. Expected value of the DV is a straight line function of each IV, holding other IVs fixed

2. Slope of that line does not depend on values of other IVs

3. Effects of different IVs are additive, i.e. total effect of the X’s on the expected value of Y is the
sum of their separate effects.
A2 Constant Variance of error terms (Homoscedasticity)
1. over time (in case of time series data)

2. over predictions

3. over the values of IVs

A3 Statistical Independence of error terms

A4 Normality of error terms

If any of these assumptions is violated then the model results may

be at best inefficient and at worst seriously biased or misleading!!!

- 32 -
Normality of Error Term - Diagnosis

Normal Quantile Plot

Actual Standardized Residual

Actual
Theoretical

Theoretical Standardized Residual

Adjusted Anderson-Darling statistic in 2.591 (P = 0)

Note: Other tests for normality are available such as Kolmogorov-Smirnov test or Jarque-
Bera test are available

- 33 -
Normality of Error Term - Diagnosis

Actual Standardized Residual Normal Quantile Plot

Actual
Theoretical

Theoretical Standardized Residual

Adjusted Anderson-Darling statistic in 0.335 (P = 0.508)

- 34 -
Normality of Error Term - Therapy

 Might result in too wide or too narrow confidence intervals.

 Could be a result of a few data points that deviate significantly from the
regression line. Thus investigating the normal quantile plot can help identifying
influential outliers.
 Non-linear transformations (log-transformations) can help.

- 35 -
Assumptions‘ Violations – Multicollinearity

Multicolinearity
Assumption:
Independent variables are not substantially correlated Non-Linearity

Multicollinearity: Heteroscedasticity

An independent variable is correlated with another independent variable

Autocorrelation
or with a linear combination of other explanatory variables.

Effect
• Least-square (LS) estimator for the regression coefficient is given by the formula: ̂   X X 1 X Y
• In the case of perfect multicollinearity, the matrix  X X  is singular and its inverse matrix does not exist
 ̂ cannot be estimated.
• In cases of high degree of multicollinearity, a solution for the LS estimator can still be found.
• But: the higher the linear association, the lower the determinant  X X  of ̂ is. The consequence is
inflation of variances and covariances of the estimated coefficients  Coefficients become imprecise
and unstable.

- 36 -
 Multicollinearity – Visualization

Multicolinearity

Ideal situation
X1 Non-Linearity

 X1 is independent of X2
Y Heteroscedasticity
 allows a good regression analysis
Autocorrelation
X2

Multicollinearity
X1
 Correlation > 0.7 between X1 and X2
 If the correlation between X1 and X2 is < 1, SPSS
Y a considers both variables (even if high)
 If the correlation between X1 and X2 is equal to 1, SPSS
put the variable off the system.
X2
 Difficulty of assigning shared variance a to either X1 or X2

- 37 -
 Multicollinearity - Diagnosis

Each independent variable is regressed against the remaining Multicolinearity

independent variables.
Non-Linearity
x2
Rij
Heteroscedasticity

Autocorrelation
x1
x3

x1    1 x2   2 x3  ...   i 1 xi

€€

x4

- 38 -
Multicollinearity - Diagnosis

Multicolinearity
Tolerance value Variance inflation factor (VIF)
Non-Linearity

1
1  Rij  Tij VIF 
Heteroscedasticity

1  Rij Autocorrelation

 Each independent variable is regressed against the remaining

independent variables.
 Tolerance is the amount of variance of the selected independent
variable not explained by other independent variables.

Standard rules of thumb such as a VIF of 10 still allow substantial amounts of

collinearity (corresponds to a multiple correlation of 0.95). A VIF of greater than
5 should cause concern researcher should assess robustness of their results.

- 39 -
Multicollinearity – Therapy

Multicolinearity
Therapy for multicollinearity:
Non-Linearity

Heteroscedasticity
Elimination of interdependent variable based on
correlation and tolerance value Autocorrelation

Factor analysis of explanatory variables

Increasing information base (larger sample)

Ridge regression, Shapley values

- 40 -
 Assumptions‘ Violations – Nonlinearity Diagnosis I

No apparent non-linearity, Non-linearity likely present

random pattern of residuals Non-random pattern of residuals
residual

residual

predicted value predicted value

- 41 -
 Assumptions‘ Violations – Nonlinearity Diagnosis II

Examine plots of IV versus DV and plots of residuals versus predicted

values

Example: tire tread and mileage

groove in millim

Residual

mileage in 1000 miles predicted value groove

- 42 -
Assumptions‘ Violations – Nonlinearity Effects and Therapy

Multicolinearity
Effect
Non-Linearity
Parameters are biased: as sample size increases, parameter estimations
tend to over or underestimate the true value. Predictions based on a linear Heteroscedasticity
model that is in fact non-linear can be substantially wrong.
Autocorrelation

Therapy:

Variables‘ transformation - Logarithm / Exponential functions

Add higher order terms of the variable y  α  β1 x1  β2 x  β3 x3  ε

2

- 43 -
Assumptions‘ Violations – Nonlinearity Effects and Therapy

Therapy: Multicolinearity

Non-Linearity

Add higher order terms of the variable y  α  β1 x1  β2 x 2  β3 x3  ε Heteroscedasticity

Autocorrelation

Linear Quadratic Cubic

- 44 -
 Example Non-Linear Regression Model with Sensory Data

Multicolinearity
With sensory data there are acceptable Non-Monotonic Relations
ranges of attribute values that lead to Non-Linearity
quasi equivalent preferences.
Heteroscedasticity
Unaccept Acceptable Unaccept-
able range able Autocorrelation
Outside this acceptable range

Preference
bitterness bitterness
preference decline substantially.

A flexible way to model non-linear

monotonic and non-monotonic
relationships is through polynomials.

Preferences can be estimated as a Bitterness of beer

function of attribute values by using
equation the form of:

Preference ( y)  α  β1 Att1  β2 Att12  β3 Att13  ε

- 45 -
 Example Non-Linear Regression Model - Results

Liking Rating ( y )  α  β1 Jelly Quantity  β2 Jelly Quantity 2  ε Multicolinearity

Key:
Non-Linearity
Acceptable Unacceptable
Liking Rating

range range
Heteroscedasticity

R²= 16.5% Autocorrelation

b1 = 0.190; b2 = -0.020
β1 = 1.409; β2 = -1.305

Jelly Quantity Scale: 1 = No Jelly, 7 = >50% of Jelly

Liking Rating ( y )  α  β1Quantity Meat  β2Quantity Meat 2  ε R²= 15%
b = 0.378; b² = -0.055
β = 2.210; β² = -2.070
Liking Rating

R²= 15%
b1 = 0.378; b2 = -0.055
β1 = 2.210; β2 = -2.070

Quantity of Meat Pieces Scale: 1 = No Meat Pieces, 7 = A Lot of Meat Pieces

- 46 -
 Heteroscedasticity – Diagnosis

Multicolinearity

Non-Linearity

Heteroscedasticity

Residuals
Residuals

Autocorrelation

𝑦ෝ𝑖 , x𝑖 𝑦ෝ𝑖 , x𝑖

No evidence – variance Absolute magnitude of

of residuals does not residual increases as X
increase over x or y increases

- 47 -
Assumptions‘ Violations – Heteroscedasticity

Homoscedasticity Multicolinearity

 Variance of the residuals is constant and doesn‘t depend on 𝑦ෝ𝑖 , x𝑖 Non-Linearity

Heteroscedasticity
Heteroscedasticity
 Variance of the residuals is not constant and doesn‘t depend on 𝑦
ෝ𝑖 , x𝑖 Autocorrelation

Effect
 OLS estimators are unbiased but no longer efficient, standard errors of the estimates
might be wrong and confidence intervals might be too narrow or too large and impact
predictions.

Diagnosis with a test:

e.g. Goldfeld & Quandt test H 0 : E ( 2 )   2

(1965) H 1 : E ( 2 )   2 xit2

- 48 -
Heteroscedasticity – Therapy

Multicolinearity

Non-Linearity
Can be a byproduct of linearity, independence assumption
thus i.e. conduct variable transformation, add higher order Heteroscedasticity
terms
Autocorrelation

Estimate separate regression functions with OLS for

which the variance of the residuals is more or less
constant (different ranges of y)

GLS (Generalized Least Square) estimation

- 49 -
Autocorrelation – Diagnosis I

Multicolinearity

Non-Linearity

Heteroscedasticity

Autocorrelation

Positive autocorrelation

- 50 -
Autocorrelation – Diagnosis II

Multicolinearity

Non-Linearity

Heteroscedasticity

Autocorrelation

Negative autocorrelation

- 51 -
Assumptions‘ Violations – Autocorrelation

Autocorrelation of the residuals Multicolinearity

 Assumption of uncorrelated residuals is violated, i.e E(et ,et 1 )  0

Non-Linearity
 Plot the residuals against time and examine their pattern – random pattern should be
seen if assumption is true (i.e. residuals are not correlated). Heteroscedasticity
 Mostly a time series problem (or measurement error)
Autocorrelation

Effect
• ̂is further on unbiased, i.e.
• But error variance is no longer minimized, i.e.
• Estimators are no longer efficient.
• All tests for regression coefficients and confidence intervals are incorrect.
• Estimators of the error variance underestimate the real error variance.

- 52 -
Autocorrelation – Durbin-Watson Test (I)

Multicolinearity
Examine correlations between the error
Durbin-Watson-Test terms. Test statistic (1st order serial Non-Linearity
correlation):
Heteroscedasticity
T
 t t 1
( e  e ) 2
Autocorrelation

H0: No autocorrelation exists. DW  t 2 T

H1: Autocorrelation exists.  et2
t 1

 Difficult to interpret exactly the value of the test statistic, since it is dependent on the
number of independent variables and the number of observations.
 Two types of serial correlation (positive & negative)  two interpretation intervals
 Rule of thumb: DW  2 means no autocorrelation
 DW test only test for auto-correlation of lag 1. Weekly/seasonal effect will not be
detected

- 53 -
Autocorrelation – Durbin-Watson Test (II)

Test for positive / negative 1st order residual autocorrelation: Multicolinearity

DW-value Result Non-Linearity

4 - dL < DW < 4 reject null hypothesis, negative serial autocorrelation

Heteroscedasticity
4 - dU < DW < 4 - dL result unclear
2 < DW < 4 - dU accept null hypothesis Autocorrelation

dU < DW < 2 accept null hypothesis

dL < DW < dU result unclear
0 < DW < dL reject null hypothesis, positive serial correlation
dL (lower critical value) and dU (upper critical value) are tabulated for different values of k
(number of explanatory variables)

Interpretation intervals
dL; /2: lower critical value at
positive negative
autocorrelation auto- significance level /2.
correlation
dU; /2: upper critical value at
significance level /2.
0 dL dU 2 4 - dU 4 - dL 4 : Unclear area

Another identification (e.g. higher order)

- 54 -
Autocorrelation – Therapy

Multicolinearity

Estimate a regression function yt   0  1 X 1t  ...   n ynt   t

Non-Linearity
for the changes instead of the
 p yt  yt  pyt 1
absolute values: Heteroscedasticity

 p xt  xit  pxt 1 Autocorrelation

 p yt   0 (1  p)  1 p xit  ...   n  p xnt  ~t

Estimate a regression for the changes instead of the absolute values

1.

2. Use timeseries analysis

- 55 -
Testing Assumptions – Sequence of Analysis

Examine residual scatterplots to test for

assumptions of normality, linearity and
homoscedasticity between predicted DV
errors of prediction.

Residuals (differences between obtained

and predicted DV scores) should be
normally distributed about the predicted
DV scores.

Residual should possess a straight line

relationship with predicted DV scores.

Variance and residuals about predicted

DV scores is the same for all predicted
scores.

- 56 -
Assumptions‘ Violations – Diagnosis and Therapy (I)

Phenomenon Diagnosis Therapy

Data transformation
Scatter plot
Nonlinearity Linearity test
Nonlinear regression with higher
order terms of IVs

Eliminate variables
Correlation matrix Factor score
Multicollinearity Tolerance test Increase information base
Ridge regression, Shapley values

Analyze changes instead of

Plot residuals against time
Autocorrelation Durbin-Watson-Test
absolute values as DV
Use time series approach

Plot residuals against dependent Transform independent variable or

variable regression relation
Heteroscedasticity Goldfeld & Quandt test or other Conduct separate regressions
tests Conduct GLS regression

- 57 -
Lecture Outline

Regression Analysis

Principles

Estimation

Testing

Assumptions, Violations and

Therapy

Practical Issues
Sample Size

In general, sample size depends on desired

Sample size power, number of predictions and expected
effect size.

 Rules of thumb by Green (1991) are N  50 + 8m (where m is the number

of IVs) for testing multiple correlations and N  104 + m for testing individual
predictions (assuming a medium sized effect  = 0.2)

Example Note

If you have a regression with 6  With large samples almost any multiple
predictions, you need 50+ correlation will become significant.
(8x6) = 98 cases to test  For stepwise regression, even more
regression and 104+6 cases to cases are needed (cases to IV ratio
test individual predictions. 40:1)

- 59 -
Non-Metrical Independent Variables

 In case the independent variables are non-metrical these variables (e.g.

Colour of product, education of a customer etc.) cannot simply be included
in the regression analysis.

 The independent variable can be dummy coded. That is for each of its levels
a zero/one variable is created that indicates whether the level is present (1)
or absent (0)

Case Preference Colour Case Preference Colour Red Yellow Blue

1 5 Red 1 5 Red 1 0 0
2 2 Yellow 2 2 Yellow 0 1 0
3 4 Blue 3 4 Blue 0 0 1
4 3 Red 4 3 Red 1 0 0
5 1 Yellow 5 1 Yellow 0 1 0
… … … … … … … … …

- 60 -
Major Types of Multiple Regression

They are 3 major analytic strategies Differences among the strategies

in multiple regression involve:
 What happens to overlapping
variability due to correlated
Standard multiple
X1…n
regression
 Who determines the order of
entry of X1…n into the equation

Sequential
regression

Statistical/Stepwise
regression

- 61 -
Major Types of Multiple Regression

Observed Correlations
X
1
X1 with X2 (high)

X1 with Y
Y
a X2 with Y
X2 X3 with Y (low)
b
X3 with X2 (negligible)

c
d
e Area a comes from X1
Area b comes from ? (ambiguity btw. X1 and X2 )
€€ Area c comes from X2
Area d comes from ? (ambiguity btw. X2 and X3 )

Area e comes from X3

X3
The allocation of areas b and d will be affected by the
choice of the strategy, as the apparent importance of the
R2 = area (a+b +c+d+e) various IVs to the solution will variates.

- 62 -
Standard Multiple Regression

X Attribution
1
• Each X is assigned only the area of its unique
contribution.
Y
a
• The overlapping areas b and d contribute to R2, but are
b X2 not assigned to any of the individual X.

c
d
e Interpretation

€€ In this case, the variable X2 appears as unimportant. The

problem is that, in standard multiple regression, it can look
so, as it is in fact highly correlated with Y.

For this reason, the full correlation AND the unique

X3 contribution for the X need to be considered in
interpretation.

R2 = area (a+b +c+d+e)

- 63 -
Sequential Multiple Regression

Attribution
X
• X1…n enter the equation in an order specified by the
1
researcher. In this case, we assume that the researcher
assigned X1 = first entry, X2 = second entry and X3 =
Y third entry.
a
• X1 has priority and is assigned areas a and b
b X2 • X2 arrives second and is assigned areas c and d
• X3 is assigned e, the only area remaining
c
d • If X2 was first, its importance would have increased
e dramatically, as it would have been assigned b, c and d.

€€ Precision
The researcher normally assigns order of entry of variable
according to logical or theoretical considerations.
X3
e.g “height” can be considered as a more important factor
than “amount of training” in accessing success as a
basketball player and would therefore be assigned first.
R2 = area (a+b +c+d+e)

- 64 -
Statistical/Stepwise Regression

Y
1
Attribution
X • Controversial procedure
a
• Order of entry of variables is based solely on statistical
b Y2 criteria.

c
d
e

€€

Y3

R2 = area (a+b +c+d+e)

- 65 -