MULTIVARIATE ANALYSIS

• Statistical techniques that simultaneously

analyze more than two variables
• Multivariate techniques – two categories
1. Dependency techniques – deal with one or
more dependent variables
• One dependent variable - data metric –
Multiple Regression Analysis
• One dependent variable – data non metric
– Discriminant Analysis
2. Interdependency techniques
• More than two variables
• Variables not segregated as dependent
and independent variables
• Interrelationships between the variables
are analyzed
• Data metric – Factor Analysis, Cluster
Analysis
1
 MULTIVARIATE ANALYSIS
Multiple Regression
Used to analyse quantitative data
To study cause and effect relationship
between a single dependent variable
and two or more than two
independent variables
Used mainly for prediction/
forecasting

2
 Multiple Regression Analysis
The general multiple regression with k
independent variables is given by:
Y ' = a + b1 X 1 + b2 X 2 + ...+ bk X k
 Greek letters are used for a (α ) and b (β ) when

 denoting population parameters.

a is the Y-intercept.
 X1 to Xk are the independent variables.
 bj is called a partial regression coefficient. It is the
net change in Y for each unit change in Xj holding all
other variables constant, (where j=1 to k) 3
ASSUMPTIONS IN MULTIPLE REGRESSION

The independent variables The dependent

variable must be
and the dependent variable
continuous and at
have a linear relationship. least interval-scaled.
The residuals should follow
the normal distributed
with mean 0.
The variation in (Y-Y’) or
residual must be the same
Successive values of the for all values of Y. When
dependent variable must this is the case, we say the
be uncorrelated or not difference exhibits
autocorrelated. homoscedasticity.
homoscedasticity 4
 Correlation Matrix
Correlation Cars Advertising Sales force
Coefficients

Cars 1.000

Advertising 0.808 1.000

Sales force 0.872 0.537 1.000

A correlation matrix is used to show all possible

simple correlation coefficients among the variables.
The matrix is useful for locating correlated
independent variables.
It shows how strongly each independent variable is
correlated with the dependent variable.
 Multiple Regression
Analysis
The least squares criterion is
used
to develop this equation.
Because determining b1, b2, etc. is
very tedious, a software package
such as Excel or MINITAB may be
used.
6
 ANOVA Explained Variation
ANOVA TABLE Variation
accounted
Source df SS MS for by the
Regression k SSR SSR/(k-1) set of
Σ (Y’–Y)2 independen
t variables.
Error n-k-1 SSE SSE/(n-k-1)
Σ (Y-Y’)2
Total n-1 SS Total
Σ (Y-Y)

Unexplained or Random Variation Total Variation

Variation not accounted for by

the independent variables. 7
EXAMPLE 1 A market researcher for Super
Markets is studying the yearly
amount families of four or more
spend on food. Three
independent variables are
thought to be related to yearly
food expenditures (Food). Those
variables are: total family
income (Income) in $00, size of
family (Size), and whether the
family has children in college
(College). 8
Expenditure = a + b1*(Income) +

b2(Size) +
b3(College)
The variable college is called a dummy
or indicator variable. It can take
only one of the two possible outcomes
i.e. a child is a college student or not.
Examples of dummy variables: gender,
the part is acceptable or not, the voter
will or will not vote for the incumbent
governor etc.
We usually code one value of the
dummy variable as “1” and the other
“0.”
9
Fam ily Food Incom e Size Student
1 3900 376 4 0
2 5300 515 5 1
3 4300 516 4 0
4 4900 468 5 0
5 6400 538 6 1
6 7300 626 7 1
7 4900 543 5 0
8 5300 437 4 0
9 6100 608 5 1
10 6400 513 6 1
11 7400 493 6 1
12 5800 563 5 0

Example 1 continued
10
 Example 1 continued

•From the analysis provided by MINITAB,

the estimated multiple regression equation is:
Y’ = 954 +1.09X1 + 748X2 +
565X3
Food Expenditure= 954+1.09*income+748*size
+565*college

? What food expenditure would you estimate for a

family of 4, with no college student, and an income
of $50,000 (which is input as 500)? 11
 Food Expend.=$954+$1.09*income+
$748*sizeExample 1 continued
+$565*college
Each additional $100 dollars of income per year will
increase the amount spent on food by $109 per year.
An additional family member will increase the
amount spent per year on food by $748.
A family with a college student will spend $565 more
per year on food than those without a college student.
So a family of 4, with no college students, and an
income of $50,000 will spend an estimated $4,491.

Food Expend.=$954+$1.09*500+$748*4+
$565*0 12
The regression equation is
Food = 954 + 1.09 Income + 748 Size + 565 Student

Predictor Coef SE Coef T P

Constant 954 1581 0.60 0.563
Income 1.092 3.153 0.35 0.738
Size 748.4 303.0 2.47 0.039
Student 564.5 495.1 1.14 0.287

S = 572.7 R-Sq = 80.4% R-Sq(adj) = 73.1%

Analysis of Variance
Source DF SS MS F P
Regression 3 10762903 3587634 10.94 0.003
Residual Error 8 2623764 327970
Total 11 13386667

13
Example 1 continued
The coefficient Food Income Size College
of determination
is 80.4 percent. Food 1.000
This means that
Income 0.587 1.000
more than 80
Size 0.876 0.609 1.000
percent of the
College 0.773 0.491 0.743 1.000
variation in the
amount spent on
food is Correlation matrix
accounted for by
the variables
income, family
The strongest correlation between the
size, and
dependent variable and an independent
student.
variable is between family size and amount
14
spent on food.
Conduct an individual test to
determine which coefficients are not
zero. This is the hypotheses for the
independent variable family size.
H0 : β2 = 0 H1: β2 ≠ 0
From the MINITAB
output, the only Thus, using the
significant variable 5% level of
is FAMILY (family significance,
size) using the p- reject H0 if the p-
values. The other
value < .05.
variables can be
omitted from the Example 1 continued15
If we rerun the analysis using only the
significant independent variable i.e.
family size. The new regression
equation is:
Y’ = 340 + 1031X2
The coefficient of determination is
76.8 percent. We dropped two
independent variables, and the R-
square term was reduced by only 3.6
percent.

Example 1 continued16
 Regression Analysis: Food versus Size

The regression equation is:

Food = 340 + 1031 Size
Predictor Coef SE Coef T P
Constant 339.7 940.7 0.36 0.726
Size 1031.0 179.4 5.75 0.000

S = 557.7 R-Sq = 76.8% R-Sq(adj) = 74.4%

Analysis of Variance

Source DF SS MS F P
Regression 1 10275977 10275977 33.03 0.000
Residual Error 10 3110690 311069
Total 11 13386667

Example 1 continued
17
A residual is the difference between the actual
value of Y and the predicted value Y’.

Residuals should follow the normal distribution.

Histograms are useful in checking this
requirement.

A plot of the residuals and their corresponding

Y’ values is used for showing that there are no
trends or patterns in the residuals.

Analysis of Residuals
18
 Residual Plots against Estimated Values of Y

1000
Residuals

500

-500
4500 6000 7500

Y’ Residual Plot
19
 8
7
6
Frequency

5
4
3
2
1
0
-600 -200 200 600 1000

Residuals

Histograms of Residuals
20