Ch2 - Econometrics For Finance (Regression Part)
Ch2 - Econometrics For Finance (Regression Part)
Regression Analysis
Simple & Multiple Regression
Assumptions
Estimation method(OLS)& Their properties (Precision)
Application Example
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
2. The Concept of Regression Analysis
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
….Regression and related concepts
1). Statistical Vs. Deterministic Relationships
In regression analysis, we are concerned with statistical dependence among variables (not
stochastic variables that use probability concepts or distribution. Hence, we couldn’t deal
deterministic r/nship.
2) Regression Vs. Causation
Regression does not necessarily imply causation, b/c statistical relationship cannot logically
imply causation rather causation comes from theory, intuition, common-sense etc.
3) Regression Vs. Correlation
random).
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
2.1. Simple Regression
Simple regression is an analysis of a single dependent variable, Y,
as a function of a single independent variable, X.
PRF Vs SRF
A population regression function (PRF) hypothesizes a theoretical
(population) relationship between a dependent variable and a set of
independent or explanatory variables.
Intercept w/c is fixed & unknown. Slope (parameter) w/c is fixed & unknown
𝑌𝑖 = 𝛽0 + 𝛽1 𝑋𝑖 + 𝜀𝑖 (2.1)
Dependent variable,
explained variable, Error term, disturbance,
response variable,… Independent variable, unobservables,…
explanatory variable,
regressor,…
(
Econometric for finance by Teshome H. MSc in Econometrics ) October 3, 2023
..cont’d
The sample regression function(SRF) is an equation that represents the
relationship between the Y variable and X variable(s) that is based
only on the information in a sample of the population.
Fitted(predicted)value Residuals
𝑌𝑖 = 𝛽0 + 𝛽1 𝑋𝑖 + 𝜀𝑖 (2.2)
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
…cont’d
Why we introduced error term?
We introduce the error tem to captures the effect of:
1). Omitted variables: in reality, a given variable is influenced by large
number of factors and all can’t be included b/c of:
Some factor may not be known
Some factor may not be measurable (e.g. test, preference, motivation etc. ).
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
2.1.1. Assumptions of Simple Regression (Gauss-Markov Assumptions)
2) The explanatory variable (X) are non-stochastic, but variability in value is necessary
3) No model specification and measurement error:
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
..cont’d
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
2.1.2. Methods of Estimations(Ordinary Least square)
After specifying the model and set their assumptions, the next step is estimation of
the parameters.
Method of moments
mathematically much simpler than other methods & has some desirable properties.
𝑛 2 n
𝜕 𝜀
𝑖=1 𝑖
=0 2 Yi − β0 − β1 Xi −1 = 0
𝜕β0
i=1
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
…cont’d
n n
− i=1 Yi + nβ0 + β1 i=1 X i = 0 , divided both side by 2
n n
Yi = nβ0 + β1 Xi
i=1 i=1
𝑌 = β0 + β1 X
𝛃𝟎 = 𝐘 − 𝛃𝟏 𝐗 (2.3)
The necessary condition for slope coefficient β1 (continued):
𝑛 2 n
𝜕 𝑖=1 𝜀𝑖
=0 2 Yi − β0 − β1 Xi −Xi = 0
𝜕β1 i=1
n n n
− Xi Yi + β0 Xi + β1 Xi 2 = 0
i=1 i=1 i=1
n n n
Xi Yi = β0 Xi + β1 Xi 2 (2.4)
i=1 i=1 i=1
Then, substitute equations (2.3) into equation (2.4) and obtained:
n n n
Xi Yi = 𝑌 − β1 X Xi + β1 Xi 2
i=1 i=1 i=1
(
Econometric for finance by Teshome H. MSc in Econometrics ) October 3, 2023
…cont’d
n n n n
Xi Yi = 𝑌 Xi − β1 X Xi + β1 Xi 2
i=1 i=1 i=1 i=1
n n n n
Xi Yi = 𝑌 Xi + β1 Xi 2 − β1 X Xi
i=1 i=1 i=1 i=1
n n n n
Xi Yi − Y Xi = β1 Xi 2 − X Xi
i=1 i=1 i=1 i=1
n n n
i=1 Xi Yi −Y i=1 Xi i=1 Xi Yi −nYX
β1 = n X 2 −X n X = n X 2 −nX2 , or
i=1 i i=1 i i=1 i
n n n
i=1 Xi Yi −nYX i=1 Xi −X Yi −Y i=1 xi yi Cov X,Y
β1 = n X 2 −nX2 = n 2 = n x 2 (2.5) Or β1 =
i=1 i i=1 Xi −X i=1 i Var X
(
Econometric for finance by Teshome H. MSc in Econometrics ) October 3, 2023
2.1.3. Properties of OLS estimators
Under the Gauss-Markov Theorem. the least squares estimators (OLS) should
Mathematically,
𝐸 β1 = β1
Similarly, 𝐸 β0 = β0
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
..cont’d
Property 3: Best: Minimum Variance
The efficient property of any estimator says that the estimator is
the minimum variance unbiased estimator.
OLS estimators have the least variance among the class of all
linear unbiased estimators.
M𝐚𝐭𝐡𝐞𝐦𝐚𝐭𝐢𝐜𝐚𝐥𝐥𝐲
𝜎2 ∗
var β1 = 2 < 𝑣𝑎𝑟 𝛽1
𝑥𝑖
𝑋𝑖2 ∗
var β0 = 𝜎 2 2 < 𝑣𝑎𝑟 𝛽0
𝑛 𝑥𝑖
Where : 𝑥𝑖 = 𝑋𝑖 − 𝑋, in deviation form and 𝑋𝑖 is in observed form, 𝛽0∗ & 𝛽1∗ are an
estimator obtained by other than OLS and hence, 𝑣𝑎𝑟 𝛽0∗ & 𝑣𝑎𝑟 𝛽1∗ are their variance,
respectively.
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
.cont’d
Property 4: Consistency
An estimator is said to be consistent if its value approaches
the actual, true parameter (population) value as the sample
size increases.
An estimator is consistent if it satisfies two conditions:
It is asymptotically unbiased
Mathematically,
Plim 𝛽 ~ 𝛽
𝑛→∞
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
…cont’d
The variance of error term(𝝈𝟐 )
It is necessary to estimate the variance of the errors terms(𝜎 2 ) from sample counterpart
𝜀𝑖2
because the population error term is not directly observable.𝜎𝜀2 = 𝑛−2 = 𝑆𝑆𝑅 𝑛 − 2
where SSR denotes the sum of the squared residuals and K is the number of regressors
(including the constant).
Note: The quantity 𝜎𝜀 is also called the Standard Error (S.E.) of the regression or the
Root Mean Squared Error (RMSE).
Thus, the reliability or precision of the estimators (β0 &β1 ) is given by its
standard error:
𝑋𝑖2
SE β0 = 𝜎𝜀
𝑛 𝑥𝑖2
1
SE β1 = 𝜎𝜀
𝑥𝑖2
The smallest standard error implies, the more reliability of the estimator .
(
Econometric for finance by Teshome H. MSc in Econometrics ) October 3, 2023
Application example for simple regression
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
..cont’d
Solution
A) To fit the regression equation we do the following computations.
n Yi Xi Yi Xi Xi2
1 10 30 300 900
2 20 50 1000 2500
3 30 60 1800 3600
Sum =60 =140 3100 7000
Mean Y = 20 140
X=
3
n XY ( X )( Y )
3(3100) (140)(60)
ˆ1 0.64
3(7000) (140) 2
n X 2
( X ) 2
(
Econometric for finance by Teshome H. MSc in Econometrics ) October 3, 2023
..cont’d
B) Interpretation:
The value of the intercept term,-10, implies that the value of the dependent
variable ‘Y’ is – 10 when the value of the explanatory variable is zero.
The value of the slope coefficient (β1 ) is a measure of the marginal change
in the dependent variable ‘Y’ when the value of the explanatory variable
increases by one. That is as income increase by birr 1, consumption
expenditure increase by 0.64 cents. 3
C). Predict expenditure when income is birr 45.
= -10+(0.64)(45) =18.8
That means when X assumes a value of 45, the value of Y on average is expected to
be 18.8.
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
2.2. Multiple Regression
Motivation for multiple regression
Incorporate more explanatory factors into the model
Thus, multiple linear regression (MLR) is a statistical technique that uses several
explanatory variables to predict the outcome of a response variable.
The general multiple regression model can be written in the population as:
(2.1)
Econometric
20 ( )
for finance by Teshome H. MSc in Econometrics October 3, 2023
..cont’d
The multiple linear regression model manages to hold the values of other
explanatory variables fixed even if, in reality, they are correlated with the
explanatory variable under consideration „Ceteris paribus“-interpretation
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
..cont’d
Other factors
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
…cont’d
In a simplified(vector-matrix) form equation (2.1) can be written as:
𝑦 = 𝑋𝛽 + 𝜀 (2.2)
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
2.2.1. Assumptions of Multiple CLRM
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
…cont’d
Assumption 3: Exogeneity
The regressors X1 , X2 , . . , Xk are distributed independently of the random
disturbance term
i.e. 𝐸 𝜀 𝑋 = 𝐸 𝜀 = 0
Implications
The explanatory variables are not predictors of the error terms.
Notes:
1 The condition of constant variances is called homoscedasticity.
(
Econometric for finance by Teshome H. MSc in Econometrics ) October 3, 2023
..cont’d
Comments: The two assumptions (homoscedasticity and nonautocorrelation)
imply that:
𝐸 𝜀−𝐸 𝜀 𝜀−𝐸 𝜀 ′ =𝐸 𝜀−0 𝜀−0 ′ =𝐸 𝜀 𝜀 ′
𝜀1 𝜀1 2 𝜀1 𝜀2 … 𝜀1 𝜀𝑛
𝜀2 𝜀2 𝜀1 𝜀2 2 … 𝜀2 𝜀𝑛
= 𝐸 . 𝜀1 𝜀2 . . 𝜀𝑛 =𝐸 ..
.
𝜀𝑛 𝜀𝑛 𝜀1 𝜀𝑛 𝜀2 … 𝜀𝑛 2
𝐸 𝜀1 2 𝐸 𝜀1 𝜀2 … 𝐸 𝜀1 𝜀𝑛 𝜎2 0 … 0
𝐸 𝜀2 𝜀1 𝐸 𝜀2 2 . . . 𝐸 𝜀2 𝜀𝑛 0 𝜎2 … 0 2𝐼
= = = 𝜎
.. ...
𝐸 𝜀𝑛 𝜀1 𝐸 𝜀𝑛 𝜀2 … 𝐸 𝜀𝑛 2 0 0 … 𝜎2
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
..cont’d
𝐸 𝜀′ 𝜀
Where 𝜎𝜀2 can be estimated from the sample variance, 𝜎 = 𝐸 𝜎 2 2
=
𝑛−𝑘
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
Application example
Example: Le see the effect of debt and income on consumption
expenditure over 15 observations defend by a model :
𝑐𝑜𝑛𝑠𝑒𝑥𝑝𝑒𝑛𝑑 = 𝛽0 + 𝛽1 𝑑𝑒𝑏𝑡 + 𝛽2 𝑖𝑛𝑐𝑜𝑚𝑒 + 𝜀
and the following data have been calculated from the
original X’s.
, 𝜀 ′ 𝜀 = 10.96
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
..cont’d
𝛽0 𝟏. 𝟏
= −𝟒. 𝟒
𝛽 = 𝛽1 = 𝑋 ′ 𝑋 −1
𝑋′𝑌 = 𝟏𝟗. 𝟖𝟖
𝛽2
To calculate the standard errors, we need an estimate of
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023
..cont’d
. The fitted model with its standard error is given by:
( )
Econometric for finance by Teshome H. MSc in Econometrics October 3, 2023