Introduction To Econometrics
Uploaded by
yesuneh98Introduction To Econometrics
Uploaded by
yesuneh98Hawassa University
Department of Economics Introduction to Econometrics
CHAPTER 1
Introduction
1.1. Definition and Scope of Econometrics
The economic theories you have learnt in various economics courses suggests
many relationships among economic variables. For instance, in microeconomics
we learn demand and supply models in which the quantities demanded and
supplied of a good depend on its price. In macroeconomics, we study ‘investment
function’ to explain the amount of aggregate investment in the economy as the
rate of interest changes; and ‘consumption function’ that relates aggregate
consumption to the level of aggregate disposable income.
Each of such specifications involves a relationship among economic variables. As
economists, we may be interested in questions such as: If one variable changes in
a certain magnitude, by how much will another variable change? Also, given that
we know the value of one variable; can we forecast or predict the corresponding
value of another? The purpose of studying the relationships among economic
variables and attempting to answer questions of the type raised here help us to
understand the real economic world we live in.
However, economic theories that postulate the relationships between economic
variables have to be checked against data obtained from the real world. If
empirical data verify the relationship proposed by economic theory, we accept the
theory as valid. If the theory is incompatible with the observed behavior, we either
reject the theory or in the light of the empirical evidence of the data, modify the
theory. To provide a better understanding of economic relationships and a better
guidance for economic policy making we also need to know the quantitative
By: TDT Page 1
Hawassa University
Department of Economics Introduction to Econometrics
relationships between the different economic variables. We obtain these
quantitative measurements taken from the real world. The field of knowledge which
helps us to carry out such an evaluation of economic theories in empirical terms is
econometrics.
What is Econometrics?
Literally interpreted, econometrics means “economic measurement”, but the scope
of econometrics is much broader as described by leading econometricians. Various
econometricians used different ways of wordings to define econometrics. But if we
distill the fundamental features/concepts of all the definitions, we may obtain the
following definition.
“Econometrics is the science which integrates economic theory, economic statistics, and
mathematical economics to investigate the empirical support of the general schematic law
established by economic theory. It is a special type of economic analysis and research in
which the general economic theories, formulated in mathematical terms, is combined with
empirical measurements of economic phenomena. Starting from the relationships of
economic theory, we express them in mathematical terms so that they can be measured.
We then use specific methods, called econometric methods in order to obtain numerical
estimates of the coefficients of the economic relationships.”
Measurement is an important aspect of econometrics. However, the scope of
econometrics is much broader than measurement. As D.Intriligator rightly stated
the “metric” part of the word econometrics signifies ‘measurement’, and hence
econometrics is basically concerned with measuring of economic relationships.
By: TDT Page 2
Hawassa University
Department of Economics Introduction to Econometrics
In short, econometrics may be considered as the integration of economics,
mathematics, and statistics for the purpose of providing numerical values for the
parameters of economic relationships and verifying economic theories.
1.2. Econometrics vs. Mathematical Economics
Mathematical economics states economic theory in terms of mathematical
symbols. There is no essential difference between mathematical economics and
economic theory. Both state the same relationships, but while economic theory
uses verbal exposition, mathematical economics uses symbol expression. Both
express economic relationships in an exact or deterministic form. Neither
mathematical economics nor economic theory allows for random elements which
might affect the relationship and make it stochastic. Furthermore, they do not
provide numerical values for the coefficients of economic relationships.
Econometrics differs from mathematical economics in that, although econometrics
presupposes, the economic relationships to be expressed in mathematical forms, it
does not assume exact or deterministic relationship. Econometrics assumes
random relationships among economic variables. Econometric methods are
designed to take into account random disturbances which relate deviations from
exact behavioral patterns suggested by economic theory and mathematical
economics. Furthermore, econometric methods provide numerical values of the
coefficients of economic relationships.
1.3. Econometrics vs. Statistics
Econometrics differs from both mathematical statistics and economic statistics.
An economic statistician gathers empirical data, records them, tabulates them or
charts them, and attempts to describe the pattern in their development over time
By: TDT Page 3
Hawassa University
Department of Economics Introduction to Econometrics
and perhaps detect some relationship between various economic magnitudes.
Economic statistics is mainly a descriptive aspect of economics. It does not
provide explanations of the development of the various variables and it does not
provide measurements the coefficients of economic relationships.
Mathematical (or inferential) statistics deals with the method of measurement
which is developed on the basis of controlled experiments. But statistical methods
of measurement are not appropriate for a number of economic relationships
because for most economic relationships controlled or carefully planned
experiments cannot be designed due to the fact that the nature of relationships
among economic variables are stochastic or random. Yet the fundamental ideas of
inferential statistics are applicable in econometrics, but they must be adapted to
the problem economic life. Econometric methods are adjusted so that they may
become appropriate for the measurement of economic relationships which are
stochastic. The adjustment consists primarily in specifying the stochastic (random)
elements that are supposed to operate in the real world and enter into the
determination of the observed data.
1.4. Economic Models vs. Econometric Models
I) Economic Models:
Any economic theory is an observation from the real world. For one reason, the
immense complexity of the real world economy makes it impossible for us to
understand all interrelationships at once. Another reason is that all the
interrelationships are not equally important as such for the understanding of the
economic phenomenon under study. The sensible procedure is therefore, to pick
up the important factors and relationships relevant to our problem and to focus
our attention on these alone. Such a deliberately simplified analytical framework
By: TDT Page 4
Hawassa University
Department of Economics Introduction to Econometrics
is called an economic model. It is an organized set of relationships that describes
the functioning of an economic entity under a set of simplifying assumptions. All
economic reasoning is ultimately based on models. Economic models consist of
the following three basic structural elements.
1. A set of variables
2. A list of fundamental relationships and
3. A number of strategic coefficients
Ii) Econometric Models:
The most important characteristic of econometric relationships is that they contain
a random element which is ignored by mathematical economic models which
postulate exact relationships between economic variables.
Example 1.1: Economic theory postulates that the demand for a commodity
depends on its price, on the prices of other related commodities, on consumers’
income and on tastes. This is an exact relationship which can be written
mathematically as:
Q b 0 b1 P b 2 P0 b 3 Y b 4 t
The above demand equation is exact. However, many more factors may affect
demand. In econometrics the influence of these ‘other’ factors is taken into
account by the introduction into the economic relationships of random variable.
In our example, the demand function studied with the tools of econometrics
would be of the stochastic form:
Q b0 b1 P b2 P0 b3Y b4t u
By: TDT Page 5
Hawassa University
Department of Economics Introduction to Econometrics
Where u stands for the random factors which affect the quantity demanded.
1.5. Methodology of Econometrics
Econometric research is concerned with the measurement of the parameters of
economic relationships and with the prediction of the values of economic
variables. The relationships of economic theory which can be measured with
econometric techniques are relationships in which some variables are postulated
as causes of the variation of other variables. Starting with the postulated
theoretical relationships among economic variables, econometric research or
inquiry generally proceeds along the following lines/stages.
1. Specification the model
2. Estimation of the model
3. Evaluation of the estimates
4. Evaluation of the forecasting power of the estimated model
1. Specification of the Model
In this step the econometrician has to express the relationships between economic
variables in mathematical form. This step involves the determination of three
important tasks:
a) The dependent and independent (explanatory) variables which will be
included in the model.
b) The a priori theoretical expectations about the size and sign of the
parameters of the function.
c) The mathematical form of the model (number of equations, specific form of
the equations, etc.)
By: TDT Page 6
Hawassa University
Department of Economics Introduction to Econometrics
Note: The specification of the econometric model will be based on economic
theory and on any available information related to the phenomena under
investigation. Thus, specification of the econometric model presupposes
knowledge of economic theory and familiarity with the particular phenomenon
being studied.
Specification of the model is the most important and the most difficult stage of any
econometric research. It is often the weakest point of most econometric
applications. In this stage there exists enormous degree of likelihood of
committing errors or incorrectly specifying the model. Some of the common
reasons for incorrect specification of the econometric models are:
a. The imperfections, looseness of statements in economic theories.
b. The limitation of our knowledge of the factors which are operative in any
particular case.
c. The formidable obstacles presented by data requirements in the estimation
of large models.
The most common errors of specification are:
a. Omissions of some important variables from the function.
b. The omissions of some equations (for example, in simultaneous
equations model).
c. The mistaken mathematical form of the functions.
2. Estimation of the Model
This is purely a technical stage which requires knowledge of the various
econometric methods, their assumptions and the economic implications for the
estimates of the parameters. This stage includes the following activities.
By: TDT Page 7
Hawassa University
Department of Economics Introduction to Econometrics
a. Gathering of the data on the variables included in the model.
b. Examination of the identification conditions of the function (especially
for simultaneous equations models).
c. Examination of the aggregations problems involved in the variables of
the function.
d. Examination of the degree of correlation between the explanatory
variables (i.e. examination of the problem of multicollinearity).
e. Choice of appropriate econometric techniques for estimation, i.e. to
decide a specific econometric method to be applied in estimation; such
as, OLS, MLM, Logit, Probit… etc.
3. Evaluation of the Estimates
This stage consists of deciding whether the estimates of the parameters are
theoretically meaningful and statistically satisfactory. This stage enables the
econometrician to evaluate the results of calculations and determine the reliability
of the results. For this purpose we use various criteria which may be classified
into three groups:
a. Economic a priori criteria: These criteria are determined by economic theory
and refer to the size and sign of the parameters of economic relationships.
b. Statistical criteria (first-order tests): These are determined by statistical
theory and aim at the evaluation of the statistical reliability of the estimates
of the parameters of the model. Correlation coefficient test, standard error
test, t-test, F-test, and R2-test are some of the most commonly used
statistical tests.
By: TDT Page 8
Hawassa University
Department of Economics Introduction to Econometrics
c. Econometric criteria (second-order tests): These are set by the theory of
econometrics and aim at the investigation of whether the assumptions of
the econometric method employed are satisfied or not in any particular
case. The econometric criteria serve as a second order test (as test of the
statistical tests) i.e. they determine the reliability of the statistical criteria;
they help us establish whether the estimates have the desirable properties
of unbiasedness, consistency etc. Econometric criteria aim at the detection
of the violation or validity of the assumptions of the various econometric
techniques.
4. Evaluation of the Forecasting Power of the Model:
Forecasting is one of the aims of econometric research. However, before using an
estimated model for forecasting by some way or another the predictive power of
the model. It is possible that the model may be economically meaningful and
statistically and econometrically correct for the sample period for which the model
has been estimated; yet it may not be suitable for forecasting due to various factors
(reasons). Therefore, this stage involves the investigation of the stability of the
estimates and their sensitivity to changes in the size of the sample. Consequently,
we must establish whether the estimated function performs adequately outside
the sample of data. i.e. we must test an extra sample performance the model.
1.6. Desirable Properties of an Econometric Model
An econometric model is a model whose parameters have been estimated with
some appropriate econometric technique. The ‘goodness’ of an econometric
model is judged customarily according to the following desirable properties.
By: TDT Page 9
Hawassa University
Department of Economics Introduction to Econometrics
I. Theoretical plausibility. The model should be compatible with the postulates
of economic theory. It must describe adequately the economic phenomena
to which it relates.
II. Explanatory ability. The model should be able to explain the observations of
the actual world. It must be consistent with the observed behavior of the
economic variables whose relationship it determines.
III. Accuracy of the estimates of the parameters. The estimates of the coefficients
should be accurate in the sense that they should approximate as best as
possible the true parameters of the structural model. The estimates should
if possible possess the desirable properties of unbiasedness, consistency
and efficiency.
IV. Forecasting ability. The model should produce satisfactory predictions of
future values of the dependent (endogenous) variables.
V. Simplicity. The model should represent the economic relationships with
maximum simplicity. The fewer the equations and the simpler their
mathematical form, the better the model is considered, ceteris paribus (that
is to say provided that the other desirable properties are not affected by the
simplifications of the model).
1.7. Goals of Econometrics
Three main goals of Econometrics are identified:
a. Analysis i.e. testing economic theory
b. Policy making i.e. Obtaining numerical estimates of the coefficients of
economic relationships for policy simulations.
By: TDT Page 10
Hawassa University
Department of Economics Introduction to Econometrics
c. Forecasting i.e. using the numerical estimates of the coefficients in order to
forecast the future values of economic magnitudes.
1.8. The Sources, Types and Nature of Data
Economic data sets come in a variety of types. While some econometric methods
can be applied with little or no modification to many different kinds of data sets,
the special features of some data sets must be accounted for or should be
exploited. There are broadly three types of data that can be employed in
quantitative analysis of financial problems: We next describe the most important
data structures encountered in applied work.
1.8.1. Cross-Sectional Data
Cross-sectional data are data on one or more variables collected at a single point
in time. It consists of a sample of individuals, households, firms, cities, states,
countries, or a variety of other units, taken at a given point in time. Sometimes
the data on all units do not correspond to precisely the same time period. For
example, several families may be surveyed during different weeks within a year.
In a pure cross section analysis we would ignore any minor timing differences in
collecting the data. If a set of families was surveyed during different weeks of the
same year, we would still view this as a cross-sectional data set.
An important feature of cross-sectional data is that we can often assume that they
have been obtained by random sampling from the underlying population.
Sometimes random sampling is not appropriate as an assumption for analyzing
cross-sectional data. For example, suppose we are interested in studying factors
that influence the accumulation of family wealth. We could survey a random
sample of families, but some families might refuse to report their wealth. If, for
example, wealthier families are less likely to disclose their wealth, then the
By: TDT Page 11
Hawassa University
Department of Economics Introduction to Econometrics
resulting sample on wealth is not a random sample from the population of all
families. Another violation of random sampling occurs when we sample from
units that are large relative to the population, particularly geographical units.
The potential problem in such cases is that the population is not large enough to
reasonably assume the observations are independent draws.
Cross-sectional data are widely used in economics and other social sciences. In
economics, the analysis of cross-sectional data is closely aligned with the applied
microeconomics fields, such as labor economics, state and local public finance,
industrial organization, urban economics, demography, and health economics.
Data on individuals, households, firms, and cities at a given point in time are
important for testing microeconomic hypotheses and evaluating economic
policies.
1.8.2. Time Series Data
Time series data, as the name suggests, are data that have been collected over a
period of time on one or more variables. Time series data have associated with
them a particular frequency of observation or collection of data points. The
frequency is simply a measure of the interval over, or the regularity with which, the
data are collected or recorded. Because past events can influence future events
and lags in behavior are prevalent in the social sciences, time is an important
dimension in a time series data set. Unlike the arrangement of cross-sectional
data, the chronological ordering of observations in a time series conveys
potentially important information.
1.8.3. Panel or Longitudinal Data
A panel data (or longitudinal data) set consists of a time series for each cross-
sectional member in the data set. It has the dimensions of both time series and
cross-sections. As an example, suppose we have wage, education, and
employment history for a set of individuals followed over a ten-year period. Or
By: TDT Page 12
Hawassa University
Department of Economics Introduction to Econometrics
we might collect information, such as investment and financial data, about the
same set of firms over a five-year time period. Panel data can also be collected on
geographical units. For example, we can collect data for the same set of counties
in the Ethiopia on immigration flows, tax rates, wage rates, government
expenditures, etc., for the years 1995, 2000, and 2005.
CHAPTER 2
The Classical Regression Analysis
[The Simple Linear Regression Model]
Economic theories are mainly concerned with the relationships among various
economic variables. These relationships, when phrased in mathematical terms, can
predict the effect of one variable on another. The functional relationships of these
variables define the dependence of one variable upon the other variable (s) in the
specific form. The specific functional forms may be linear, quadratic, logarithmic,
exponential, hyperbolic, or any other form.
In this chapter we shall consider a simple linear regression model, i.e. a
relationship between two variables related in a linear form. We shall first discuss
two important forms of relations: stochastic and non-stochastic, among which we
shall be using the former in econometric analysis.
2.1. Stochastic and Non-Stochastic (Deterministic) Relationships
A relationship between X and Y, characterized as Y = f(X) is said to be
deterministic or non-stochastic if for each value of the independent variable (X)
there is one and only one corresponding value of dependent variable (Y). On the
other hand, a relationship between X and Y is said to be stochastic if for a
particular value of X there is a whole probabilistic distribution of values of Y. In
By: TDT Page 13
Hawassa University
Department of Economics Introduction to Econometrics
such a case, for any given value of X, the dependent variable Y assumes some
specific value only with some probability. Let’s illustrate the distinction between
stochastic and non-stochastic relationships with the help of a supply function.
Assuming that the supply for a certain commodity depends on its price (other
determinants taken to be constant) and the function being linear, the relationship
can be put as:
Q f ( P ) P ( 2.1)
The above relationship between P and Q is such that for a particular value of P,
there is only one corresponding value of Q. This is, therefore, a deterministic
(non-stochastic) relationship since for each price there is always only one
corresponding quantity supplied. This implies that all the variation in Y is due
solely to changes in X, and that there are no other factors affecting the dependent
variable.
If this were true all the points of price-quantity pairs, if plotted on a two-
dimensional plane, would fall on a straight line. However, if we gather
observations on the quantity actually supplied in the market at various prices and
we plot them on a diagram we see that they do not fall on a straight line.
The deviation of the observation from the line may be attributed to several factors.
By: TDT Page 14
Hawassa University
Department of Economics Introduction to Econometrics
a. Omission of variables from the function
b. Random behavior of human beings
c. Imperfect specification of the mathematical form of the model
d. Error of aggregation
e. Error of measurement
In order to take into account the above sources of errors we introduce in
econometric functions a random variable which is usually denoted by the letter ‘u’
or ‘ ’ and is called error term or random disturbance or stochastic term of the
function, so called because u is supposed to ‘disturb’ the exact linear relationship
which is assumed to exist between X and Y. By introducing this random variable
in the function the model is rendered stochastic of the form:
Yi X ui ………………………………………………………. (2.2)
Thus a stochastic model is a model in which the dependent variable is not only
determined by the explanatory variable(s) included in the model but also by
others which are not included in the model.
2.2. Simple Linear Regression Model.
The above stochastic relationship (2.2) with one explanatory variable is called
simple linear regression model. The true relationship which connects the variables
involved is split into two parts: a part represented by a line and a part represented
by the random term ‘u’.
By: TDT Page 15
Hawassa University
Department of Economics Introduction to Econometrics
The scatter of observations represents the true relationship between Y and X. The
line represents the exact part of the relationship and the deviation of the
observation from the line represents the random component of the relationship.
Were it not for the errors in the model, we would observe all the points on the line
Y1' , Y2' ,......,Yn' corresponding to X 1 , X 2 ,...., X n . However because of the random
disturbance, we observe Y1 , Y2 ,......,Yn corresponding to X 1 , X 2 ,...., X n . These points
diverge from the regression line by u1 , u 2 ,...., u n .
Yi xi ui
the dependent var iable the regression line random var iable
The first component in the bracket is the part of Y explained by the changes in X
and the second is the part of Y not explained by X, that is to say the change in Y is
due to the random influence of ui .
2.2.1. Assumptions of the Classical Linear Stochastic Regression Model.
The classicals made important assumption in their analysis of regression .The
most important of these assumptions are discussed below.
By: TDT Page 16
Hawassa University
Department of Economics Introduction to Econometrics
1. The model is linear in parameters.
The classicals assumed that the model should be linear in the parameters
regardless of whether the explanatory and the dependent variables are linear or
not. This is because if the parameters are non-linear it is difficult to estimate them
since their value is not known but you are given with the data of the dependent
and independent variable.
Example
1. Y x u is linear in both parameters and the variables, so it satisfies
the assumption.
2. ln Y ln x u is linear only in the parameters. Since the classicals worry
on the parameters, the model satisfies the assumption.
2. U i is a random real variable
This means that the value which u may assume in any one period depends on
chance; it may be positive, negative or zero. Every value has a certain probability
of being assumed by u in any particular instance.
3. The mean value of the random variable(U) in any particular period is zero
This means that for each value of x, the random variable(u) may assume various
values, some greater than zero and some smaller than zero, but if we considered
all the positive and negative values of u, for any given value of X, they would
have on average value equal to zero. In other words the positive and negative
values of u cancel each other.
By: TDT Page 17
Hawassa University
Department of Economics Introduction to Econometrics
Mathematically, E (U i ) 0 ………………………………..…. (2.3)
4. The variance of the random variable(U) is constant in each period (The
assumption of homoscedasticity)
For all values of X, the u’s will show the same dispersion around their mean. In
Fig.2.c this assumption is denoted by the fact that the values that u can assume lie
with in the same limits, irrespective of the value of X. For X 1 , u can assume any
value with in the range AB; for X 2 , u can assume any value with in the range CD
which is equal to AB and so on.
Graphically;
Mathematically;
Var (U i ) E[U i E (U i )] 2 E (U i ) 2 2 (Since E (U i ) 0 ).This constant variance is
called homoscedasticity assumption and the constant variance itself is called
homoscedastic variance.
By: TDT Page 18
Hawassa University
Department of Economics Introduction to Econometrics
5. The random variable (U) has a normal distribution
This means the values of u (for each x) have a bell shaped symmetrical
distribution about their zero mean and constant variance 2 , i.e.
U i N (0, 2 ) ………………………………………..……2.4
6. The random terms of different observations U i , U j are independent. (The
assumption of no autocorrelation)
This means the value which the random term assumed in one period does not
depend on the value which it assumed in any other period.
Algebraically,
Cov (u i u j ) [(u i (u i )][ u j (u j )]
E (u i u j ) 0 …………………………..…. (2.5)
7. The X i are a set of fixed values in the hypothetical process of repeated
sampling which underlies the linear regression model.
This means that, in taking large number of samples on Y and X, the X i values are
the same in all samples, but the ui values do differ from sample to sample, and so
of course do the values of y i .
8. The random variable (U) is independent of the explanatory variables.
This means there is no correlation between the random variable and the
explanatory variable. If two variables are unrelated their covariance is zero.
Hence Cov ( X i ,U i ) 0 ………………………………………..….(2.6)
Proof:-
cov(XU ) [( X i ( X i )][U i (U i )]
By: TDT Page 19
Hawassa University
Department of Economics Introduction to Econometrics
[( X i ( X i )(U i )] given E (U i ) 0
( X iU i ) ( X i )(U i )
( X iU i )
X i (U i ) , given that the xi are fixed
0
9. The explanatory variables are measured without error
U absorbs the influence of omitted variables and possibly errors of measurement
in the y’s. i.e., we will assume that the regressors are error free, while y values
may or may not include errors of measurement.
We can now use the above assumptions to derive the following basic concepts.
A. The dependent variable Yi is normally distributed.
i.e. Yi ~ N( x i ), 2 ……………………………… (2.7)
Proof: Yi
Mean: (Y ) xi ui
X i Since (ui ) 0
Variance: Var(Yi ) Yi (Yi ) 2
X i ui ( X i )
2
(u i ) 2
By: TDT Page 20
Hawassa University
Department of Economics Introduction to Econometrics
2 (Since (u i ) 2 2 )
var(Yi ) 2 ………………………………………. (2.8)
The shape of the distribution of Yi is determined by the shape of the distribution
of ui which is normal by assumption 4. Since and , being constant, they don’t
affect the distribution of y i . Furthermore, the values of the explanatory variable,
xi , are a set of fixed values by assumption 5 and therefore don’t affect the shape of
the distribution of y i .
Yi ~ N( x i , 2 )
B. Successive values of the dependent variable are independent, i.e.
Cov (Yi , Y j ) 0
Proof:
Cov (Yi , Y j ) E{[Yi E (Yi )][Y j E (Y j )]}
E{[ X i U i E ( X i U i )][ X j U j E ( X j U j )}
(Since Yi X i U i and Y j X j U j )
= E[( X i Ui X i )( X j U j X j )] , Since (ui ) 0
E (U iU j ) 0 (From equation (2.5))
Therefore, Cov (Yi , Y j ) 0 .
2.2.2. Methods of Estimation
Specifying the model and stating its underlying assumptions are the first stage of
any econometric application. The next step is the estimation of the numerical
By: TDT Page 21
Hawassa University
Department of Economics Introduction to Econometrics
values of the parameters of economic relationships. The parameters of the simple
linear regression model can be estimated by various methods. Three of the most
commonly used methods are:
a. Ordinary least square method (OLS)
b. Maximum likelihood method (MLM)
c. Method of moments (MM)
But, here we will deal with the OLS and the MLM methods of estimation.
2.2.2.1 The ordinary least square (OLS) method
The model Yi X i U i is called the true relationship between Y and X.
Because Y and X represent their respective population value, and and are
called the true parameters since they are estimated from the population value of Y
and X. But it is difficult to obtain the population value of Y and X because of
technical or economic reasons. So we are forced to take the sample value of Y and
X. The parameters estimated from the sample value of Y and X are called the
estimators of the true parameters and and are symbolized as ˆ and ˆ .
The model Yi ˆ ˆX i ei , is called estimated relationship between Y and X since
ˆ and ˆ are estimated from the sample of Y and X and ei represents the sample
counterpart of the population random disturbance U i .
Estimation of and by least square method (OLS) or classical least square
(CLS) involves finding values for the estimates ˆ and ˆ which will minimize the
sum of square of the squared residuals ( ei2 ).
From the estimated relationship Yi ˆ ˆX i ei , we obtain:
By: TDT Page 22
Hawassa University
Department of Economics Introduction to Econometrics
ei Yi (ˆ ˆX i ) …………………………………………………… (2.6)
e 2
i (Yi ˆ ˆX i ) 2 …………………………………………. (2.7)
To find the values of ˆ and ˆ that minimize this sum, we have to partially
differentiate e 2
i with respect to ˆ and ˆ and set the partial derivatives equal to
zero.
ei2
1. 2 (Yi ˆ ˆX i ) 0.......................................................(2.8)
ˆ
Rearranging this expression we will get: Y i nˆ ˆX i …… (2.9)
If you divide (2.9) by ‘n’ and rearrange, we get
ˆ Y ˆX ..........................................................................(2.10)
ei2
2. 2 X i (Yi ˆ ˆX ) 0..................................................(2.11)
ˆ
Note: at this point that the term in the parenthesis in equation 2.8 and 2.11 is the
residual, e Yi ˆ ˆX i . Hence it is possible to rewrite (2.8) and (2.11) as
2 ei 0 and 2 X i ei 0 . It follows that;
e i 0 and X e i i 0............................................(2.12)
If we rearrange equation (2.11) we obtain;
Y X i i ˆX i ˆX i2 ………………………………………. (2.13)
Equation (2.9) and (2.13) are called the Normal Equations. Substituting the values
of ̂ from (2.10) to (2.13), we get:
Y X i i X i (Y ˆX ) ˆX i2
By: TDT Page 23
Hawassa University
Department of Economics Introduction to Econometrics
Y X i ˆXX i ˆX i2
Y Xi i Y X i ˆ (X i2 XX i )
XY nXY = ˆ ( X i2 nX 2)
XY nXY
ˆ …………………. (2.14)
X i2 nX 2
Equation (2.14) can be rewritten in somewhat different way as follows;
( X X )(Y Y ) ( XY XY XY XY )
XY Y X XY nXY
XY nY X nXY nXY
( X X )(Y Y ) XY n X Y (2.15)
( X X ) 2 X 2 nX 2 (2.16)
Substituting (2.15) and (2.16) in (2.14), we get
( X X )(Y Y )
ˆ
( X X ) 2
Now, denoting ( X i X ) as xi , and (Yi Y ) as y i we get;
xi yi
ˆ ……………………………………… (2.17)
xi2
The expression in (2.17) to estimate the parameter coefficient is termed is the
formula in deviation form.
By: TDT Page 24
Hawassa University
Department of Economics Introduction to Econometrics
2.2.2.2 Estimation of a function with zero intercept
Suppose it is desired to fit the line Yi X i U i , subject to the restriction 0.
To estimate ˆ , the problem is put in a form of restricted minimization problem
and then Lagrange method is applied.
n
We minimize: ei2 (Yi ˆ ˆX i ) 2
i 1
Subject to: ˆ 0
The composite function then becomes
Z (Yi ˆ ˆX i ) 2 ˆ , where is a Lagrange multiplier.
We minimize the function with respect to ˆ , ˆ , and
Z
2(Yi ˆ ˆX i ) 0 (i )
ˆ
Z
2(Yi ˆ ˆX i ) ( X i ) 0 (ii )
ˆ
z
2 0 (iii )
Substituting (iii) in (ii) and rearranging we obtain:
X i (Yi ˆX i ) 0
Yi X i ˆX i 0
2
X i Yi
ˆ ……………………………………..(2.18)
X i2
This formula involves the actual values (observations) of the variables and not their
deviation forms, as in the case of unrestricted value of ˆ .
By: TDT Page 25
Hawassa University
Department of Economics Introduction to Econometrics
PROPERTIES OF OLS ESTIMATORS
The ideal or optimum properties that the OLS estimates possess may be
summarized by well known theorem known as the Gauss-Markov Theorem.
Statement of the theorem: “Given the assumptions of the classical linear regression model, the OLS
estimators, in the class of linear and unbiased estimators, have the minimum variance, i.e. the OLS
estimators are BLUE.
According to this theorem, under the basic assumptions of the classical linear
regression model, the least squares estimators are linear, unbiased and have
minimum variance (i.e. are best of all linear unbiased estimators). Some times the
theorem referred as the BLUE theorem i.e. Best, Linear, Unbiased Estimator. An
estimator is called BLUE if:
a. Linear: a linear function of the random variable, such as, the dependent
variable Y.
b. Unbiased: its average or expected value is equal to the true population
parameter.
c. Minimum variance: It has a minimum variance in the class of linear and
unbiased estimators. An unbiased estimator with the least variance is
known as an efficient estimator.
According to the Gauss-Markov theorem, the OLS estimators possess all the BLUE
properties. The detailed proof of these properties are presented below
Let’s proof these properties one by one.
a) Linearity: (for ˆ )
Proposition: ˆ & ˆ are linear in Y.
By: TDT Page 26
Hawassa University
Department of Economics Introduction to Econometrics
Proof: From (2.17) of the OLS estimator of ˆ is given by:
xi y i xi (Y Y ) xi Y Y xi
ˆ ,
xi2 xi2 xi2
(but xi ( X X ) X nX nX nX 0 )
xi Y xi
ˆ ; Now, let Ki (i 1,2,.....n)
xi2 xi2
ˆ K i Y (2.19)
̂ K1Y1 K 2Y2 K 3Y3 K nYn
ˆ is linear in Y
b) Unbiasedness:
Proposition: ˆ & ˆ are the unbiased estimators of the true parameters &
From your statistics course, you may recall that if ˆ is an estimator of then
E (ˆ) the amount of bias and if ˆ is the unbiased estimator of then bias =0 i.e.
E (ˆ) 0 E (ˆ)
In our case, ˆ & ˆ are estimators of the true parameters & .To show that they
are the unbiased estimators of their respective parameters means to prove that:
( ˆ ) and (ˆ )
Proof (1): Prove that ˆ is unbiased i.e. ( ˆ ) .
We know that ̂ kYi k i ( X i U i )
By: TDT Page 27
Hawassa University
Department of Economics Introduction to Econometrics
k i k i X i k i ui ,
but ki 0 and ki X i 1
xi ( X X ) X nX nX nX
k i 0
xi
2
xi2
xi2 xi2
k i 0 ………………………………………………………………… (2.20)
xi X i ( X X ) Xi
k i X i
xi2 xi2
X 2 XX X 2 nX 2
1
X 2 nX 2 X 2 nX 2
k i X i 1.............................……………………………………………(2.21)
ˆ k i ui ˆ k i ui (2.22)
( ˆ ) E ( ) k i E (ui ), Since ki are fixed
( ˆ ) , since (ui ) 0
Therefore, ˆ is unbiased estimator of .
Proof(2): prove that ̂ is unbiased i.e.: (ˆ )
From the proof of linearity property under 2.2.2.3 (a), we know that:
̂ 1 n Xk i Yi
1 n Xk i X i U i , Since Yi X i U i
1
n X i 1 n u i Xk i Xk i X i Xk i u i
1 n u i X k i u i , ˆ 1
n u i Xk i u i
1 n Xk i )u i ……………………(2.23)
By: TDT Page 28
Hawassa University
Department of Economics Introduction to Econometrics
(ˆ ) 1 n (u i ) Xk i (u i )
(ˆ ) (2.24)
̂ is an unbiased estimator of .
c) Minimum variance of ˆ and ˆ
Now, we have to establish that out of the class of linear and unbiased estimators
of and , ˆ and ˆ possess the smallest sampling variances. For this, we shall
first obtain variance of ˆ and ˆ and then establish that each has the minimum
variance in comparison of the variances of other linear and unbiased estimators
obtained by any other econometric methods than OLS.
a. Variance of ˆ
var( ) ( ˆ ( ˆ )) 2 ( ˆ ) 2 …………………………………… (2.25)
Substitute (2.22) in (2.25) and we get
var( ˆ ) E ( k i u i ) 2
[k12 u12 k 22 u 22 ............ k n2 u n2 2k1 k 2 u1u 2 ....... 2k n 1 k n u n 1u n ]
[k12 u12 k 22 u 22 ............ k n2 u n2 ] [2k1 k 2 u1u 2 ....... 2k n 1 k n u n 1u n ]
( k i2 ui2 ) (k i k j ui u j ) i j
k i2 (ui2 ) 2k i k j (ui u j ) 2 k i2 (Since (ui u j ) =0)
x i xi2 1
k i 2 , and therefore, k i
2
2
x i ( x i )
2 2
x i
ˆ 2
var( ) k i 2 ……………………………………………..(2.26)
2 2
x i
By: TDT Page 29
Hawassa University
Department of Economics Introduction to Econometrics
b. Variance of ̂
var(ˆ ) (ˆ ( )
2
ˆ (2.27 )
2
Substituting equation (2.23) in (2.27), we get
var(ˆ ) 1 n Xk i u i2
2
1 n Xk i (u i ) 2
2
2 ( 1 n Xk i ) 2
2 ( 1 n2 2 n Xk i X 2 k i2 )
2 ( 1 n 2 X n k i X 2 k i2 ) , Since k i 0
2 ( 1 n X 2 k i2 )
1 X2 xi2 1
2( ) , Since k 2
2
n xi ( x i ) x i
2 i 2 2
Again:
1 X 2 xi2 nX 2 X 2
n xi2 nxi2 nxi
2
2 1 X2 X i2
var( ) n 2
ˆ 2 …………………………………………(2.28)
xi nx 2
i
We have computed the variances OLS estimators. Now, it is time to check whether
these variances of OLS estimators do possess minimum variance property
compared to the variances other estimators of the true and , other than
ˆ and ˆ .
By: TDT Page 30
Hawassa University
Department of Economics Introduction to Econometrics
To establish that ˆ and ˆ possess minimum variance property, we compare their
variances with that of the variances of some other alternative linear and unbiased
estimators of and , say * and * . Now, we want to prove that any other
linear and unbiased estimator of the true population parameter obtained from any
other econometric method has larger variance that that OLS estimators.
Lets first show minimum variance of ˆ and then that of ̂ .
1. Minimum variance of ˆ
Suppose: * an alternative linear and unbiased estimator of and;
Let * w i Y i ......................................... ……………………………… (2.29)
where , wi ki ; but: wi ki ci
* wi ( X i ui ) Since Yi X i U i
wi wi X i wi ui
( *) wi wi X i ,since (ui ) 0
Since * is assumed to be an unbiased estimator, then for * is to be an unbiased
estimator of , there must be true that wi 0 and wi X 1 in the above equation.
But, wi ki ci
wi (k i ci ) ki ci
Therefore, ci 0 since ki wi 0
Again wi X i (ki ci ) X i ki X i ci X i
Since w i X i 1 and k i X i 1 c i X i 0 .
By: TDT Page 31
Hawassa University
Department of Economics Introduction to Econometrics
From these values we can drive ci xi 0, where xi X i X
ci xi ci ( X i X ) ci X i Xci
Since ci xi 1 ci 0 ci xi 0
Thus, from the above calculations we can summarize the following results.
wi 0, wi xi 1, ci 0, ci X i 0
To prove whether ˆ has minimum variance or not lets compute var( *) to
compare with var( ˆ ) .
var( *) var( wi Yi )
wi var(Yi )
2
var( *) 2 wi2 Since Var(Yi ) 2
But, wi 2 (k i ci ) 2 k i2 2k i ci ci2
ci xi
wi2 k i2 ci2 Since k i ci 0
xi2
Therefore, var( *) 2 (k i2 ci2 ) 2 k i2 2 ci2
var( *) var( ˆ ) 2 ci2
Given that ci is an arbitrary constant, 2 ci2 is a positive i.e. it is greater than zero.
Thus var( *) var( ˆ ) . This proves that ˆ possesses minimum variance property.
In the similar way we can prove that the least square estimate of the constant
intercept ( ̂ ) possesses minimum variance.
2. Minimum Variance of ̂
By: TDT Page 32
Hawassa University
Department of Economics Introduction to Econometrics
We take a new estimator * , which we assume to be a linear and unbiased
estimator of function of . The least square estimator ̂ is given by:
ˆ ( 1 n Xk i )Yi
By analogy with that the proof of the minimum variance property of ˆ , let’s use
the weights wi = ci + ki Consequently;
* ( 1 n Xwi )Yi
Since we want * to be on unbiased estimator of the true , that is, ( *) , we
substitute for Y xi ui in * and find the expected value of * .
* ( 1 n Xwi )( X i u i )
X ui
( Xwi XX i wi Xwi u i )
n n n
* X ui / n Xwi Xwi X i Xwi ui
For * to be an unbiased estimator of the true , the following must hold.
( wi ) 0, ( wi X i ) 1 and ( wi ui ) 0
i.e., if wi 0, and wi X i 1 . These conditions imply that ci 0 and ci X i 0 .
As in the case of ˆ , we need to compute Var ( * ) to compare with var( ̂ )
var( *) var ( 1 n Xwi )Yi
( 1 n Xwi ) 2 var(Yi )
2 ( 1 n Xwi ) 2
2 ( 1 n 2 X 2 wi 2 1 n Xwi )
2
By: TDT Page 33
Hawassa University
Department of Economics Introduction to Econometrics
2 ( n n 2 X 2 wi 2 X wi )
2 1
n
var( *) 2
1
n X 2 wi
2
,Since wi 0
but wi 2 k i2 ci2
var( *) 2 1
n X 2 (k i2 ci2
1 X2
var( *) 22
2 X 2 ci2
n xi
X i2
2
2 X 2 ci2
nx i
2
The first term in the bracket it var(ˆ ) , hence
var( *) var(ˆ ) 2 X 2 ci2
var( *) var(ˆ ) , Since 2 X 2 ci2 0
Therefore, we have proved that the least square estimators of linear regression
model are best, linear and unbiased (BLU) estimators.
The variance of the random variable (Ui)
You may observe that the variances of the OLS estimates involve 2 , which is the
population variance of the random disturbance term. But it is difficult to obtain
the population data of the disturbance term because of technical and economic
reasons. Hence it is difficult to compute 2 ; this implies that variances of OLS
estimates are also difficult to compute. But we can compute these variances if we
take the unbiased estimate of 2 which is ˆ 2 computed from the sample value of
the disturbance term ei from the expression:
ei2
ˆ u2 …………………………………..2.30
n2
By: TDT Page 34
Hawassa University
Department of Economics Introduction to Econometrics
To use ˆ 2 in the expressions for the variances of ˆ and ˆ , we have to prove
e
2
whether ˆ 2 is the unbiased estimator of 2 , i.e. E (ˆ 2 ) E i 2
n2
To prove this we have to compute e from the expressions of Y, Yˆ , y,
2
i
yˆ and e i .
Proof:
Yi ˆ ˆX i ei
Yˆ ˆ ˆx
Y Yˆ ei …………………………………………………………… (2.31)
ei Yi Yˆ …………………………………………………………… (2.32)
Summing (2.31) will result the following expression
Yi yi ei
Yi Yˆi sin ce (ei ) 0
Dividing both sides the above by ‘n’ will give us
Y Yˆi
Y Yˆ (2.33)
n n
Putting (2.31) and (2.33) together and subtract
Y Yˆ e
Y Yˆ
(Y Y ) (Yˆ Yˆ ) e
yi yˆ i e ……………………………………………… (2.34)
By: TDT Page 35
Hawassa University
Department of Economics Introduction to Econometrics
From (2.34):
ei yi yˆ i ……………………………………………….. (2.35)
Where the y’s are in deviation form.
Now, we have to express yi and yˆ i in other expression as derived below.
From: Yi X i U i
Y X U
We get, by subtraction
y i (Yi Y ) i ( X i X ) (U i U ) xi (U U )
y i x (U U ) ……………………………………………………. (2.36)
Note that we assumed earlier that, (u ) 0 , i.e. in taking a very large number
samples we expect U to have a mean value of zero, but in any particular single
sample U is not necessarily zero.
Similarly: From;
Yˆ ˆ ˆx
Y ˆ ˆx
We get, by subtraction
Yˆ Yˆ ˆ ( X X )
yˆ ̂ x ……………………………………………………………. (2.37)
Substituting (2.36) and (2.37) in (2.35) we get
ei xi (ui u ) ˆxi
By: TDT Page 36
Hawassa University
Department of Economics Introduction to Econometrics
(ui u ) ( ˆi ) xi
The summation over the n sample values of the squares of the residuals over the
‘n’ samples yields:
ei2 [(ui u ) ( ˆ ) xi ]2
[(ui u ) 2 (ˆ ) 2 xi 2(ui u )( ˆ ) xi ]
2
(ui u ) 2 ( ˆ ) 2 xi 2[( ˆ )xi (ui u )]
2
Taking expected values we have:
(ei2 ) [(ui u ) 2 ] [( ˆ ) 2 xi ] 2[( ˆ )xi (ui u )] …………… (2.38)
2
The right hand side terms of (2.38) may be rearranged as follows
A. [(u u ) 2 ] (u i2 u u i )
2 ( u i ) 2
u i
n
1
(u i2 ) ( u ) 2
n
n 2 1n (u1 u 2 ....... u i ) 2 Since (u i2 ) u2
n 2 1n (ui2 2ui u j )
n 2 1n ((ui2 ) 2ui u j ) i j
n 2 1n n u2 n2 (ui u j )
n u2 u2 ( given (ui u j ) 0)
By: TDT Page 37
Hawassa University
Department of Economics Introduction to Econometrics
u2 (n 1) …………………………………………….. (2.39)
B. [( ˆ ) 2 xi 2 ] xi2 .( ˆ ) 2
Given that the X’s are fixed in all samples and we know that
1
( ˆ ) 2 var( ˆ ) u2
x 2
1
Hence xi2 .(ˆ ) 2 xi2 . u2
x 2
xi2 .( ˆ ) 2 u2 …………………………………………… (2.40)
C. -2 [( ˆ )xi (ui u )] 2[( ˆ )(xi ui u xi )]
= -2 [( ˆ )(xi u i )] , sin ce xi 0
But from (2.22), ( ˆ ) k i ui and substitute it in the above expression, we will get:
-2 [( ˆ )xi (ui u ) 2(k i ui )(xi ui )]
xi u i
= -2
( x i u i ) xi
,since k i
xi x
2
i
2
(x u ) 2
2 i 2i
xi
x i 2 u i 2 2 x i x j u i u j
2
x i
2
x 2 (u i 2 ) 2( xi x j )(u i u j )
2 i j
xi xi
2 2
x 2 (u i )
2
2 ( given (u i u j ) 0)
xi
2
By: TDT Page 38
Hawassa University
Department of Economics Introduction to Econometrics
2(u i2 ) 2 2 ………………………………………. (2.41)
Consequently, Equation (2.38) can be written interms of (2.39), (2.40) and (2.41) as
follows:
ei2 n 1 u2 2 2 u2 (n 2) u2 ………………………….(2.42)
From which we get
e 2
i E (ˆ u2 ) u2 ………………………………………………..(2.43)
n2
ei2
Since ˆ 2
n2
u
ei2
Thus, ˆ 2 is unbiased estimate of the true variance of the error term ( 2 ).
n2
The conclusion that we can drive from the above proof is that we can substitute
ei2
ˆ 2 for ( 2 ) in the variance expression of ˆ and ˆ , since E (ˆ 2 ) 2 .
n2
Hence the formula of variance of ˆ and ˆ becomes;
ˆ 2 ei2
Var ( ˆ ) 2 = …………………………………… (2.44)
x i ( n 2) x i
2
2 X i ei X i …………………………… (2.45)
2 2 2
Var( )
ˆ ˆ
2
nxi n(n 2) xi
2
e can be computed as ei 2 yi ̂ xi yi .
2
Note: i
2
Do not worry about the derivation of this expression! We will perform the
derivation of it in our subsequent subtopic.
By: TDT Page 39
Hawassa University
Department of Economics Introduction to Econometrics
2.2.2.3. Statistical test of Significance of the OLS Estimators
(First Order tests)
After the estimation of the parameters and the determination of the least square
regression line, we need to know how ‘good’ is the fit of this line to the sample
observation of Y and X, that is to say we need to measure the dispersion of
observations around the regression line. This knowledge is essential because the
closer the observation to the line, the better the goodness of fit, i.e. the better is the
explanation of the variations of Y by the changes in the explanatory variables.
We divide the available criteria into three groups: the theoretical a priori criteria, the
statistical criteria, and the econometric criteria. Under this section, our focus is on
statistical criteria (first order tests). The two most commonly used first order tests
in econometric analysis are:
I. The coefficient of determination (the square of the correlation coefficient i.e. R2). This
test is used for judging the explanatory power of the independent variable(s).
II. The standard error tests of the estimators. This test is used for judging the statistical
reliability of the estimates of the regression coefficients.
1. TESTS OF THE ‘GOODNESS OF FIT’ WITH R2
R2 shows the percentage of total variation of the dependent variable that can be
explained by the changes in the explanatory variable(s) included in the model. To
elaborate this let’s draw a horizontal line corresponding to the mean value of the
dependent variable Y . (see figure ‘d’ below). By fitting the line Yˆ ˆ 0 ˆ1 X we try
to obtain the explanation of the variation of the dependent variable Y produced by
the changes of the explanatory variable X.
.Y
By: TDT Page 40
Hawassa University
Department of Economics Introduction to Econometrics
Y = e Y Yˆ
Y Y = Yˆ Yˆ ˆ 0 ˆ1 X
= Yˆ Y
Y.
Figure ‘d’. Actual and estimated values of the dependent variable Y.
As can be seen from fig.(d) above, Y Y represents measures the variation of the
sample observation value of the dependent variable around the mean. However
the variation in Y that can be attributed the influence of X, (i.e. the regression line)
is given by the vertical distance Yˆ Y . The part of the total variation in Y about Y
that can’t be attributed to X is equal to e Y Yˆ which is referred to as the residual
variation.
In summary:
ei Yi Yˆ = deviation of the observation Yi from the regression line.
y i Y Y = deviation of Y from its mean.
yˆ Yˆ Y = deviation of the regressed (predicted) value ( Yˆ ) from the mean.
Now, we may write the observed Y as the sum of the predicted value ( Yˆ ) and the
residual term (ei.).
Yi Yˆ ei
predicted Y
Observed Yi i Re sidual
By: TDT Page 41
Hawassa University
Department of Economics Introduction to Econometrics
From equation (2.34) we can have the above equation but in deviation form
y yˆ e . By squaring and summing both sides, we obtain the following
expression:
y 2 ( yˆ 2 e) 2
y 2 ( yˆ 2 e i2 2ye i )
y i 2 e i2 2yˆ e i
But ŷei = e(Yˆ Y ) e(ˆ ˆxi Y )
ˆei ˆex i Yˆei
(But ei 0, ex i 0 )
yˆe 0 ………………………………………………(2.46)
Therefore;
y i2
yˆ 2 ei2 ………………………………...(2.47)
Total Explained Un exp lained
var iation var iation var ation
OR,
Total sum of Explained sum Re sidual sum
square of square of square
TSS ESS RSS
i.e.
TSS ESS RSS ……………………………………….(2.48)
Mathematically; the explained variation as a percentage of the total variation is
explained as:
By: TDT Page 42
Hawassa University
Department of Economics Introduction to Econometrics
ESS yˆ 2
………………………………………. (2.49)
TSS y 2
From equation (2.37) we have yˆ ˆx . Squaring and summing both sides give us
yˆ 2 ˆ 2 x 2 (2.50 )
We can substitute (2.50) in (2.49) and obtain:
ˆ 2 x 2
ESS / TSS …………………………………(2.51)
y 2
xy xi x y
22
2 , Since ˆ i 2 i
x y
2
xi
xy xy
……………………………………… (2.52)
x 2 y 2
Comparing (2.52) with the formula of the correlation coefficient:
r = Cov (X, Y) / x2x2 = xy / nx2x2 = xy / ( x 2 y 2 )1/2 ………(2.53)
Squaring (2.53) will result in: r2 = ( xy )2 / ( x 2 y 2 ). …………. (2.54)
Comparing (2.52) and (2.54), we see exactly the expressions. Therefore:
xy xy
ESS/TSS = r2
x 2 y 2
From (2.48), RSS =TSS-ESS. Hence R2 becomes;
TSS RSS RSS e 2
R2 1 1 i2 ………………………….………… (2.55)
TSS TSS y
From equation (2.55) we can drive;
RSS ei2 y i2 (1 R 2 ) (2.56 )
By: TDT Page 43
Hawassa University
Department of Economics Introduction to Econometrics
The limit of R2: The value of R2 falls between zero and one. i.e. 0 R 2 1 .
Interpretation of R2
Suppose R 2 0.9 , this means that the regression line gives a good fit to the
observed data since this line explains 90% of the total variation of the Y value
around their mean. The remaining 10% of the total variation in Y is unaccounted
for by the regression line and is attributed to the factors included in the
disturbance variable u i .
2. TESTING THE SIGNIFICANCE OF OLS PARAMETERS
To test the significance of the OLS parameter estimators we need the following:
Variance of the parameter estimators
Unbiased estimator of 2
The assumption of normality of the distribution of error term.
We have already derived that:
ˆ 2
var( ˆ )
x 2
ˆ 2 X 2
var(ˆ )
nx 2
e 2 RSS
ˆ 2
n2 n2
For the purpose of estimation of the parameters the assumption of normality is not
used, but we use this assumption to test the significance of the parameter
estimators; because the testing methods or procedures are based on the
assumption of the normality assumption of the disturbance term. Hence before
By: TDT Page 44
Hawassa University
Department of Economics Introduction to Econometrics
we discuss on the various testing methods it is important to see whether the
parameters are normally distributed or not.
We have already assumed that the error term is normally distributed with mean
zero and variance 2 , i.e. U i ~ N ( 02 ),. Similarly, we also proved that
Yi ~ N[( x), 2 ] . Now, we want to show the following:
2
1. ˆ ~ N , 2
x
2 X 2
2. ˆ ~ N ,
nx 2
To show whether ˆ and ˆ are normally distributed or not, we need to make use of
one property of normal distribution. “........ any linear function of a normally
distributed variable is itself normally distributed.”
ˆ k i Yi k1Y1 k 2 Y2i .... k n Yn
ˆ wi Yi w1Y1 w2 Y2i .... wn Yn
Since ˆ and ˆ are linear in Y, it follows that
ˆ 2 2 X 2
~ N , 2 ; ~ N ,
ˆ
x n x 2
The OLS estimates ˆ and ˆ are obtained from a sample of observations on Y and
X. Since sampling errors are inevitable in all estimates, it is necessary to apply test
of significance in order to measure the size of the error and determine the degree
of confidence in order to measure the validity of these estimates. This can be done
by using various tests. The most common ones are:
I. Standard error test II) Student’s t-test III) Confidence interval
By: TDT Page 45
Hawassa University
Department of Economics Introduction to Econometrics
All of these testing procedures reach on the same conclusion. Let us now see these
testing methods one by one.
i. Standard error test
This test helps us decide whether the estimates ˆ and ˆ are significantly different
from zero, i.e. whether the sample from which they have been estimated might
have come from a population whose true parameters are zero. 0 and / or 0 .
Formally we test the null hypothesis
H 0 : i 0 against the alternative hypothesis H 1 : i 0
The standard error test may be outlined as follows.
First: Compute standard error of the parameters.
SE( ˆ ) var( ˆ )
SE(ˆ ) var(ˆ )
Second: compare the standard errors with the numerical values of ˆ and ˆ .
Decision rule:
If SE( ˆi ) 1 2 ˆi , accept the null hypothesis and reject the alternative
hypothesis. We conclude that ̂ i is statistically insignificant.
If SE( ˆ i ) 1 2 ˆ i , reject the null hypothesis and accept the alternative
hypothesis. We conclude that ̂ i is statistically significant.
The acceptance or rejection of the null hypothesis has definite economic meaning.
Namely, the acceptance of the null hypothesis 0 (the slope parameter is zero)
implies that the explanatory variable to which this estimate relates does not in fact
By: TDT Page 46
Hawassa University
Department of Economics Introduction to Econometrics
influence the dependent variable Y and should not be included in the function,
since the conducted test provided evidence that changes in X leave Y unaffected.
In other words acceptance of H0 implies that the relation ship between Y and X is
in fact Y (0) x , i.e. there is no relationship between X and Y.
Numerical example: Suppose that from a sample of size n=30, we estimate the
following supply function.
Q 120 0.6 p ei
SE : (1.7) (0.025 )
Test the significance of the slope parameter at 5% level of significance using the
standard error test.
SE ( ˆ ) 0.025
( ˆ ) 0.6
1
2 ˆ 0.3
This implies that SE( ˆi ) 1 2 ˆi . The implication is ˆ is statistically significant at 5%
level of significance.
Note: The standard error test is an approximated test (which is approximated from the z-
test and t-test) and implies a two tail test conducted at 5% level of significance.
ii. Student’s t-test
Like the standard error test, this test is also important to test the significance of the
parameters. From your statistics, any variable X can be transformed into t using
the general formula:
X
t , with n-1 degree of freedom.
sx
By: TDT Page 47
Hawassa University
Department of Economics Introduction to Econometrics
Where i value of the population mean
s x Sample estimate of the population standard deviation
( X X ) 2
sx
n 1
n Sample size
We can derive the t-value of the OLS estimates
ˆi
t ˆ
SE ( ˆ )
With n-k degree of freedom.
ˆ
tˆ
SE (ˆ )
Where:
SE = is standard error
k = number of parameters in the model.
Since we have two parameters in simple linear regression with intercept different
from zero, our degree of freedom is n-2. Like the standard error test we formally
test the hypothesis: H 0 : i 0 against the alternative H1 : i 0 for the slope
parameter; and H0 : 0 against the alternative H 1 : 0 for the intercept.
To undertake the above test we follow the following steps.
Step 1: Compute t*, which is called the computed value of t, by taking the value of
in the null hypothesis. In our case 0 , then t* becomes:
ˆ 0 ˆ
t*
SE ( ˆ ) SE ( ˆ )
By: TDT Page 48
Hawassa University
Department of Economics Introduction to Econometrics
Step 2: Choose level of significance. Level of significance is the probability of
making ‘wrong’ decision, i.e. the probability of rejecting the hypothesis when it is
actually true or the probability of committing a type I error. It is customary in
econometric research to choose the 5% or the 1% level of significance. This means
that in making our decision we allow (tolerate) five times out of a hundred to be
‘wrong’ i.e. reject the hypothesis when it is actually true.
Step 3: Check whether there is one tail test or two tail tests. If the inequality sign
in the alternative hypothesis is , then it implies a two tail test and divide the
chosen level of significance by two; decide the critical rejoin or critical value of t
called tc. But if the inequality sign is either > or < then it indicates one tail test and
there is no need to divide the chosen level of significance by two to obtain the
critical value of to from the t-table.
Example:
If we have H 0 : i 0
Against: H1 : i 0
Then this is a two tail test. If the level of significance is 5%, divide it by two to
obtain critical value of t from the t-table.
Step 4: Obtain critical value of t, called tc at
2 and n-2 degree of freedom for two
tail test.
Step 5: Compare t* (the computed value of t) and tc (critical value of t)
If t*> tc , reject H0 and accept H1. The conclusion is ˆ is statistically significant.
By: TDT Page 49
Hawassa University
Department of Economics Introduction to Econometrics
If t*< tc , accept H0 and reject H1. The conclusion is ˆ is statistically
insignificant.
Numerical Example:
Suppose that from a sample size n=20 we estimate the following consumption
function:
C 100 0.70 e
(75 .5) (0.21)
The values in the brackets are standard errors. We want to test the null
hypothesis: H 0 : i 0 against the alternative H1 : i 0 using the t-test at 5% level
of significance.
The t-value for the test statistic is:
ˆ 0 ˆ 0.70
t* = 3 .3
SE ( ˆ ) SE ( ˆ ) 0.21
Since the alternative hypothesis (H1) is stated by inequality sign ( ), it is a
two tail test, hence we divide
2 0.05 2 0.025 to obtain the critical value of ‘t’
at
2 =0.025 and 18 degree of freedom (df) i.e. (n-2=20-2). From the t-table
‘tc’ at 0.025 level of significance and 18 df is 2.10.
Since t*=3.3 and tc=2.1, t*>tc. It implies that ˆ is statistically significant.
iii. Confidence interval
Rejection of the null hypothesis doesn’t mean that our estimate ˆ and ˆ is the
correct estimate of the true population parameter and . It simply means that
our estimate comes from a sample drawn from a population whose parameter
is different from zero.
By: TDT Page 50
Hawassa University
Department of Economics Introduction to Econometrics
In order to define how close the estimate to the true parameter, we must construct
confidence interval for the true parameter, in other words we must establish
limiting values around the estimate with in which the true parameter is expected
to lie within a certain “degree of confidence”. In this respect we say that with a
given probability the population parameter will be within the defined confidence
interval (confidence limits).
We choose a probability in advance and refer to it as confidence level (interval
coefficient). It is customarily in econometrics to choose the 95% confidence level.
This means that in repeated sampling the confidence limits, computed from the
sample, would include the true population parameter in 95% of the cases. In the
other 5% of the cases the population parameter will fall outside the confidence
interval.
In a two-tail test at level of significance, the probability of obtaining the specific
t-value either –tc or tc is
2 at n-2 degree of freedom. The probability of obtaining
ˆ
any value of t which is equal to at n-2 degree of freedom is
SE ( ˆ )
1 2 2 i.e. 1 .
i.e. Pr t c t* t c 1 ………………………………………… (2.57)
ˆ
but t* …………………………………………………….(2.58)
SE ( ˆ )
Substitute (2.58) in (2.57) we obtain the following expression.
ˆ
Pr t c t c 1 ………………………………………..(2.59)
SE ( ˆ )
By: TDT Page 51
Hawassa University
Department of Economics Introduction to Econometrics
Pr SE( ˆ )t c ˆ SE( ˆ )t c 1 by multiplyin g SE( ˆ )
Pr ˆ SE( ˆ )t c ˆ SE( ˆ )t c 1 by subtractin g ˆ
Pr ˆ SE( ˆ ) ˆ SE( ˆ )t c 1 by multiplyin g by 1
Pr ˆ SE( ˆ )t c ˆ SE( ˆ )t c 1 int erchanging
The limit within which the true lies at (1 )% degree of confidence is:
[ ˆ SE( ˆ )t c , ˆ SE( ˆ )t c ] ; Where tc is the critical value of t at
2 confidence interval
and n-2 degree of freedom.
The test procedure is outlined as follows.
H0 : 0
H1 : 0
Decision rule: If the hypothesized value of in the null hypothesis is within the
confidence interval, accept H0 and reject H1. The implication is that ˆ is
statistically insignificant; while if the hypothesized value of in the null
hypothesis is outside the limit, reject H0 and accept H1. This indicates ˆ is
statistically significant.
Numerical Example:
Suppose we have estimated the following regression line from a sample of 20
observations.
Y 128 .5 2.88 X e
(38 .2) (0.85)
The values in the bracket are standard errors.
1. Construct 95% confidence interval for the slope of parameter
By: TDT Page 52
Hawassa University
Department of Economics Introduction to Econometrics
2. Test the significance of the slope parameter using constructed confidence
interval.
Solution:
1) The limit within which the true lies at 95% confidence interval is:
ˆ SE ( ˆ )t c
ˆ 2.88
SE ( ˆ ) 0.85
t c at 0.025 level of significance and 18 degree of freedom is 2.10.
ˆ SE( ˆ )t c 2.88 2.10(0.85) 2.88 1.79.
The confidence interval is:
(1.09, 4.67)
2) The value of in the null hypothesis is zero which implies it is outside the
confidence interval. Hence is statistically significant.
2.2.3 Reporting the Results of Regression Analysis
The results of the regression analysis derived are reported in conventional
formats. It is not sufficient merely to report the estimates of ’s. In practice we
report regression coefficients together with their standard errors and the value of
R2. It has become customary to present the estimated equations with standard
errors placed in parenthesis below the estimated parameter values. Sometimes,
the estimated coefficients, the corresponding standard errors, the p-values, and
some other indicators are presented in tabular form.
By: TDT Page 53
Hawassa University
Department of Economics Introduction to Econometrics
These results are supplemented by R2 on ( to the right side of the regression
equation).
Y 128 .5 2.88 X
Example: , R2 = 0.93. The numbers in the
(38 .2) (0.85)
parenthesis below the parameter estimates are the standard errors. Some
econometricians report the t-values of the estimated coefficients in place of the
standard errors.
CHAPTER 3
THE CLASSICAL REGRESSION ANALYSIS
[The Multiple Linear Regression Model]
3.1. Introduction
In simple regression we study the relationship between a dependent variable and a
single explanatory (independent variable). But it is rarely the case that economic
relationships involve just two variables. Rather a dependent variable Y can depend on a
whole series of explanatory variables or regressors. For instance, in demand studies we
study the relationship between quantity demanded of a good and price of the good, price
of substitute goods and the consumer’s income. The model we assume is:
Yi 0 1 P1 2 P2 3 X i ui --------------------------------------------------------- (3.1)
Where Yi quantity demanded, P1 is price of the good, P2 is price of substitute goods, Xi
is consumer’s income, and ' s are unknown parameters and ui is the disturbance.
By: TDT Page 54
Hawassa University
Department of Economics Introduction to Econometrics
Equation (3.1) is a multiple regression with three explanatory variables. In general for K-
explanatory variable we can write the model as follows:
Yi 0 1 X 1i 2 X 2i 3 X 3i ......... k X ki ui ----------------------------------- (3.2)
Where X k i (i 1,2,3,......., K ) are explanatory variables, Yi is the dependent variable and
j ( j 0,1,2,....(k 1)) are unknown parameters and ui is the disturbance term. The
disturbance term is of similar nature to that in simple regression, reflecting:
the basic random nature of human responses
errors of aggregation
errors of measurement
Errors in specification of the mathematical form of the model and any other
(minor) factors, other than xi that might influence Y.
In this chapter we will first start our discussion with the assumptions of the multiple
regressions and we will proceed our analysis with the case of two explanatory variables
and then we will generalize the multiple regression model in the case of k-explanatory
variables using matrix algebra.
3.2. Assumptions of Multiple Regression Model
In order to specify our multiple linear regression models and proceed our analysis with
regard to this model, some assumptions are compulsory. But these assumptions are the
same as in the single explanatory variable model developed earlier except the
assumption of no perfect multicollinearity. These assumptions are:
1. Randomness of the error term: The variable u is a real random variable.
2. Zero mean of the error term: E (ui ) 0
3. Homoscedasticity: The variance of each ui is the same for all the xi values. i.e.
E (ui ) u (constant)
2 2
4. Normality of u: The values of each ui are normally distributed. i.e. U i ~ N (0, 2 )
By: TDT Page 55
Hawassa University
Department of Economics Introduction to Econometrics
5. No auto or serial correlation: The values of ui (corresponding to Xi ) are independent
from the values of any other ui (corresponding to Xj) for i j. i.e. E (u i u j ) 0 for
xi j
6. Independence of ui and Xi : Every disturbance term ui is independent of the
explanatory variables. i.e. E (ui X 1i ) E (ui X 2i ) 0 . This condition is automatically
fulfilled if we assume that the values of the X’s are a set of fixed numbers in all
(hypothetical) samples.
7. No perfect multicollinearity: The explanatory variables are not perfectly linearly
correlated.
We can’t exclusively list all the assumptions but the above assumptions are some of the
basic assumptions that enable us to proceeds our analysis.
3.3. A Model with Two Explanatory Variables
In order to understand the nature of multiple regression model easily, we start our
analysis with the case of two explanatory variables, then extend this to the case of k-
explanatory variables.
3.3.1. Estimation of parameters of two-explanatory variables model
The model: Y 0 1 X 1 2 X 2 U i ………………………………..…………………… (3.3)
is multiple regressions with two explanatory variables. The expected value of the above
model is called population regression equation i.e.
E (Y ) 0 1 X 1 2 X 2 , Since E (U i ) 0 . …………………............................. (3.4)
Where i is the population parameter. 0 is referred to as the intercept and 1 and 2
are also sometimes known as regression slopes of the regression. Note that, 2 for
example measures the effect on E (Y ) of a unit change in X 2 when X 1 is held constant.
Since the population regression equation is unknown to any investigator, it has to be
By: TDT Page 56
Hawassa University
Department of Economics Introduction to Econometrics
estimated from sample data. Let us suppose that the sample data has been used to
estimate the population regression equation. We leave the method of estimation
unspecified for the present and merely assume that equation (3.4) has been estimated by
sample regression equation, which we write as:
Yˆ ˆ0 ˆ1 X 1 ˆ2 X 2 ……………………………………………………………. (3.5)
Where ˆ j are estimates of the j and Yˆ is known as the predicted value of Y.
Now it is time to state how (3.3) is estimated. Given sample observation on Y , X 1 & X 2 ,
we estimate (3.3) using the method of least square (OLS).
Y ˆ0 ˆ1X1i ˆ2 X 2 i ei ……………………………………………………. (3.6)
is sample relation between Y , X 1 & X 2 .
ei Yi Yˆ Yi ˆ0 ˆ1 X 1 ˆ2 X 2 ………………………………….. (3.7)
To obtain expressions for the least square estimators, we partially differentiate e 2
i with
respect to ˆ0 , ˆ1 and ˆ 2 and set the partial derivatives equal to zero.
ei2
2 Yi ˆ0 ˆ1 X 1i ˆ 2 X 2i 0 ………………………. (3.8)
ˆ0
ei2
2 X 1i Yi ˆ0 ˆ1 X 1i ˆ1 X 1i 0 ……………………. (3.9)
ˆ
1
ei2
2 X 2i Yi ˆ0 ˆ1 X 1i ˆ 2 X 2i 0 ………… ……….. (3.10)
ˆ
2
Summing from 1 to n, the multiple regression equation produces three Normal
Equations:
Y nˆ 0 ˆ1X 1i ˆ 2 X 2i ……………………………………. (3.11)
X 2i iY ˆ 0 X 1i ˆ1X 12i ˆ 2 X 1i X 1i ………………………… (3.12)
By: TDT Page 57
Hawassa University
Department of Economics Introduction to Econometrics
X 2i iY ˆ 0 X 2i ˆ1X 1i X 2i ˆ 2 X 22i ………………………... (3.13)
From (3.11) we obtain ̂ 0
ˆ0 Y ˆ1 X 1 ˆ 2 X 2 ---------------------------------------------------- (3.14)
Substituting (3.14) in (3.12), we get:
X Y (Y ˆ1 X 1 ˆ 2 X 2 )X 1i ˆ1X 1i ˆ 2 X 2i
2
1i i
X Y YˆX 1i ˆ1 (X 1i X 1X 2i ) ˆ 2 (X 1i X 2i X 2 X 2i )
2
1i i
X 1i Yi nY X 1i ˆ 2 (X 1i nX 1i ) ˆ 2 (X 1i X 2 nX 1 X 2 ) ------- (3.15)
2 2
We know that
X Yi (X i Yi nX i Yi ) xi y i
2
i
X X i (X i nX i ) xi
2 2 2 2
i
Substituting the above equations in equation (3.15), the normal equation (3.12) can be
written in deviation form as follows:
x y ˆ x ˆ 2 x1 x 2 ………………………………………… (3.16)
2
1 1 1
Using the above procedure if we substitute (3.14) in (3.13), we get
x y ˆ1x1 x 2 ˆ 2 x 2 ……………………………………….. (3.17)
2
2
Let’s bring (2.16) and (2.17) together
x y ˆ x ˆ 2 x1 x 2 ………………………………………. (3.18)
2
1 1 1
x y ˆ1x1 x 2 ˆ 2 x 2 ………………………………………. (3.19)
2
2
̂1 and ˆ 2 can easily be solved using matrix
We can rewrite the above two equations in matrix form as follows.
By: TDT Page 58
Hawassa University
Department of Economics Introduction to Econometrics
x 1
2
x x
1 2 ̂1 = x 2 y …………. (3.20)
x x x 1 2 2
2
ˆ 2 x 3y
If we use Cramer’s rule to solve the above matrix we obtain
x1 y . x2 x1 x2 . x 2 y
2
ˆ1 …………………………..…………….. (3.21)
x1 . x2 ( x1 x2 ) 2
2 2
x y . x x1 x2 . x1 y
2
ˆ 2 2 2 1 2 ………………….……………………… (3.22)
x1 . x2 ( x1 x2 ) 2
We can also express ˆ1 and ˆ 2 in terms of covariance and variances of Y , X 1 and X 2
Cov ( X 1 , Y ) . Var( X 1 ) Cov ( X 1 , X 2 ) . Cov ( X 2 , Y )
ˆ1 (3.23)
Var( X 1 ).Var( X 2 ) [cov( X 1 , X 2 )] 2
Cov ( X 2 , Y ) . Var( X 1 ) Cov ( X 1 , X 2 ) . Cov ( X 1 , Y )
ˆ 2 (3.24)
Var( X 1 ).Var( X 2 ) [Cov ( X 1 , X 2 )] 2
3.3.2. The coefficient of determination ( R2): two explanatory variables case
In the simple regression model, we introduced R2 as a measure of the proportion of
variation in the dependent variable that is explained by variation in the explanatory
variable. In multiple regression model the same measure is relevant, and the same
formulas are valid but now we talk of the proportion of variation in the dependent
variable explained by all explanatory variables included in the model. The coefficient of
determination is:
e
2
ESS RSS
R 2
1 1 i 2 ------------------------------------- (3.25)
TSS TSS yi
In the present model of two explanatory variables:
ei2 ( yi ˆ1 x1i ˆ2 x2i ) 2
By: TDT Page 59
Hawassa University
Department of Economics Introduction to Econometrics
ei ( yi ˆ1 x1i ˆ 2 x2i )
ei y ˆ1x1i ei ˆ2 ei x2i
ei yi Since ei x1i ei x2i 0
yi ( yi ˆ1 x1i ˆ2 x2i )
i.e ei2 y 2 ˆ1x1i yi ˆ2 x2i yi
ˆ1x1i y i ˆ 2 x 2i y i e i ----------------- (3.26)
2
y2
Total sum of Explained sum of Re sidual sum of squares
square (Total square ( Explained ( un exp lained var iation )
var iation ) var iation )
ESS ˆ1x1i y i ˆ 2 x 2i y i
R2 ---------------------------------- (3.27)
TSS y 2
As in simple regression, R2 is also viewed as a measure of the prediction ability of the
model over the sample period, or as a measure of how well the estimated regression fits
the data. The value of R2 is also equal to the squared sample correlation coefficient
between Yˆ & Yt . Since the sample correlation coefficient measures the linear association
between two variables, if R2 is high, that means there is a close association between the
values of Yt and the values of predicted by the model, Yˆt . In this case, the model is said to
“fit” the data well. If R2 is low, there is no association between the values of Yt and the
values predicted by the model, Yˆt and the model does not fit the data well.
3.3.3. Adjusted Coefficient of Determination ( R 2 )
One difficulty with R 2 is that it can be made large by adding more and more variables,
even if the variables added have no economic justification. Algebraically, it is the fact
that as the variables are added the sum of squared errors (RSS) goes down (it can remain
unchanged, but this is rare) and thus R 2 goes up. If the model contains n-1 variables then
R 2 =1. The manipulation of model just to obtain a high R 2 is not wise. An alternative
By: TDT Page 60
Hawassa University
Department of Economics Introduction to Econometrics
measure of goodness of fit, called the adjusted R 2 and often symbolized as R 2 , is usually
reported by regression programs. It is computed as:
ei2 / n k n 1
R 2 1 1 (1 R 2 ) -------------------------------- (3.28)
y / n 1 nk
2
This measure does not always goes up when a variable is added because of the degree of
freedom term n-k is the numerator. As the number of variables k increases, RSS goes
down, but so does n-k. The effect on R 2 depends on the amount by which R 2 falls. While
solving one problem, this corrected measure of goodness of fit unfortunately introduces
another one. It loses its interpretation; R 2 is no longer the percent of variation explained.
This modified R 2 is sometimes used and misused as a device for selecting the
appropriate set of explanatory variables.
3.4. General Linear Regression Model and Matrix Approach
So far we have discussed the regression models containing one or two explanatory
variables. Let us now generalize the model assuming that it contains k variables. It will
be of the form:
Y 0 1 X 1 2 X 2 ...... k X k U
There are k parameters to be estimated. The system of normal equations consist of k+1
equations, in which the unknowns are the parameters 0 , 1 , 2 ....... k and the known
terms will be the sums of squares and the sums of products of all variables in the
structural equations.
Least square estimators of the unknown parameters are obtained by minimizing the sum
of the squared residuals.
ei2 ( yi ˆ0 ˆ1 X 1 ˆ2 X 2 ...... ˆk X k ) 2
With respect to j ( j 0,1,2,....(k 1))
The partial derivations are equated to zero to obtain normal equations.
By: TDT Page 61
Hawassa University
Department of Economics Introduction to Econometrics
ei2
2(Yi ˆ0 ˆ1 X 1 ˆ 2 X 2 ...... ˆ k X k ) 0
ˆ
0
ei2
2(Yi ˆ0 ˆ1 X 1 ˆ 2 X 2 ...... ˆ k X k )( xi ) 0
ˆ1
……………………………………………………..
ei2
2(Yi ˆ0 ˆ1 X 1 ˆ 2 X 2 ...... ˆ k X k )( x ki ) 0
ˆ
k
The general form of the above equations (except first ) may be written as:
ei2
2(Yi ˆ0 ˆ1 X 1i ˆ k X ki ) 0 ; where ( j 1,2,....k )
ˆ
j
The normal equations of the general linear regression models are
Yi nˆ0 ˆ1X 1i ˆ2 X 2i ............................... ˆk X ki
Yi X 1i ˆ0 X 1i ˆ1X 1i ................................. ˆ k X 1i X ki
2
Yi X 2i ˆ0 X 21i ˆ1X 1i X 2i ˆ 2 X 2i .......... ˆk X 2i X ki
2
…………………………………………………..............................................
Yi X ki ˆ 0 X ki ˆ1X 1i X ki X 2i X ki .................. ˆ k X ki
2
Solving the above normal equations will result in algebraic complexity. But we can solve
this easily using matrix. Hence in the next section we will discuss the matrix approach to
linear regression model.
3.4.1. Matrix Approach to Linear Regression Model
The general linear regression model with k explanatory variables is written in the form:
Yi 0 1 X 1i 2 X 2i ............. k X ki +Ui
By: TDT Page 62
Hawassa University
Department of Economics Introduction to Econometrics
Where (i 1,2,3,........n) and 0 the intercept, 1 to k = partial slope coefficients
U=stochastic disturbance term and i=ith observation, ‘n’ being the size of the observation.
Since i represent the ith observation, we shall have ‘n’ number of equations with ‘n’
number of observations on each variable.
Y1 0 1 X 11 2 X 21 3 X 31............. k X k1 U1
Y2 0 1 X 12 2 X 22 3 X 32 ............. k X k 2 U 2
Y3 0 1 X 13 2 X 23 3 X 33 ............. k X k 3 U 3
…………………………………………………...
Yn 0 1 X 1n 2 X 2n 3 X 3n ............. k X kn U n
These equations are put in matrix form as:
Y1 1 X11 X 21 ....... X k1 0 U1
Y 1 X12 X 22 ....... Xk2 U
2
1 2
Y3 1 X13 X 23 ....... Xk3 2 U 3
. . . . ....... . . .
Yn 1 X1n X 2n ....... X kn n U n
Y X . U
In short Y X U …………………………………………………… (3.29)
The order of matrix and vectors involved are:
Y (n 1), X (n (k 1), (k 1) 1 and U (n 1)
To derive the OLS estimators of , under the usual (classical) assumptions mentioned
earlier, we define two vectors ˆ and ‘e’ as:
By: TDT Page 63
Hawassa University
Department of Economics Introduction to Econometrics
ˆ0 e1
ˆ e
1 2
ˆ . and e .
. .
ˆ en
k
Thus we can write: Y X̂ e and e Y X̂
We have to minimize:
e
i 1
2
i e12 e22 e32 ......... en2
e1
e
2
[e1 , e2 ......en ] . e' e
.
en
ei2 e' e
e' e (Y Xˆ )' (Y Xˆ )
YY ' ˆ ' X ' Y Y ' Xˆ ˆ ' X ' Xˆ ………………….… (3.30)
Since ˆ ' X ' Y ' is scalar (1x1), it is equal to its transpose;
ˆ ' X ' Y Y ' Xˆ
e' e Y ' Y 2ˆ ' X ' Y ˆ ' X ' Xˆ ------------------------------------- (3.31)
Minimizing e’e with respect to the elements in ˆ
ei2 (e' e)
2 X ' Y 2 X ' Xˆ
ˆ ˆ
By: TDT Page 64
Hawassa University
Department of Economics Introduction to Econometrics
( X ' AX )
Since 2 AX and also too 2X’A
ˆ
Equating the expression to null vector 0, we obtain:
2 X ' Y 2 X ' Xˆ 0 X ' Xˆ X ' Y
ˆ ( X ' X ) 1 X ' Y ………………………………. ………. (3.32)
Hence ˆ is the vector of required least square estimators, ˆ0 , ˆ1 , ˆ 2 ,........ˆ k .
3.4.2. Statistical Properties of the Parameters (Matrix) Approach
We have seen, in simple linear regression that the OLS estimators (ˆ & ˆ ) satisfy the
small sample property of an estimator i.e. BLUE property. In multiple regression, the
OLS estimators also satisfy the BLUE property. Now we proceed to examine the desired
properties of the estimators in matrix notations:
1. Linearity
We know that: ˆ ( X ' X ) 1 X ' Y
Let C= ( X X ) 1 X
̂ CY ……………………………………………. (3.33)
Since C is a matrix of fixed variables, equation (3.33) indicates us ˆ is linear in Y.
2. Unbiased ness
ˆ ( X ' X ) 1 X ' Y
By: TDT Page 65
Hawassa University
Department of Economics Introduction to Econometrics
ˆ ( X ' X ) 1 X ' ( X U )
ˆ ( X ' X ) 1 X 'U …….……………………………... (3.34)
Since ( X ' X ) 1 X ' X I
( ˆ ) ( X ' X ) 1 X 'U
( ) ( X ' X ) 1 X 'U
( X ' X ) 1 X ' (U )
, Since (U ) 0
Thus, least square estimators are unbiased.
3. Minimum variance
Before showing all the OLS estimators are best (possess the minimum variance property),
it is important to derive their variance.
We know that, var( ˆ ) ( ˆ ) 2 ( ˆ )( ˆ )'
( ˆ )( ˆ )'
( ˆ1 1 ) 2
( ˆ1 1 )( ˆ 2 2 ) .......
( ˆ1 1 )( ˆ k k )
ˆ ˆ
( 2 2 )( 1 1 ) ( ˆ 2 2 ) 2
....... ( ˆ 2 2 )( ˆ k k )
: : :
: : :
( ˆ k k )( ˆ1 1 ) ( ˆ k k )( ˆ 2 2 ) ........ ( ˆ k k ) 2
var( ˆ1 ) cov(ˆ1 , ˆ 2 ) ....... cov(ˆ1 , ˆ k )
ˆ ˆ
cov( 2 , 1 ) var( ˆ 2 ) ....... cov(ˆ 2 , ˆ k )
: : :
: : :
cov(ˆ k , ˆ1 ) cov(ˆ k , ˆ 2 ) ....... var( ˆ k )
By: TDT Page 66
Hawassa University
Department of Economics Introduction to Econometrics
The above matrix is a symmetric matrix containing variances along its main diagonal and
covariance of the estimators everywhere else. This matrix is, therefore, called the
Variance-covariance matrix of least squares estimators of the regression slopes. Thus,
var( ˆ ) ( ˆ )( ˆ )' ……………………………………………(3.35)
From (3.15) ˆ ( X ' X ) 1 X 'U
ˆ ( X X ) 1 X U ………………………………………………(3.36)
Substituting (3.17) in (3.16)
var( ˆ ) ( X ' X ) 1 X 'U ( X ' X ) 1 X 'U '
var( ˆ ) ( X ' X ) 1 X 'UU ' X ( X ' X ) 1
( X ' X ) 1 X ' (UU ' ) X ( X ' X ) 1
( X ' X ) 1 X ' u2 I n X ( X ' X ) 1
u2 ( X ' X ) 1 X ' X ( X ' X ) 1
var( ˆ ) u2 ( X ' X ) 1 ………………………………………….……..(3.37)
Note: ( u2 being a scalar can be moved in front or behind of a matrix while identity matrix I n can
be suppressed).
Thus we obtain, var( ˆ ) u2 ( X ' X ) 1
n X 1n ....... X kn
X X 2
....... X 1n X kn
1 1n 1n
Where, ( X ' X ) : : :
: : :
X kn X 1n X kn ....... X 2 kn
We can, therefore, obtain the variance of any estimator say ̂1 by taking the ith term from
the principal diagonal of ( X ' X ) 1 and then multiplying it by u2 .
By: TDT Page 67
Hawassa University
Department of Economics Introduction to Econometrics
Where the X’s are in their absolute form. When the x’s are in deviation form we can write
the multiple regression in matrix form as ;
̂ ( x x) 1 x y
ˆ1 x 2 1 x1 x 2 ....... x1 x k
ˆ 2
x 2 x1 x 2 ....... x 2 x k
2
Where ˆ = : and ( x x ) : : :
: : :
: 2
ˆ k x n x1 x n x 2 ....... x k
The above column matrix ˆ doesn’t include the constant term ̂ 0 .Under such conditions
the variances of slope parameters in deviation form can be written as:
var( ˆ ) u2 ( x' x) 1 …………………………………………………….(2.38)
(the proof is the same as (3.37) above). In general we can illustrate the variance of the
parameters by taking two explanatory variables.
The multiple regressions when written in deviation form that has two explanatory
variables is,
y1 ˆ1 x1 ˆ 2 x 2
var( ˆ ) ( ˆ )( ˆ )'
( ˆ1 1 )
ˆ
In this model; ( )
( ˆ 2 2 )
( ˆ )' ( ˆ1 1 )( ˆ 2 2 )
( ˆ1 1 )
( ˆ )( ˆ )'
ˆ
( ˆ1 1 )( ˆ 2 2 )
( 2 2 )
( ˆ1 1 ) 2 ( ˆ1 1 )( ˆ 2 2 )
ˆ ˆ
and ( )( )'
ˆ ˆ
( 1 1 )( 2 2 ) ( ˆ 2 2 ) 2
By: TDT Page 68
Hawassa University
Department of Economics Introduction to Econometrics
var( ˆ1 ) cov(ˆ1 , ˆ 2 )
ˆ ˆ var( ˆ 2 )
cov(1 , 2 )
In case of two explanatory variables, x in the deviation form shall be:
x11 x 21
x12 x 22 x x12 ....... x1n
x and x' 11
: : x12 x 22 ....... x 2 n
x
1n x 2 n
1
x12 x1 x 2
( x' x)
2 1
2
x1 x 2 x 22
u u
x 22 x1 x 2
u2
Or u2 ( x' x) 1 x1 x 2 x12
x12 x1 x 2
x1 x 2 x 22
u2 x 22
i.e., var( ˆ1 ) ……………………………………(3.39)
x12 x 22 (x1x 2 ) 2
u2 x12
and, var( ˆ 2 ) ………………. …….…….(3.40)
x12 x 22 (x1x 2 ) 2
() u2 x1 x 2
cov(ˆ1 , ˆ 2 ) …………………………………….(3.41)
x12 x 22 (x1x 2 ) 2
The only unknown part in variances and covariance of the estimators is u2 .
e 2
As we have seen in simple regression model ˆ 2 i . For k-parameters (including
n 2
ei2
the constant parameter) ˆ
2
.
n k
In the above model we have three parameters including the constant term and
e 2
ˆ 2 i
n 3
By: TDT Page 69
Hawassa University
Department of Economics Introduction to Econometrics
e y i 1 x1 y 2 x 2 y......... K x K y ………………………(3.42) this is for k
2 2
i
explanatory variables. For two explanatory variables
e y i 1 x1 y 2 x 2 y ………………………………………...(3.43)
2 2
i
This is all about the variance covariance of the parameters. Now it is time to see the
minimum variance property.
Minimum variance of ˆ
To show that all the i ' s in the ˆ vector are Best Estimators, we have also to prove that
the variances obtained in (3.37) are the smallest amongst all other possible linear
unbiased estimators. We follow the same procedure as followed in case of single
explanatory variable model where, we first assumed an alternative linear unbiased
estimator and then it was established that its variance is greater than the estimator of the
regression model.
ˆ
Assume that ˆ is an alternative unbiased and linear estimator of . Suppose that
ˆ ( X ' X ) 1 X ' BY
ˆ
Where B is (k x n) matrix of known constants.
ˆ
ˆ ( X ' X ) 1 X ' B X U
ˆ
ˆ ( X ' X ) 1 X ' ( X U ) B( X U )
ˆ
(ˆ ) ( X ' X ) 1 X ' ( X U ) B( X U )
( X ' X ) 1 X ' X ( X ' X ) 1 X 'U BX BU
BX , [Since E(U) = 0].……………………………….(3.44)
By: TDT Page 70
Hawassa University
Department of Economics Introduction to Econometrics
ˆ
Since our assumption regarding an alternative ˆ is that it is to be an unbiased estimator
ˆ
of , therefore, ( ˆ ) should be equal to ; in other words ( XB ) should be a null matrix.
ˆ
Thus we say, BX should be =0 if ( ˆ ) ( X ' X ) 1 X ' B Y is to be an unbiased estimator. Let
us now find variance of this alternative estimator.
ˆ ˆ ˆ
var( ˆ ) ( ˆ )( ˆ )'
( X ' X ) 1 X ' B Y ( X ' X ) 1 X ' B Y '
( X ' X ) 1 X ' B ( X U ) ( X ' X ) 1 X ' B ( X U ) '
[ ( X ' X ) 1 X ' X ( X ' X ) 1 X 'U BX BU
( X ' X ) 1
X ' X ( X ' X ) X 'U BX BU '}
1
[ ( X ' X ) 1 X 'U BU ( X ' X ) 1 X 'U BU '}
( BX 0)
( X ' X ) 1 X 'U BU U ' X ( X ' X ) 1 U ' B'
( X ' X ) 1 B UU ' X ( X ' X ) 1 U ' B'
( X ' X ) 1 X ' B (UU ' ) X ( X ' X ) 1 B'
u2 I n ( X ' X ) 1 X ' B X ( X ' X ) 1 B'
u2 ( X ' X ) 1 X ' X ( X ' X ) 1 BX ( X ' X ) 1 ( X ' X ) 1
u2 ( X ' X ) 1 X ' X ( X ' X ) 1 BX ( X ' X ) 1 ( X ' X ) 1 X ' B' BB'
u2 ( X ' X ) 1 BB' ( BX 0)
ˆ
var( ˆ ) u ( X ' X ) u BB' ……………………………………….(3.45)
2 1 2
By: TDT Page 71
Hawassa University
Department of Economics Introduction to Econometrics
ˆ
Or, in other words, var( ˆ ) is greater than var( ˆ ) by an expression u2 BB' and it proves
that ˆ is the best estimator.
3.4.3. Coefficient of Determination in Matrix Form
The coefficient of determination ( R 2 ) can be derived in matrix form as follows.
We know that ei2 e' e Y ' Y 2ˆ ' X ' Y ˆ ' X ' Xˆ since ( X ' X ) ˆ X ' Y and Y Y Y
2
i
e' e Y ' Y 2ˆ ' X ' Y ˆ ' X ' Y
e' e Y ' Y ˆ ' X ' Y ……………………………………...…….. (3.46)
ˆ ' X ' Y e' e Y ' Y ……………………………………………….(3.47)
We know, y i Yi Y
1
y i2 Yi 2 (Yi ) 2
n
1
In matrix notation, y i2 Y ' Y (Yi ) 2 …………………………………………… (3.48)
n
Equation (3.48) gives the total sum of squares variations in the model.
Explained sum of squares y i2 ei2
1
Y 'Y ( y ) 2 e ' e
n
1
ˆ ' X ' Y (Yi ) 2 ……………………….(3.49)
n
Explained sum of squares
Since R 2
Total sum of squares
By: TDT Page 72
Hawassa University
Department of Economics Introduction to Econometrics
1
ˆ ' X ' Y
( Yi )2
ˆ ' X ' Y nY 2 …………………… (3.50)
R
2 n
1
Y ' Y ( Yi ) 2 Y ' Y nY 2
n
From the discussion made so far on multiple regression model,
i. Model: Y X U
ii. Estimators: ˆ ( X ' X ) 1 X ' Y
iii. Statistical properties: BLUE
iv. Variance-covariance: var( ˆ ) 2 ( X ' X ) 1
u
v. Estimation of (e’e): e' e Y ' Y ˆ ' X ' Y
1
ˆ ' X ' Y (Yi ) 2 ̂ ' X'Y nY 2
vi. Coeff. of determination: R 2 n
1
Y ' Y (Yi ) 2 Y ' Y nY 2
n
3.5. Hypothesis Testing in Multiple Regression Model
In multiple regression models we will undertake two tests of significance. One is
significance of individual parameters of the model. This test of significance is the same as
the tests discussed in simple regression model. The second test is overall significance of
the model.
3.5.1. Tests of individual significance
If we invoke the assumption that U i ~. N (0, 2 ) , then we can use either the t-test or
standard error test to test a hypothesis about any individual partial regression coefficient.
To illustrate consider the following example.
Let Y ˆ0 ˆ1 X 1 ˆ2 X 2 ei ………………………………… (3.51)
A. H 0 : 1 0
H 1 : 1 0
By: TDT Page 73
Hawassa University
Department of Economics Introduction to Econometrics
B. H 0 : 2 0
H1 : 2 0
The null hypothesis (A) states that, holding X2 constant X1 has no (linear) influence on Y.
Similarly hypothesis (B) states that holding X1 constant, X2 has no influence on the
dependent variable Yi.To test these null hypothesis we will use the following tests:
i. Standard error test: under this and the following testing methods we test only for
̂1 .The test for ˆ 2 will be done in the same way.
ˆ 2 x 22i ei2
SE ( ˆ1 ) var( ˆ1 ) ; where ˆ
2
x x
2
1i
2
2i ( x1 x 2 ) 2 n3
If SE ( ˆ1 ) 1
2 ˆ1 , we accept the null hypothesis that is, we can conclude that the estimate
i is not statistically significant.
If SE ( ˆ1 1
2 ˆ1 , we reject the null hypothesis that is, we can conclude that the estimate i
is statistically significant.
Note: The smaller the standard errors, the stronger the evidence that the estimates are
statistically reliable.
ii. The student’s t-test: We compute the t-ratio for each ̂ i
ˆi
t* ~ t n -k , where n is number of observation and k is number of parameters.
SE ( ˆi )
If we have 3 parameters, the degree of freedom will be n-3. So;
ˆ 2 2
t* ; with n-3 degree of freedom
SE( ˆ 2 )
In our null hypothesis 2 0, the t* becomes:
By: TDT Page 74
Hawassa University
Department of Economics Introduction to Econometrics
ˆ 2
t*
SE ( ˆ 2 )
If t*<t (tabulated), we accept the null hypothesis, i.e. we can conclude that ˆ 2 is not
significant and hence the regressor does not appear to contribute to the explanation of
the variations in Y.
If t*>t (tabulated), we reject the null hypothesis and we accept the alternative one; ˆ 2 is
statistically significant. Thus, the greater the value of t* the stronger the evidence that i
is statistically significant.
3.5.2. Test of Overall Significance
Throughout the previous section we were concerned with testing the significance of the
estimated partial regression coefficients individually, i.e. under the separate hypothesis
that each of the true population partial regression coefficients was zero. In this section
we extend this idea to joint test of the relevance of all the included explanatory variables.
Now consider the following:
Y 0 1 X 1 2 X 2 ......... k X k U i
H 0 : 1 2 3 ............ k 0
H 1 : at least one of the k is non-zero
This null hypothesis is a joint hypothesis that 1 , 2 ,........ k are jointly or simultaneously
equal to zero. A test of such a hypothesis is called a test of overall significance of the
observed or estimated regression line, that is, whether Y is linearly related to
X 1 , X 2 ,........X k .
Can the joint hypothesis be tested by testing the significance of individual significance of
̂ i ’s as the above? The answer is no, and the reasoning is as follows.
By: TDT Page 75
Hawassa University
Department of Economics Introduction to Econometrics
In testing the individual significance of an observed partial regression coefficient, we assumed
implicitly that each test of significance was based on different (i.e. independent) sample. Thus, in
testing the significance of ˆ 2 under the hypothesis that 2 0 , it was assumed tacitly that the
testing was based on different sample from the one used in testing the significance of ̂ 3 under the
null hypothesis that 3 0 . But to test the joint hypothesis of the above, we shall be violating the
assumption underlying the test procedure.
“…..testing a series of single (individual) hypothesis is not equivalent to testing those
same hypothesis. The institutive reason for this is that in a joint test of several
hypotheses any single hypothesis is affected by the information in the other hypothesis.”1
The test procedure for any set of hypothesis can be based on a comparison of the sum of
squared errors from the original, the unrestricted multiple regression model to the sum
of squared errors from a regression model in which the null hypothesis is assumed to be
true. When a null hypothesis is assumed to be true, we in effect place conditions or
constraints, on the values that the parameters can take, and the sum of squared errors
increases. The idea of the test is that if these sum of squared errors are substantially
different, then the assumption that the joint null hypothesis is true has significantly
reduced the ability of the model to fit the data, and the data do not support the null
hypothesis.
If the null hypothesis is true, we expect that the data are compliable with the conditions
placed on the parameters. Thus, there would be little change in the sum of squared
errors when the null hypothesis is assumed to be true.
Let the Restricted Residual Sum of Square (RRSS) be the sum of squared errors in the
model obtained by assuming that the null hypothesis is true and URSS be the sum of the
1
Gujurati, 3rd ed.pp
By: TDT Page 76
Hawassa University
Department of Economics Introduction to Econometrics
squared error of the original unrestricted model i.e. unrestricted residual sum of square
(URSS). It is always true that RRSS - URSS 0.
Consider Yˆ ˆ0 ˆ1 X 1 ˆ2 X 2 ......... ˆk X k ei .
This model is called unrestricted. The test of joint hypothesis is that:
H 0 : 1 2 3 ............ k 0
H 1 : at least one of the k is different from zero.
We know that: Yˆ ˆ0 ˆ1 X 1i ˆ 2 X 2i ......... ˆk X ki
Yi Yˆ e
ei Yi Yˆi
ei2 (Yi Yˆi ) 2
This sum of squared error is called unrestricted residual sum of square (URSS). This is
the case when the null hypothesis is not true. If the null hypothesis is assumed to be true,
i.e. when all the slope coefficients are zero.
Y ˆ 0 ei
̂ 0
Y i
Y (applying OLS)…………………………….(3.52)
n
e Y ̂ 0 but ̂ 0 Y
e Y Y
ei2 (Yi Yˆi ) 2 y 2 TSS
The sum of squared error when the null hypothesis is assumed to be true is called
Restricted Residual Sum of Square (RRSS) and this is equal to the total sum of square
(TSS).
By: TDT Page 77
Hawassa University
Department of Economics Introduction to Econometrics
RRSS URSS / K 1
The ratio: ~ F( k 1,n k ) ……………………… (3.53); (has an F-ditribution
URSS / n K
with k-1 and n-k degrees of freedom for the numerator and denominator respectively)
RRSS TSS
URSS ei2 y 2 ˆ1yx1 ˆ 2 yx 2 ..........ˆ k yx k RSS
(TSS RSS ) / k 1
F
RSS / n k
ESS / k 1
F ………………………………………………. (3.54)
RSS / n k
If we divide the above numerator and denominator by y 2 TSS then:
ESS
/ k 1
F TSS
RSS
/k n
TSS
R2 / k 1
F …………………………………………..(3.55)
1 R2 / n k
This implies the computed value of F can be calculated either as a ratio of ESS & TSS or
R2 & 1-R2. If the null hypothesis is not true, then the difference between RRSS and URSS
(TSS & RSS) becomes large, implying that the constraints placed on the model by the null
hypothesis have large effect on the ability of the model to fit the data, and the value of F
tends to be large. Thus, we reject the null hypothesis if the F test static becomes too large.
This value is compared with the critical value of F which leaves the probability of in
the upper tail of the F-distribution with k-1 and n-k degree of freedom.
If the computed value of F is greater than the critical value of F (k-1, n-k), then the
parameters of the model are jointly significant or the dependent variable Y is linearly
related to the independent variables included in the model.
By: TDT Page 78
Hawassa University
Department of Economics Introduction to Econometrics
Application of Multiple Regressions.
In order to help you understand the working of matrix algebra in the estimation of the
regression coefficient, variance of the coefficients and testing of the parameters and the
model, consider the following numerical example.
By: TDT Page 79
Hawassa University
Department of Economics Introduction to Econometrics
Example 1. Consider the data given in Table 2.1 below to fit a linear function:
Y 1 X 1 2 X 2 3 X 3 U
Table: 2.1. Numerical example for the computation of the OLS estimators.
n Y X1 X2 X3 yi x1 x2 x3 y i2 x1 x 2 x 2 x3 x1 x3 x12 x22 x32 x1 yi x2 yi x3 y i
1 49 35 53 200 -3 -7 -9 0 9 63 0 0 49 81 0 21 27 0
2 40 35 53 212 -12 -7 -9 12 144 63 -108 -84 49 81 144 84 108 -144
3 41 38 50 211 -11 -4 -12 11 121 48 -132 -44 16 144 121 44 132 -121
4 46 40 64 212 -6 -2 2 12 36 -4 24 -24 4 4 144 12 -12 -72
5 52 40 70 203 0 -2 8 3 0 -16 24 -6 4 64 9 0 0 0
6 59 42 68 194 7 0 6 -6 49 0 -36 0 0 36 36 0 42 -42
7 53 44 59 194 1 2 -3 -6 1 -6 18 -12 4 9 36 2 -3 -06
8 61 46 73 188 9 4 11 -12 81 44 -132 -48 16 121 144 36 99 -108
9 55 50 59 196 3 8 -3 -4 9 -24 12 -32 64 9 16 24 -9 -12
1 64 50 71 190 12 8 9 -10 144 72 -90 -80 64 81 100 96 108 -120
0
520 420 620 2000
Σx1x2=240
Σx12=270
Σx22=630
Σx32=750
Σyi2=594
Σx2x3=-420
Σx1x3=-330
Σx3yi=319
Σx2yi=492
Σx3yi=-625
Σyi=0
Σx1=0
Σx2=0
Σx3=0
From the table, the means of the variables are computed and given below:
Y =52; X 1 =42; X 2 =62; X 3 =200
By: TDT Page 80
Hawassa University
Department of Economics Introduction to Econometrics
Based on the above table and model answer the following questions.
(i) Estimate the parameter estimators using the matrix approach
(ii) Compute the variance of the parameters.
(iii)Compute the coefficient of determination (R2)
(iv) Report the regression result.
Solution:
In the matrix notation: ˆ ( x' x) 1 x' y ; (when we use the data in deviation form),
ˆ1 x11 x 21 x31
Where, ˆ ˆ 2 , x x12 x 22 x32 ; so that
ˆ : : :
3 x
1n x2n x3n
x12 x1 x 2 x1 x3 x1 y
( x ' x ) x1 x 2 x 2
1 x 2 x3 and x' y x 2 y
x1 x3 x 2 x 3 x32 x3 y
(i) Substituting the relevant quantities from table 2.1 we have;
270 240 330 319
( x' x) 240 630
420 and x' y 492
330 420 750 625
Note: the calculations may be made easier by taking 30 as common factor from all the
elements of matrix (x’x). This will not affect the final results.
270 240 330
| x' x | 240 630 420 4716000
330 420 750
0.0085 0.0012 0.0031
( x' x) 1
0.0012 0.0027 0.0009
0.0031 0.0009 0.0032
By: TDT Page 81
Hawassa University
Department of Economics Introduction to Econometrics
ˆ1 0.0085 0.0012 0.0031 319 0.2063
0.0012
ˆ ˆ 2 ( x' x) 1 x' y 0.0027 0.0009 492 0.3309
ˆ 0.0031 0.0009 0.0032 625 0.5572
3
And
Y ˆ1 X 1 ˆ2 X 2 ˆ3 X 3
52 (0.2063 )( 42 ) (0.3309 )( 62 ) ( 0.5572 )( 200 )
52 8.6633 20.5139 111 .4562 134 .2789
(ii) The elements in the principal diagonal of ( x' x) 1 when multiplied u2 give the variances of the
regression parameters, i.e.,
var( ˆ1 ) u2 (0.0085 )
2 ei2 17 .11
ˆ
var( 2 ) u (0.0027 )ˆ u
2
2.851
nk 6
var( ˆ3 ) u2 (0.0032 )
var( ˆ1 ) 0.0243 , SE ( ˆ1 ) 0.1560
var( ˆ 2 ) 0.0077 , SE ( ˆ 2 ) 0.0877
var( ˆ ) 0.0093 , SE ( ˆ ) 0.0962
3 3
1
ˆ ' X ' Y (Yi ) 2
n ˆ1x1 y ˆ 2 x 2 y ˆ3 x3 y 575 .98
(iii) R2 0.97
1 y 2
594
Y ' Y (Yi ) 2 i
n
(iv) The estimated relation may be put in the following form:
Yˆ 134 .28 0.2063 X 1 0.3309 X 2 0.5572 X 3
SE( ˆi ) (0.1560 ) (0.0877 ) (0.0962 ) R 2 0.97
t* (1.3221) (3.7719 ) (5.7949 )
The variables X 1 , X 2 and X 3 explain 97 percent of total variations.
By: TDT Page 82
Hawassa University
Department of Economics Introduction to Econometrics
We can test the significance of individual parameters using the student’s t-test.
The computed value of ‘t’ is given above as t * .these values indicates us only ̂1 is
insignificant.
Example 2. The following matrix gives the variances and covariance of the of the three
variables:
y x1 x2
y 7.59 3.12 26 .99
x1 29 .16 30 .80
x 2 133 .00
The first raw and the first column of the above matrix shows y 2
and the first raw and
the second column shows yx 1i and so on.
Consider the following model
Y1 AY21 Y2 2 e vi
Where; Y1 is food consumption per capita
Y2 is food price
Y3 is disposable income per capita
And Y ln Y1 , X 1 ln Y2 and X 2 ln Y3
y Y Y , x1 X X , and x2 X X
Using the values in above matrix answer the following questions.
a. Estimate 1 and 2
ˆ ˆ
b. Compute variance of 1 and 2
c. Compute coefficient of determination
d. Report the regression result.
By: TDT Page 83
Hawassa University
Department of Economics Introduction to Econometrics
Solution:
It is difficult to estimate the above model as it is, to estimate the above model easily let’s
take the natural log of the above model;
ln Y1 ln A 1 ln Y2 2 ln Y3 Vi
And let: 0 ln A , Y ln Y1 , X 1 ln Y2 and X 2 ln Y3 the above model becomes:
Y 0 X 1 X 2 Vi
The above matrix is based on the transformed model. Using values in the matrix
we can now estimate the parameters of the original model.
We know that ˆ ( x' x) 1 x' y
In the present question:
x11 x12
ˆ1 x x 22
ˆ and , x 21
ˆ 2 : :
x1n x nn
x 2 x1 x2 x y
x' x 1 2
and x' y 1
x1 x2 x2 x2 y
Substituting the relevant quantities from the given variance-covariance matrix, we
obtain:
29 .16 30 .80 3.12
x' x and x' y
30 .80 133 .00 26 .99
29 .16 30 .80
| x' x | 2929 .64
30 .80 133 .00
1 133 .00 30 .80 0.0454 0.0105
( x' x) 1
2929 .64 30 .80 29 .16 0.0105 0.0099
By: TDT Page 84
Hawassa University
Department of Economics Introduction to Econometrics
ˆ
(a) ˆ ( x' x) 1 x' y 1
ˆ 2
0.0454 0.0105 3.12 0.1421
0.0105
0.0099 26 .99 0.2358
(b). The element in the principal diagonal of ( x' x) 1 when multiplied by u2 give the
variances of the ˆ and ˆ .
1.6680
ˆ u2 0.0981
17
var(ˆ ) (0.0981)(0.0454 ) 0.0045 , (ˆ ) SE 0.0667
var( ˆ ) (0.0981)(0.0099 ) 0.0009 , ( ˆ ) SE 0.0312
ˆx 2 y ˆx3 y
(c). R 2
y i2
(0.1421)(3.12 ) (0.2358 )( 26 .99 )
7.59
R 2 0.78; ei2 (1 R 2 )(y i2 ) 1.6680
(d).The results may be put in the following form:
Yˆ1 AY1
( 0.1421) ( 0.2358)
Y3
SE (0.0667 )(0.0312 ) R 2 0.78
t* (2.13) (7.55)
The (constant) food price elasticity is negative but income elasticity is positive. Also
income elasticity if highly significant. About 78 percent of the variations in the
consumption of food are explained by its price and income of the consumer.
By: TDT Page 85
Hawassa University
Department of Economics Introduction to Econometrics
Example 3:
Consider the model: Y 1 X 1i 2 X 2i U i
On the basis of the information given below answer the following question
X 12 3200 X 1 X 2 4300 X 2 400
X 22 7300 X 1Y 8400 X 2Y 13500
Y 800 X 1 250 n 25
Yi 28,000
2
a. Find the OLS estimate of the slope coefficient
ˆ
b. Compute variance of 2
c. Test the significant of 2 slope parameter at 5% level of significant
2 2
d. Compute R and R and interpret the result
e. Test the overall significance of the model
Solution:
a. Since the above model is a two explanatory variable model, we can estimate ˆ1 and ˆ 2 using the
formula in equation (3.21) and (3.22) i.e.
x 2 yx 22 x 2 yx1 x 2
ˆ1
x12 x 22 (x1 x 2 ) 2
x 2 yx12 x1 yx1 x 2
ˆ 2
x12 x 22 (x1 x 2 ) 2
Since the x’s and y’s in the above formula are in deviation form we have to find the
corresponding deviation forms of the above given values.
We know that:
x1 x2 X 1 X 2 nX 1 X 2
By: TDT Page 86
Hawassa University
Department of Economics Introduction to Econometrics
4300 ( 25)(10 )(16 )
300
x1 y X 1Y nX 1Y
8400 25(10 )(32 )
400
x2 y X 2Y nX 2Y
13500 25(16)(32 )
700
x12 X 12 nX 12
3200 25(10 ) 2
700
x22 X 22 nX 22
7300 25(16 ) 2
900
Now we can compute the parameters.
x 2 yx 22 x 2 yx1 x 2
ˆ1
x12 x 22 (x1 x 2 ) 2
(400 )(900 ) (700 )(300 )
(900 )(700 ) (300 ) 2
0.278
ˆ x 2 yx12 x1 yx1 x 2
2
x12 x 22 (x1 x 2 ) 2
(700 )(700 ) (400 )(300 )
(900 )(700 ) (300 ) 2
0.685
By: TDT Page 87
Hawassa University
Department of Economics Introduction to Econometrics
The intercept parameter can be computed using the following formula.
ˆ Y ˆ1 X 1 ˆ 2 X 2
32 (0.278 )(10 ) (0.685 )(16 )
18 .26
ˆ 2 x 22
b. var( ˆ1 )
x12 x 22 (x1 x 2 ) 2
ei2
̂ 2 Where k is the number of parameter
nk
In our case k=3
ei2
ˆ 2
n3
e12 y 2 ˆ1x1 y ˆ 2 x 2 y
2400 0.278 (400 ) (0.685 )(700 )
1809 .3
ei2
ˆ 2
n3
1809 .3
25 3
82 .24
(82.24)(900 )
var( ˆ1 ) 0.137
540 ,000
SE( ˆ ) var( ˆ1 ) 0.137 0.370
ˆ 2 x12
var( ˆ 2 )
x12 x12 (x1 x 2 ) 2
(82.24)(700 )
0.1067
540 ,000
By: TDT Page 88
Hawassa University
Department of Economics Introduction to Econometrics
SE( ˆ ) var( ˆ2 ) 0.1067 0.327
c. ̂1 can be tested using students t-test
This is done by comparing the computed value of t and critical value of t which is
obtained from the table at
2 level of significance and n-k degree of freedom.
t* 0.278
Hence; 0.751
SE ( ˆ1 ) 0.370
The critical value of t from the t-table at
2 0.05 2 0.025 level of significance and 22
degree of freedom is 2.074.
t c 2.074
t* 0.755
t* t c
The decision rule if t* t c is to reject the alternative hypothesis that says is different
from zero and to accept the null hypothesis that says is equal to zero. The conclusion
is ̂1 is statistically insignificant or the sample we use to estimate ̂1 is drawn from the
population of Y & X1in which there is no relationship between Y and X1(i.e. 1 0 ).
d. R 2 can be easily using the following equation
ESS RSS
R2 1-
TSS TSS
We know that RSS ei2
and TSS y 2 and ESS yˆ 2 ˆ1x1 y ˆ2 x2 y ...... ˆ k xk y
For two explanatory variable model:
RSS 10809 .3
R 2 1- 1
TSS 2400
0.24
By: TDT Page 89
Hawassa University
Department of Economics Introduction to Econometrics
24% of the total variation in Y is explained by the regression line
Yˆ 18.26 0.278 X 1 0.685 X 2 ) or by the explanatory variables (X1 and X2).
ei2 / n k (1 R 2 )( n 1)
Adjusted R 2 1 1
y 2 / n 1 nk
(1 0.24 )( 24 )
1
22
0.178
e. Let’s set first the joint hypothesis as
H 0 : 1 2 0
against H 1 : at least one of the slope parameter is different from zero.
The joint test hypothesis is testing using the F-test given below.
ESS / k 1
F *( k 1),( n k )
RSS / n k
R2 / k 1
1 R2 / n k
From (d) R 2 0.24 and k 3
F *( 2, 22) 3.4736 this is the computed value of F. Let’s compare this with the
critical value F at 5% level of significance and (3,.23) numerator and denominator
respectively. F (2,22) at 5%level of significance = 3.44.
F*(2,22) = 3.47
Fc(2,22)=3.44
F*>Fc, the decision rule is to reject H0 and accept H1. We can say that the model is
significant i.e. the dependent variable is, at least, linearly related to one of the
explanatory variables.
By: TDT Page 90
You might also like
- TRS601 University Success 9780134400785 FULL100% (2)TRS601 University Success 9780134400785 FULL368 pages
- Oxford Learner's Thesaurus: A Dictionary of SynonymsNo ratings yetOxford Learner's Thesaurus: A Dictionary of Synonyms4 pages
- 1macroeconomics Under Graduate Course Chapters 123No ratings yet1macroeconomics Under Graduate Course Chapters 123159 pages
- Introduction To Introduction To Econometrics Econometrics Econometrics Econometrics (ECON 352) (ECON 352)100% (2)Introduction To Introduction To Econometrics Econometrics Econometrics Econometrics (ECON 352) (ECON 352)12 pages
- Slides TKT CA Comparative-statics-analysis-2CreditsNo ratings yetSlides TKT CA Comparative-statics-analysis-2Credits80 pages
- Chapter-One Overview of Public Finance & TaxationNo ratings yetChapter-One Overview of Public Finance & Taxation211 pages
- Application of Econometrics in EconomicsNo ratings yetApplication of Econometrics in Economics160 pages
- Introduction To Economics: Haramaya University College of Business and Economics Department of EconomicsNo ratings yetIntroduction To Economics: Haramaya University College of Business and Economics Department of Economics79 pages
- Natural Resource Economics: An Overview: (Chapter 6)No ratings yetNatural Resource Economics: An Overview: (Chapter 6)16 pages
- Chapter Two Introduction To The Theory of Consumer Behavior100% (2)Chapter Two Introduction To The Theory of Consumer Behavior169 pages
- Industrial Economics Chapter Seven SlidesNo ratings yetIndustrial Economics Chapter Seven Slides5 pages
- Introduction To Stata and Data ManagementNo ratings yetIntroduction To Stata and Data Management30 pages
- 405 Econometrics Odar N. Gujarati: Prof. M. El-Sakka100% (1)405 Econometrics Odar N. Gujarati: Prof. M. El-Sakka27 pages
- Macroeconomic Policy in An Open Economy - 20No ratings yetMacroeconomic Policy in An Open Economy - 2023 pages
- Financial Econometrics Assignment 2 2021No ratings yetFinancial Econometrics Assignment 2 20215 pages
- Module 1 Introduction To Economics and Managerial Economics100% (1)Module 1 Introduction To Economics and Managerial Economics61 pages
- Econometrics: Introduction To The ModuleNo ratings yetEconometrics: Introduction To The Module31 pages
- AlgoBulls - Python Developer - Strategy - Coding Assignment 1No ratings yetAlgoBulls - Python Developer - Strategy - Coding Assignment 17 pages
- Super Easy DIY Outer Space Bedroom Wall Idea SteNo ratings yetSuper Easy DIY Outer Space Bedroom Wall Idea Ste2 pages
- End User Procedure (EUP) General Analyses ME80FN: PurposeNo ratings yetEnd User Procedure (EUP) General Analyses ME80FN: Purpose10 pages
- How To Craft A Trading Plan and A Trading Journal V.2100% (1)How To Craft A Trading Plan and A Trading Journal V.226 pages
- Schneider Georgakis 2013 How To NOT Make The Extended Kalman Filter FailNo ratings yetSchneider Georgakis 2013 How To NOT Make The Extended Kalman Filter Fail29 pages
- (Process Industry Practices) Design of ASME B31.3100% (1)(Process Industry Practices) Design of ASME B31.324 pages
- NIA Project Registration and PEA DocumentNo ratings yetNIA Project Registration and PEA Document6 pages
