Scribd
Upload
17 views

Regression

linear regression

Uploaded by

f20230883
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
17 views

Regression

linear regression

Uploaded by

f20230883
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

Probability and Statistics (MATH F113)

Introduction to Regression

Himadri Mukherjee

Department of Mathematics
BITS PILANI K K Birla Goa Campus, Goa

April 27, 2024


Introduction

The regression curve Recall that in a joint distribution of random


variables X, Y , the curve µY |x is called the curve of regression.
Linear regression model

Definition
There are parameters β0 , β1 , σ 2 , such that for any fixed value of
the independent variable x the dependent variable is a random
variable related to x through the model

Y = β0 + β1 x + 

The quantity  is a random variable assumed to be normally


distributed with E() = 0 and V () = σ 2 .

Note the random variable  is called the random deviation or


random error of the model.
Regression
Model parameters

Let us define a few quantities


• Sxy =
P P P
xi yi − ( xi yi )/n
P 2
• Sxx = xi − ( xi )2 /n
P

• SST ”total sum of squares”=Syy = yi2 − ( yi )2 /n


P P

• SSE ”error sum of squares”= (yi − yˆi )2


P

•
(yi − yˆi )2
P
2 2 SSE
s = σ̂ = =
n−2 n−2
• Coefficient of determination r2 = 1 − SSE SST
P 2 ˆ
• SSE= yi − β0 yi − β1 xi yi
P ˆ P

• (yi − yˆi ) are called the residual values.


The parameters

The fitted regression line is y = βˆ0 + βˆ1 x, where the estimators


are given by:
P
ˆ (xi − x)(yi − y) Sxy
β1 = P =
(xi − x) Sxx
yi − βˆ1 xi
P P
βˆ0 = = y − βˆ1 x
n
A few words on σ̂ 2 and r2 etc

SSE follows a σ 2 χ2n−2 distribution. Equivalently, SSE


σ2
follows a
SSE
χn−2 distribution. S = n−2 is an unbiased estimator of σ 2 .
2 2
Continued

The coefficient of determination r2 quantifies the amount of


observed variation of Y that can be attributed to a linear relation
between X, Y .
Continued

If the r2 value is higher, the linear model is more suited, smaller


r2 means other models are needed or there are other variables
involved.
Continued

Be careful about rounding errors, as small errors magnify


eventually because of the involvement of small numbers.
Example 1.
Continued

You might also like