Mirza Mohammad Lutfe Elahi

Department of
CSE 445 Machine Learning
Electrical and Computer Engineering
Multiple Linear Regression
 2

Multiple Linear Regression

One feature

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 3

Multiple Linear Regression

𝑥! 𝑥" 𝑥# 𝑥$ 𝑗 = 1…4
𝑛=4
𝑖=2

𝑥⃗ (") = 1416 3 2 40
(")
𝑥$ =2

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 4

Multiple Linear Regression

𝑓%,' 𝑥 = 𝑤!𝑥! + 𝑤"𝑥" + 𝑤#𝑥# + 𝑤$𝑥$ + 𝑏

example
𝑓%,' 𝑥 = 0.1𝑥! + 4𝑥" + 10𝑥# + (−2)𝑥$ + 80
size #bedrooms #floors years base price

𝑓%,' 𝑥 = 𝑤!𝑥! + 𝑤"𝑥" + ⋯ + 𝑤( 𝑥( + 𝑏

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 5

Multiple Linear Regression

𝑓%,' 𝑥 = 𝑤!𝑥! + 𝑤"𝑥" + ⋯ + 𝑤( 𝑥( + 𝑏
𝑤 = 𝑤! 𝑤" 𝑤# … 𝑤(
𝑏 is a number parameters of the model

vector 𝑥⃗ = 𝑥! 𝑥" 𝑥# … 𝑥(
﹒
𝑤 = 𝑤! 𝑤" 𝑤# … 𝑤(
𝑓%,' 𝑥⃗ = 𝑤 4 𝑥⃗ + 𝑏 = 𝑤!𝑥! + 𝑤"𝑥" + ⋯ + 𝑤( 𝑥( + 𝑏
dot product
multiple linear regression
not multivariate regression

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 6

Multiple Linear Regression - Vectorization

𝑛=3

𝑤0 𝑤1 𝑤2

𝑥0 𝑥1 𝑥2

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 7

Multiple Linear Regression - Vectorization

in parallel

efficient ➞ scale to large datasets

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 8

Multiple Linear Regression - Vectorization

𝑏 parameters
derivatives

learning rate 𝛼

0.1 ∗ 𝑑

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 9

Multiple Linear Regression - Vectorization

vector of length 𝑛

𝑏 still a number

dot product

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 10

Multiple Linear Regression - Vectorization

𝑗=1

𝑗=𝑛

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 11

Multiple Linear Regression – Feature Scaling

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 12

Multiple Linear Regression – Feature Scaling

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 13

Multiple Linear Regression – Feature Scaling

size

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 14

Multiple Linear Regression – Feature Scaling

size #bedrooms

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 15

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 16

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 17

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

one training example

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 18

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

one training example

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 19

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

one training example

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 20

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

one training example

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 21

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

one training example

100,000K 0.5K 50K

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 22

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

one training example

100,000K 0.5K 50K

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 23

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

one training example

100,000K 0.5K 50K

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 24

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

one training example

small large

100,000K 0.5K 50K

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 25

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

one training example

small large

100,000K 0.5K 50K

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 26

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

one training example

small large

100,000K 0.5K 50K 200K 250K 50K

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 27

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

one training example

small large

100,000K 0.5K 50K 200K 250K 50K

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 28

Multiple Linear Regression – Feature Scaling

size #bedrooms large small

one training example

small large

100,000K 0.5K 50K 200K 250K 50K

more reasonable

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 29

Multiple Linear Regression – Feature Scaling

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 30

Multiple Linear Regression – Feature Scaling

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 31

Multiple Linear Regression – Feature Scaling

5
0
300 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 32

Multiple Linear Regression – Feature Scaling

5
0
300 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 33

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 34

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 35

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 36

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 37

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 38

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 39

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 40

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 41

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 42

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 43

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 44

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 45

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 46

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

0
0 1

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 47

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

0
0 1

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 48

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

0
0 1

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 49

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

0
0 1

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 50

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

0
0 1

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 51

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 52

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 53

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 54

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000
max

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 55

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000
max

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 56

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000
max

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 57

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000
max

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 58

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000
max max

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 59

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000
max max

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 60

Multiple Linear Regression – Feature Scaling

100

5
10
0 0
300 2000 1 2000
max max

0
0 1

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 61

Multiple Linear Regression – Mean Normalization

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 62

Multiple Linear Regression – Mean Normalization

100

5
10
0 0
300 2000 1 2000

-1 1

-1
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 63

Multiple Linear Regression – Mean Normalization

100
average

5
10
0 0
300 2000 1 2000

-1 1

-1
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 64

Multiple Linear Regression – Mean Normalization

100
average
𝜇! = 800

5
10
0 0
300 2000 1 2000

-1 1

-1
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 65

Multiple Linear Regression – Mean Normalization

100
average
𝜇! = 800

5
10
0 0
300 2000 1 2000

-1 1

-1
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 66

Multiple Linear Regression – Mean Normalization

100
average
𝜇! = 800

5
10
0 0
300 2000 1 2000
max - min

-1 1

-1
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 67

Multiple Linear Regression – Mean Normalization

100
average
𝜇! = 800

5
10
0 0
300 2000 1 2000
max - min

-1 1

-1
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 68

Multiple Linear Regression – Mean Normalization

100
average
𝜇! = 800

5
10
0 0
300 2000 1 2000
max - min

-1 1

-1
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 69

Multiple Linear Regression – Mean Normalization

100
𝜇! = 2.3 average
𝜇! = 800

5
10
0 0
300 2000 1 2000
max - min

-1 1

-1
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 70

Multiple Linear Regression – Mean Normalization

100
𝜇" = 2.3 average
𝜇! = 800

5
10
0 0
300 2000 1 2000
max - min

-1 1

-1
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 71

Multiple Linear Regression – Mean Normalization

100
𝜇! = 2.3 average
𝜇! = 800

5
10
0 0
300 2000 1 2000
max - min max - min

-1 1

-1
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 72

Multiple Linear Regression – Mean Normalization

100
𝜇! = 2.3 average
𝜇! = 800

5
10
0 0
300 2000 1 2000
max - min max - min

-1 1

-1
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 73

Multiple Linear Regression – Z-score Normalization

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 74

Multiple Linear Regression – Z-score Normalization

standard deviation 𝜎

100

5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 75

Multiple Linear Regression – Z-score Normalization

standard deviation 𝜎

100

𝑥!
5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 76

Multiple Linear Regression – Z-score Normalization

standard deviation 𝜎

100
𝜎!
𝑥!
𝜇!
5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 77

Multiple Linear Regression – Z-score Normalization

standard deviation 𝜎

100
𝜎! = 450
𝜎!
𝑥!
𝜇! = 800 𝜇!
5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 78

Multiple Linear Regression – Z-score Normalization

standard deviation 𝜎

100
𝜎! = 450
𝜎!
𝑥!
𝜇! = 800 𝜇!
5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 79

Multiple Linear Regression – Z-score Normalization

standard deviation 𝜎

100
𝜎! = 450
𝜎!
𝑥!
𝜇! = 800 𝜇!
5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 80

Multiple Linear Regression – Z-score Normalization

standard deviation 𝜎

100
𝜇" = 2.3 𝜎! = 450
𝜎!
𝜎" = 1.4
𝑥!
𝜇! = 800 𝜇!
5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 81

Multiple Linear Regression – Z-score Normalization

standard deviation 𝜎

100
𝜇" = 2.3 𝜎! = 450
𝜎!
𝜎" = 1.4
𝑥!
𝜇! = 800 𝜇!
5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 82

Multiple Linear Regression – Z-score Normalization

standard deviation 𝜎

100
𝜇" = 2.3 𝜎! = 450
𝜎!
𝜎" = 1.4
𝑥!
𝜇! = 800 𝜇!
5
10
0 0
300 2000 1 2000

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 83

Multiple Linear Regression – Z-score Normalization

standard deviation 𝜎

100
𝜇" = 2.3 𝜎! = 450
𝜎!
𝜎" = 1.4
𝑥!
𝜇! = 800 𝜇!
5
10
0 0
300 2000 1 2000

-3 3

-3

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 84

Multiple Linear Regression – Z-score Normalization

standard deviation 𝜎

100
𝜇" = 2.3 𝜎! = 450
𝜎!
𝜎" = 1.4
𝑥!
𝜇! = 800 𝜇!
5
10
0 0
300 2000 1 2000

-3 3

-3

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 85

Multiple Linear Regression – Feature Scaling

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 86

Multiple Linear Regression – Feature Scaling

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 87

Multiple Linear Regression – Feature Scaling

acceptable ranges

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 88

Multiple Linear Regression – Feature Scaling

acceptable ranges

okay, no rescaling

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 89

Multiple Linear Regression – Feature Scaling

acceptable ranges

okay, no rescaling
okay, no rescaling

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 90

Multiple Linear Regression – Feature Scaling

acceptable ranges

okay, no rescaling
okay, no rescaling

too large ➞ rescale

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 91

Multiple Linear Regression – Feature Scaling

acceptable ranges

okay, no rescaling
okay, no rescaling

too large ➞ rescale

too small ➞ rescale

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 92

Multiple Linear Regression – Feature Scaling

acceptable ranges

okay, no rescaling
okay, no rescaling

too large ➞ rescale

too small ➞ rescale

too large ➞ rescale

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 93

Multiple Linear Regression – Gradient Descent

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 94

Multiple Linear Regression – Gradient Descent

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 95

Multiple Linear Regression – Gradient Descent

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 96

Multiple Linear Regression – Gradient Descent

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 97

Multiple Linear Regression – Gradient Descent

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 98

Multiple Linear Regression – Gradient Descent

𝑤, 𝑏

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 99

Multiple Linear Regression – Gradient Descent

learning curve

𝑤, 𝑏

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 100

Multiple Linear Regression – Gradient Descent

learning curve

𝑤, 𝑏

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 101

Multiple Linear Regression – Gradient Descent

learning curve

𝑤, 𝑏

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 102

Multiple Linear Regression – Gradient Descent

learning curve

𝑤, 𝑏

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 103

Multiple Linear Regression – Gradient Descent

learning curve

𝑤, 𝑏

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 104

Multiple Linear Regression – Gradient Descent

learning curve

𝑤, 𝑏

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 105

Multiple Linear Regression – Gradient Descent

learning curve

𝑤, 𝑏

30 1,000 100,000
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 106

Multiple Linear Regression – Gradient Descent

learning curve

𝑤, 𝑏

30 1,000 100,000
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 107

Multiple Linear Regression – Gradient Descent

learning curve

𝑤, 𝑏

30 1,000 100,000
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 108

Multiple Linear Regression – Gradient Descent

learning curve

𝑤, 𝑏

30 1,000 100,000
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 109

Multiple Linear Regression – Gradient Descent

learning curve

𝑤, 𝑏

30 1,000 100,000
CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU
 110

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 111

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 112

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 113

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 114

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 115

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 116

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 117

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 118

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 119

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 120

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 121

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 122

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 123

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 124

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 125

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 126

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 127

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 128

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 129

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 130

Multiple Linear Regression – Learning Rate

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 131

Multiple Linear Regression – Learning Rate

𝛼 = 0.001

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 132

Multiple Linear Regression – Learning Rate

𝛼 = 0.001
𝛼 = 0.01

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 133

Multiple Linear Regression – Learning Rate

0.003
3X

𝛼 = 0.001
𝛼 = 0.01

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 134

Multiple Linear Regression – Learning Rate

0.003
3X ≈ 3X

𝛼 = 0.001
𝛼 = 0.01

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 135

Multiple Linear Regression – Learning Rate

0.003 0.03
3X ≈ 3X 3X

𝛼 = 0.001
𝛼 = 0.01

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 136

Multiple Linear Regression – Learning Rate

0.003 0.03
3X ≈ 3X 3X ≈ 3X

𝛼 = 0.001
𝛼 = 0.01

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 137

Multiple Linear Regression – Learning Rate

0.003 0.03 0.3

3X ≈ 3X 3X ≈ 3X 3X

𝛼 = 0.001
𝛼 = 0.01

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 138

Multiple Linear Regression – Learning Rate

0.003 0.03 0.3

3X ≈ 3X 3X ≈ 3X 3X ≈ 3X

𝛼 = 0.001
𝛼 = 0.01

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 139

Multiple Linear Regression – Learning Rate

0.003 0.03 0.3

3X ≈ 3X 3X ≈ 3X 3X ≈ 3X

𝛼 = 0.001
𝛼 = 0.01

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 140

Multiple Linear Regression – Learning Rate

0.003 0.03 0.3

3X ≈ 3X 3X ≈ 3X 3X ≈ 3X

𝛼 = 0.001 𝛼 too small

𝛼 = 0.01

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 141

Multiple Linear Regression – Learning Rate

0.003 0.03 0.3

3X ≈ 3X 3X ≈ 3X 3X ≈ 3X

𝛼 = 0.001 𝛼 too small

𝛼 = 0.01 𝛼 too big

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 142

Multiple Linear Regression – Learning Rate

0.003 0.03 0.3

3X ≈ 3X 3X ≈ 3X 3X ≈ 3X

𝛼 = 0.001 𝛼 too small

𝛼 = 0.01 𝛼 too big
𝛼 just right

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 143

Multiple Linear Regression – Feature Engineering

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 144

Multiple Linear Regression – Feature Engineering

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 145

Multiple Linear Regression – Feature Engineering

frontage depth

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 146

Multiple Linear Regression – Feature Engineering

frontage depth

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 147

Multiple Linear Regression – Feature Engineering

frontage depth

new feature

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 148

Multiple Linear Regression – Feature Engineering

frontage depth

new feature

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 149

Multiple Linear Regression – Feature Engineering

frontage depth

new feature

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 150

Multiple Linear Regression – Feature Engineering

frontage depth

new feature

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 151

Multiple Linear Regression – Polynomial Regression

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 152

Multiple Linear Regression – Polynomial Regression

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 153

Multiple Linear Regression – Polynomial Regression

𝑠𝑖𝑧𝑒 𝑠𝑖𝑧𝑒 !

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 154

Multiple Linear Regression – Polynomial Regression

𝑠𝑖𝑧𝑒 𝑠𝑖𝑧𝑒 !

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 155

Multiple Linear Regression – Polynomial Regression

𝑠𝑖𝑧𝑒 𝑠𝑖𝑧𝑒 !

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 156

Multiple Linear Regression – Polynomial Regression

𝑠𝑖𝑧𝑒 𝑠𝑖𝑧𝑒 ! 𝑠𝑖𝑧𝑒 "

𝑠𝑖𝑧𝑒 𝑠𝑖𝑧𝑒 !

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 157

Multiple Linear Regression – Polynomial Regression

𝑠𝑖𝑧𝑒 𝑠𝑖𝑧𝑒 ! 𝑠𝑖𝑧𝑒 "

𝑠𝑖𝑧𝑒 𝑠𝑖𝑧𝑒 !

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 158

Multiple Linear Regression – Polynomial Regression

𝑠𝑖𝑧𝑒 𝑠𝑖𝑧𝑒 ! 𝑠𝑖𝑧𝑒 "

feature scaling

𝑠𝑖𝑧𝑒 𝑠𝑖𝑧𝑒 !

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 159

Multiple Linear Regression – Polynomial Regression

𝑠𝑖𝑧𝑒 𝑠𝑖𝑧𝑒 ! 𝑠𝑖𝑧𝑒 "

1 − 10" 1 − 10# 1 − 10$

feature scaling

𝑠𝑖𝑧𝑒 𝑠𝑖𝑧𝑒 !

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 160

Multiple Linear Regression – Polynomial Regression

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 161

Multiple Linear Regression – Polynomial Regression

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 162

Multiple Linear Regression – Polynomial Regression

𝑠𝑖𝑧𝑒 𝑠𝑖𝑧𝑒

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 163

Multiple Linear Regression – Polynomial Regression

𝑠𝑖𝑧𝑒 𝑠𝑖𝑧𝑒

what features to use?

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU

 164

Reference
Ng, A. (2024). Supervised Machine Learning: Regression and Classification
[Coursera course]. Stanford University.
Retrieved from https://www.coursera.org/learn/machine-learning

CSE445 Machine Learning Supervised Learning – Linear Regression ECE@NSU