Linear and Logistic

Regression
Artificial Neural Networks
Inputs

Continuous

Discrete

Binary
Outputs

Continuous

Discrete

Binary
Variables

Independent (Input)

Dependant (Output)

Relationship ?

https://www.analyticsvidhya.com/blog/2021/10/everything-you-need-to-know-about-linear-regression/
Linear Relationship
Residual Error

https://datatab.net/tutorial/linear-regression
Residual Sum of squares (RSS) Error

Sum of Square of Residual Errors

https://statisticsbyjim.com/regression/residual-sum-of-squares-rss/
Mean Square Error

Calculate RSS

Take Mean (It can be done using various denominators)

Denominator: No of samples (For regression) : RSS /n

Denominator: Other
Other metrics

https://datamapu.com/posts/ml_concepts/regression_metrics/
Linear Regression

Example: Simple Linear Regression

Example: Multiple Linear Regression

https://datatab.net/tutorial/linear-regression
Linear Regression : Equations
Simple Linear Regression :Example

https://statisticsbyjim.com/regression/least-squares-regression-line/
Simple Linear Regression :Example

https://statisticsbyjim.com/regression/least-squares-regression-line/
Simple Linear Regression :Example

https://statisticsbyjim.com/regression/least-squares-regression-line/
Simple Linear Regression :Example

https://statisticsbyjim.com/regression/least-squares-regression-line/
Simple Linear Regression :Example

https://statisticsbyjim.com/regression/least-squares-regression-line/
Simple Linear Regression :Example

y = 11.329 + 1.0616x

https://statisticsbyjim.com/regression/least-squares-regression-line/
Outlier

https://stackoverflow.com/questions/10231206/can-scipy-stats-identify-and-mask-obvious-outliers
Outlier

https://tillbe.github.io/outlier-influence-identification.html
Linear Regression: Neural Networks

https://www.d2l.ai/chapter_linear-regression/linear-regression.html
Gradient Descent

https://towardsdatascience.com/linear-regression-using-gradient-descent-97a6c8700931
Gradient Descent: Direction

https://www.geeksforgeeks.org/gradient-descent-in-linear-regression/
Gradient Descent: Error Equations

https://towardsdatascience.com/linear-regression-using-gradient-descent-97a6c8700931
Gradient Descent: Gradient wrt Slope

https://towardsdatascience.com/linear-regression-using-gradient-descent-97a6c8700931
Gradient Descent: Gradient wrt Intercept

https://towardsdatascience.com/linear-regression-using-gradient-descent-97a6c8700931
Gradient Descent: Learning Rate

https://www.javatpoint.com/gradient-descent-in-machine-learning
Gradient Descent: Update Slope and Intercept

https://towardsdatascience.com/linear-regression-using-gradient-descent-97a6c8700931
Gradient Descent: Minima

https://www.ibm.com/topics/gradient-descent
Gradient Descent

● Batch
● Stochastic
● Min-batch
Gradient Descent: Example

https://towardsdatascience.com/linear-regression-using-gradient-descent-97a6c8700931
Polynomial Regression

https://medium.com/@tavishi.1402/polynomial-linear-regression-in-machine-learning-in-simple-words-
b39ea30aadbc
Logistic Regression

Output : 0 or 1/ True or False/ Probability (Yes/no)

https://medium.com/analytics-vidhya/logistic-regression-as-a-neural-network-b5d2a1bd696f
Logistic Regression : Example

Churn Prediction:

https://medium.com/@omar.s.zubi/predicting-customer-churn-using-logistic-regression-with-python-6
57f66416884
Linear vs Logistic Regression

https://www.geeksforgeeks.org/ml-linear-regression-vs-logistic-regression/
Classification

Simple : Binary Classification (0, 1) (True/False)

Can be done using ONE neuron?

Depends on data boundry

https://www.geeksforgeeks.org/getting-started-with-classification/
Multi Dimensional Classification

Linear
Or
Non Linear
Complex Data

Linear
Or
Non Linear

https://scikit-learn.org/1.5/modules/neural_networks_supervised.html
Positive and Negative Class

https://waterprogramming.wordpress.com/2019/01/29/intro-to-machine-learning-part-4-support-vector-machines/
Positive vs Negative

Put INPUT values in equation, and check the output sign

Y = mx + c

Y - (mx + c) is positive, negative or neutral?

Y - (x1w1 +x2w2 +x3w3 +bias)

Is positive, negative?
Logical Gates: Neural Networks

Inputs: Binary

Outputs :Binary

https://medium.com/@stanleydukor/neural-representation-of-and-or-not-xor-and-xnor-logic-gates-per
ceptron-algorithm-b0275375fea1
Sigmoid Function

Outputs value between 0 and 1

Useful in many classification problems, and others too

Use Perceptrons

Use Gradient based learning (Backpropagation)

https://www.kaggle.com/code/ritvik1909/logic-gates-using-nn