Regularization ‘Tee output mode Preiion Example Use Case rage regression Applies 2 regularization, 295 onaty term proportions = Reduces the magnitude of inthe model General faster a i doesnt involve all contributing to the outcome (9 Predicting house pics where al features te ize, lection, etc matter) Eppa ec Y= apex As text_ex ey Hi Ceok dD ~(éx5% May be slower duet the feature selection process Use when you bie only some predictors are ty importante ‘genetic stues where ony afew genes out of thousands areee eM cs cquared enor modiiel include a regularization term, of the penalty, and ware the coefficients What is Lasso Regression or (L1 Regularization) Method? Lasso regression, also known as L1 regularization isa linear regression techniaue that adds @ penalty to the Loss function to prevent overfitting This penalty is based on the absolute values of the cocfficents Lasso regression isa version of linear regression including a penalty lagnitude. By encouraging equal to the absolute value of the coefficient m: erfitting and helps some sparsity, this L1 regularization term reduces ov Coefficients to be absolutely zero, hence facilitating feature selection, ‘The standard loss function (mean squared error) is modified to include a E regularization term: ase +a = Wal Loss = MSE+2i-Inbeit eros Here, Ais the regularization parameter that controls the strength of the penalty, and wi are the coefficients, soe sours d= three bes Plonth ce 5 S — sclecti heer ee i t Hotln to dal oe ile er { — Fede Blogilile ) = Tiny Pentre. hark lerte wt Rey ¢ ia ebtipdtr- rset clei l"No multcotineariy TWsindeate ha Feature Selection: ln mut itt covtlly sete Overtting: Overtting occurs wnen te mod at ton closely fete ann or random fucusions that not presant he ue unde Gir uatneetnnaon Fectr iio esi aus now much be atc na oa pre eesson io nown as 2epusiston. este saubed magus fe Ridge reoression, also known as L2 regularization, sa technique rar tinear regression to prevent overfting by adding a penaty Ta to the Loss function. This penalty = Proportional to term quare of te magritude of te coefficients (weights wel egression[s version of net Teareision that includes Papa a prevent the model rm ovrtna especialy when fare are many predictors or not enoush data,"pes of Lin When there inn Rearession Regression 2 nY Ne independent feature it is Known 25 1 OF Un) ren there are more than none fe: ‘simple Linear "Ke Linear Regression and wh ature itis known a. Regression or Multiya, Multiple Linear 1. Simple Linear Regres involves the simplest form of linear regression and it only one independent variable and one dependent variable. nether his imps that hangs nie Sperone lesa eobenten ‘ariabe(} ino near fashion This moans that thee shouldbe a stig io tet ean De ow regression vil not bean accurate model follow abelt-shaped cure. the residuals ae not normally aistbuted, then near regression ‘Multiple Linear Rogression Muitipt inca rgrossion ivotwe one independent variable ond one dependent more tha arable. The equation for multiple tinear regressions ses TKT sB2K2". . BNkny=BDYBIXTYBZIZY PDX Viethe dependant variable 1,22, ., Xn the indopendent variables Bois theinercopt 1,2, nar the stopes the gotof to algortm toting tho best Ft Line equation ht con rade the values based on the Independent variables. amptions of Multiple Linear Rogres a our othe For ultiple Linear Regression, ol Imoddtion to this, bolow ar fowmore sumptions om Sipe Linear Regression apyCnet 8 @ ensemble method that ids mukiple decision trees and each ees the pect lfrent subset ofthe traning data The foe prditon a made By averaging the relitons ofall othe tess, For example cisions sero ses daa using tis Regression Evaluation Metrics Fraluation in machine learning measures the perfomance of model Here are some popula evaluation metres for regression * Mean Absolute Error (MAE: The average absolute diference between the Predicted and actual values ofthe tnt eae ‘Mean Squ H (MISE: The average squared difference between the predicted and actual values ofthe target variable, Root Mean Squared Error (RMSE): Square root ofthe mean squared error, ber Loss hybrid oss function that transitions from MAE to MSE for larger {rTos, providing balance between robustness and MSE's sensitivity to outliers, R2_Score: Higher values indicate beer ft ranging fom 0101 Why Linear Regression is Important? The interpretability of linear regression is one ofits greatest strengths. The model's equation offers clear coefficients that illustrate the influence of each independent variable on the dependent variable, enhancing our understanding of the underlying relationships ts simplicity is a significant advantage; linear regression is transparent, easy to implement, and serves asa foundational concept for more advanced algorithms, What is the best Fit Line? (Our primary objective while using linear regression iso locate the best-fit ine, which implies that the error between the predicted and actual values should be kept to @ minimum, Tere will be the leat error in the best-fit line ‘Assumptions are: + Lies assumes tal theres in ans erent ae ee ariestissscaleetaes ince nereae, vaso een proportional sangsin the dependent var Independence: The observation shld bs indepen cach oer hist mors from one observation should n0t influence otherRepression in machine learning F110 4 spy 10 predict a continous numerical Value hie ghee nea wher ee relationships between variables s0 that predicuy ne ent Toate ae Mone can be made, we have wo types of + Dependent Variable (large) The varigh dict og Weare trying to predict e.g houve price + Independent Variables (Heatures): tie 6 locality, numberof oom lables tha Influence the predeton Regression analysis problem works with if output variable su real or eontinuows value as “salary” or “weight, Many diferent repression nists can be used but the simplest m in them is linear regression, Types of Regres Regression can be classified into different types based on the numberof predictor variable and the nature of the relationship between variable 1. Simple Linear Regression Linear regression is one of the simplest and most widely used statistical models, This assumes that there is a linear relationship between the independent and dependent variable This means that the change in the dependent variable is proportional tothe change in the independent variables. For example predicting the price of « house based on its size 2. Multiple Linear Regressio Multiple linear regression extends simple linear repression by using multiple independent variables to predict target variable, For example predicting the price of « house based on multiple features such as size, location, number of rooms, ete 3. Polynomial Regress Polynomial reyression is used to model with non-linear relationships between the dependent variable and the independent variables, It adds polynomial terms to the linear regression model to capture more complex relationships, For example when we want to predict « non: Linear trend like population growth over time we use polynomial regression, 4, Ridge & Lasso Regressio ized versions of linear regression that help avoid Ridye & lasso regression are regula overftting by penalizing large eoetlcients. When there's a risk of overtiting duc to features we use these type of regression algorithm, 5. Support Vector Regression (SVR) Vector Machine ori hats on he So i jon tasks but it ean ed for clasific i hypenplane that minimizes the sum SVR is a ype of regression a (SVM) algorithm, SYM isa type of alyorithm dati Also be used for regression tasks. SVR works by fndin of the squared residuals between the predicted ane actual values 6. Decision Tree Regression