ML - Different Regression types

ML | Dummy variable trap in Regression Models

Last Updated : 08 Sep, 2021

Before learning about the dummy variable trap, let’s first understand what actually dummy variable is.

Dummy Variable in Regression Models:
In statistics, especially in regression models, we deal with various kinds of data. The data may be quantitative (numerical) or qualitative (categorical). The numerical data can be easily handled in regression models but we can’t use categorical data directly, it needs to be transformed in some way.

For transforming categorical attributes to numerical attributes, we can use the label encoding procedure (label encoding assigns a unique integer to each category of data). But this procedure is not alone that suitable, hence, One hot encoding is used in regression models following label encoding. This enables us to create new attributes according to the number of classes present in the categorical attribute i.e if there are n number of categories in categorical attribute, n new attributes will be created. These attributes created are called Dummy Variables. Hence, dummy variables are “proxy” variables for categorical data in regression models.
These dummy variables will be created with one-hot encoding and each attribute will have a value of either 0 or 1, representing the presence or absence of that attribute.

Dummy Variable Trap:
The Dummy variable trap is a scenario where there are attributes that are highly correlated (Multicollinear) and one variable predicts the value of others. When we use one-hot encoding for handling the categorical data, then one dummy variable (attribute) can be predicted with the help of other dummy variables. Hence, one dummy variable is highly correlated with other dummy variables. Using all dummy variables for regression models leads to a dummy variable trap. So, the regression models should be designed to exclude one dummy variable.

For Example –
Let’s consider the case of gender having two values male (0 or 1) and female (1 or 0). Including both the dummy variable can cause redundancy because if a person is not male in such case that person is a female, hence, we don’t need to use both the variables in regression models. This will protect us from the dummy variable trap.

ML - Different Regression types

deepak_jain

Improve

Article Tags :

Practice Tags :

Similar Reads

How to Transform Variables for Multiple Regression in R?

Multiple regression analysis is a robust statistical method commonly used to examine the relationships between multiple predictor variables and a single outcome variable. However, real-world datasets often exhibit non-linear relationships, heteroscedasticity, and non-normality, violating the assumpt

ML | Adjusted R-Square in Regression Analysis

Prerequisite: Linear Regression, R-square in Regression Why Adjusted-R Square Test: R-square test is used to determine the goodness of fit in regression analysis. Goodness of fit implies how better regression model is fitted to the data points. More is the value of r-square near to 1, better is the

ML - Different Regression types

Regression Analysis: It is a form of predictive modelling technique which investigates the relationship between a dependent (target) and independent variable (s) (predictor). To establish the possible relationship among different variables, various modes of statistical approaches are implemented, kn

Stepwise Regression in Python

Stepwise regression is a method of fitting a regression model by iteratively adding or removing variables. It is used to build a model that is accurate and parsimonious, meaning that it has the smallest number of variables that can explain the data. There are two main types of stepwise regression: F

Types of Regression Techniques in ML

Regression Analysis is a fundamental concept in machine learning used to model relationships between dependent and independent variables. Various regression techniques are tailored to different data structures and objectives. Below is an exploration of key regression techniques, their significance,

Deciding threshold for glm logistic regression model in R

Logistic regression is a powerful statistical method used for modeling binary outcomes. When applying logistic regression in practice, one common challenge is deciding the threshold probability that determines the classification of observations into binary classes (e.g., yes/no, 1/0). This article e

Residual plots for Nonlinear Regression

Nonlinear regression is a form of regression analysis where data is fit to a model expressed as a nonlinear function. Unlike linear regression, where the relationship between the independent and dependent variables is linear, nonlinear regression involves more complex relationships. One of the criti

Simple Linear Regression in Python

Simple linear regression models the relationship between a dependent variable and a single independent variable. In this article, we will explore simple linear regression and it's implementation in Python using libraries such as NumPy, Pandas, and scikit-learn. Understanding Simple Linear Regression

Find the Regression Output in R

In R Programming Language we can Interpret Regression Output by using various functions depending on the type of regression analysis you are conducting. The two most common types of regression analysis are linear regression and logistic regression. Here, I'll provide examples of how to find the regr

Transport Demand Prediction using Regression

Transport demand prediction is a crucial aspect of transportation planning and management. Accurate demand forecasts enable efficient resource allocation, improved service planning, and enhanced customer satisfaction. Regression analysis, a statistical method for modeling relationships between varia