REGRESSION ANALYSIS

Regression analysis is used to assess the relationship between one dependent variable
(DV) and several independent variables (IVs). This is the most commonly used
technique in much of the social sciences research. Given below are some common
applications of regression analysis in business and social sciences:
Marketing: The marketing manager wants to know if sales is dependent on factors
such as advertising spend, number of products introduced, number of sales personnel
etc.
Human Resource: The HR department wants to predict the efficiency of management
trainees based on their academic performance, leadership abilities, IQ level etc.
Finance: Regression is often used in finance to determine how many specific factors
such as the price of a commodity, interest rates, particular industries, or sectors
influence the price movement of an asset.
In order to understand regression analysis fully, it’s essential to comprehend the
following terms:
Dependent Variable: This is the main factor that you’re trying to understand or
predict.
Independent Variables: These are the factors that you hypothesize have an impact
on your dependent variable.

The general form of each type of regression is:

Simple linear regression: Y = a + bX + u

Multiple linear regression: Y = a + b1X1 + b2X2 + b3X3 + ... + btXt + u

Where:
Y = the variable that you are trying to predict (dependent variable).
X = the variable that you are using to predict Y (independent variable).
a = the intercept.
b = the slope.
u = the regression residual.

BASIC CONCEPTS
R Values
R represents the correlation between the observed values and the predicted values
(based on the regression equation obtained) of the DV. R Square is the square of R
and gives the proportion of variance in the dependent variable accounted for by the
set of IVs chosen for the model. R Square is used to find out how well the IVs are able
to predict the DV. However, the R Square value tends to be a bit inflated when the
number of IVs is more or when the number of cases is large. The adjusted R Square
takes into account these things and gives more accurate information about the fitness
of the model. For example, an adjusted R Square value of 0.70 would mean that the
IVs in the model can predict 70% of the variance in the DV. However, R Square is
acceptable in SOCIAL SCIENCE RESEARCH.
Regression Coefficient
Regression coefficient is a measure of how strongly each IV (also known as predictor
variable) predicts the DV. There are two types of regression coefficients—
unstandardized coefficients and standardized coefficients, also known as beta value.
The unstandardized coefficients can be used in the equation as coefficients of
different IVs along with the constant term to predict the value of DV. The
standardized coefficient (beta) is, however, measured in standard deviations. A beta
value of 2 associated with a particular IV indicates that a change of 1 standard
deviation in that particular IV will result in a change of 2 standard deviations in the
DV. If there is just one IV to predict one DV, the beta value obtained would be same
as the correlation coefficient between the DV and the IV.

What is an Intercept and Slope?

The intercept (often labeled the constant) is the expected mean value of Y when all
X=0. Start with a regression equation with one predictor, X. If X sometimes equals 0,
the intercept is simply the expected mean value of Y at that value. If X never equals
0, then the intercept has no intrinsic meaning. In scientific research, the purpose of a
regression model is to understand the relationship between predictors and the
response. If so, and if X never = 0, there is no interest in the intercept. It doesn’t tell
you anything about the relationship between X and Y.
The slope indicates the steepness of a line and the intercept indicates the location
where it intersects an axis. The slope and the intercept define the linear relationship
between two variables, and can be used to estimate an average rate of change. The
greater the magnitude of the slope, the steeper the line and the greater the rate of
change.
By examining the equation of a line, you quickly can discern its slope and y-intercept
(where the line crosses the y-axis).

The slope is positive 5. When x increases by 1, y increases by 5.The y-intercept is 2.

 The slope is negative 0.4. When x increases by 1, y decreases by 0.4. The y-intercept is 7.2.

The slope is 0. When x increases by 1, y neither increases or decreases. The y-intercept is -4.

Usually, this relationship can be represented by the equation y = b 0 + b1x, where b0 is

the y-intercept and b1 is the slope. For example, a company determines that job
performance for employees in a production department can be predicted using the
regression model y = 130 + 4.3x, where x is the hours of in-house training they receive
(from 0 to 20) and y is their score on a job skills test. The value of the y-intercept
(130) indicates the average job skill score for an employee with no training. The value
of the slope (4.3) indicates that for each hour of training, the job skill score
increases, on average, by 4.3 points.