Least Square Method | Definition Graph and Formula

The Least Square method is a popular mathematical approach used in data fitting, regression analysis, and predictive modeling. It helps find the best-fit line or curve that minimizes the sum of squared differences between the observed data points and the predicted values. This technique is widely used in statistics, machine learning, and engineering applications.

Least Squares Method is used to derive a generalized linear equation between two variables. When the value of the dependent and independent variables they are represented as x and y coordinates in a 2D Cartesian coordinate system. Initially, known values are marked on a plot. The plot obtained at this point is called a scatter plot.

Then, we try to represent all the marked points as a straight line or a linear equation. The equation of such a line is obtained with the help of the Least Square method. This is done to get the value of the dependent variable for an independent variable for which the value was initially unknown. This helps us to make predictions for the value of a dependent variable.

Why use the Least Square Method?

In statistics, when the data can be represented on a Cartesian plane by using the independent and dependent variables as the x and y coordinates, it is called scatter data. This data might not be useful in making interpretations or predicting the values of the dependent variable for the independent variable. So, we try to get an equation of a line that fits best to the given data points with the help of the Least Square Method. 

Least Square Method Formula

The Least Square Method formula finds the best-fitting line through a set of data points. For a simple linear regression, which is a line of the form y=mx+c, where y is the dependent variable, x is the independent variable, a is the slope of the line, and b is the y-intercept, the formulas to calculate the slope (m) and intercept (c) of the line are derived from the following equations:

• Slope (m) Formula: m = n(∑xy)−(∑x)(∑y) / n(∑x²)−(∑x)²​
• Intercept (c) Formula: c = (∑y)−a(∑x) / n​

Where:

• n is the number of data points,
• ∑xy is the sum of the product of each pair of x and y values,
• ∑x is the sum of all x values,
• ∑y is the sum of all y values,
• ∑x² is the sum of the squares of the x values.

Steps to find the line of Best Fit by using the Least Squares Method :

• Step 1: Denote the independent variable values as x_i and the dependent ones as y_i.
• Step 2: Calculate the average values of x_i and y_i as X and Y.
• Step 3: Presume the equation of the line of best fit as y = mx + c, where m is the slope of the line and c represents the intercept of the line on the Y-axis.
• Step 4: The slope m can be calculated from the following formula:

m = [Σ (X - x_i)×(Y - y_i)] / Σ(X - x_i)²

• Step 5: The intercept c is calculated from the following formula:

c = Y - mX

Thus, we obtain the line of best fit as y = mx + c, where values of m and c can be calculated from the formulae defined above.

These formulas are used to calculate the parameters of the line that best fits the data according to the criterion of the least squares, minimizing the sum of the squared differences between the observed values and the values predicted by the linear model.

Least Square Method Graph

Let us have a look at how the data points and the line of best fit obtained from the Least Square method look when plotted on a graph.

The red points in the above plot represent the data points for the sample data available. Independent variables are plotted as x-coordinates, and dependent ones are plotted as y-coordinates. The equation of the line of best fit obtained from the Least Square method is plotted as the red line in the graph. 

We can conclude from the above graph how the Least Square method helps us to find a line that best fits the given data points and hence can be used to make further predictions about the value of the dependent variable where it is not known initially.

Limitations of the Least Square Method

The Least Squares method assumes that the data is evenly distributed and doesn't contain any outliers for deriving a line of best fit. However, this method doesn't provide accurate results for unevenly distributed data or data containing outliers.

Check: Least Square Regression Line

How Do You Calculate Least Square?

To calculate the least squares solution, you typically need to:

1. Determine the equation of the line you believe best fits the data.
2. Calculate the residuals (differences) between the observed values and the values predicted by your model.
3. Square each of these residuals and sum them up.
4. Adjust the model to minimize this sum.

Least Square Method Solved Examples

Problem 1: Find the line of best fit for the following data points using the Least Square method: (x,y) = (1,3), (2,4), (4,8), (6,10), (8,15).

Solution:

Here, we have x as the independent variable and y as the dependent variable. First, we calculate the means of x and y values denoted by X and Y respectively.

X = (1+2+4+6+8)/5 = 4.2
Y = (3+4+8+10+15)/5 = 8

	xi	yi	X - xi	Y - yi	(X-xi)*(Y-yi)	(X - xi)2
	1	3	3.2	5	16	10.24
	2	4	2.2	4	8.8	4.84
	4	8	0.2	0	0	0.04
	6	10	-1.8	-2	3.6	3.24
	8	15	-3.8	-7	26.6	14.44
Sum (Σ)			0	0	55	32.8

The slope of the line of best fit can be calculated from the formula as follows:
m = (Σ (X - x_i)*(Y - y_i)) /Σ(X - x_i)²
m = 55/32.8 = 1.68 (rounded upto 2 decimal places)

Now, the intercept will be calculated from the formula as follows:
c = Y - mX
c = 8 - 1.68*4.2 = 0.94

Thus, the equation of the line of best fit becomes, y = 1.68x + 0.94.

Problem 2: Find the line of best fit for the following data of heights and weights of students of a school using the Least Squares method:

• Height (in centimeters): [160, 162, 164, 166, 168]
• Weight (in kilograms): [52, 55, 57, 60, 61]

Solution:

Here, we denote Height as x (independent variable) and Weight as y (dependent variable). Now, we calculate the means of x and y values denoted by X and Y respectively.

X = (160 + 162 + 164 + 166 + 168 ) / 5 = 164
Y = (52 + 55 + 57 + 60 + 61) / 5 = 57

	xi	yi	X - xi	Y - yi	(X-xi)*(Y-yi)	(X - xi)2
	160	52	4	5	20	16
	162	55	2	2	4	4
	164	57	0	0	0	0
	166	60	-2	-3	6	4
	168	61	-4	-4	16	16
Sum ( Σ )			0	0	46	40

Now, the slope of the line of best fit can be calculated from the formula as follows:
m = (Σ (X - x_i)✕(Y - y_i)) / Σ(X - x_i)²
m = 46/40 = 1.15

Now, the intercept will be calculated from the formula as follows:
c = Y - mX
c = 57 - 1.15*164 = -131.6

Thus, the equation of the line of best fit becomes, y = 1.15x - 131.6

Real-Life Application of the Least Squares Method

• Predicting housing prices using linear regression.
• Estimating trends in the financial market.
• Fitting Curves to experimental data in physics and chemistry.
• Calibrating sensors in engineering.

vivek1208

Follow

Improve

Article Tags :

• Mathematics
• School Learning
• Math-Concepts