Chapter 02

PARTIAL DIFFERENTIATION

2.1 FUNCTION OF SEVERAL VARIABLES.

In single-variable calculus we were concerned with functions that map the real numbers ℝ to ℝ,
sometimes called real functions of one variable, meaning the input is a single real number and
the output is likewise a single real number. Now we turn to functions of several variables, meaning
several input variables, functions 𝑓: ℝ! → ℝ. We will deal primarily with 𝑛 = 2 and to a lesser
extent 𝑛 = 3; in fact many of the techniques we discuss can be applied to larger values of 𝑛 as
well.
A function 𝑓: ℝ" → ℝ maps a pair of values (𝑥, 𝑦) to a single real number. The three-dimensional
coordinate system we have already used is a convenient way to visualize such functions: above
each point (𝑥, 𝑦) in the 𝑥𝑦 −plane we graph the point (𝑥, 𝑦, 𝑧), where of course 𝑧 = 𝑓(𝑥, 𝑦).

Example 2.1.1 Consider 𝑓(𝑥, 𝑦) = 3𝑥 + 4𝑦 − 5. Writing this as 𝑧 = 3𝑥 + 4𝑦 − 5 and then 3𝑥 +

4𝑦 − 𝑧 = 5 we recognize the equation of a plane. In the form 𝑓(𝑥, 𝑦) = 3𝑥 + 4𝑦 − 5 the
emphasis has shifted: we now think of 𝑥 and 𝑦 as independent variables and 𝑧 as a variable
dependent on them, but the geometry is unchanged.

Example 2.1.2 Note that 𝑥 " + 𝑦 " + 𝑧 " = 4 represents a sphere of radius 2. We cannot write this
in the form 𝑓(𝑥, 𝑦), since for each 𝑥 and 𝑦 in the disk 𝑥 " + 𝑦 " < 4 there are two corresponding
points on the sphere. As with the equation of a circle, we can resolve this equation into two
functions, 𝑓(𝑥, 𝑦) = 54 − 𝑥 " − 𝑦 " , and 𝑓(𝑥, 𝑦) = −54 − 𝑥 " − 𝑦 " 𝑟𝑒𝑝𝑟𝑒𝑠𝑒𝑛𝑡 the upper and
lower hemispheres. Each of these is an example of a function with a restricted domain: only certain
values of x and y make sense (namely, those for which 𝑥 " + 𝑦 " ≤ 4) and the graphs of these
functions are limited to a small region of the plane.

Example 2.1.3 Consider 𝑓(𝑥, 𝑦) = 𝑥 " + 𝑦 " .

1 Prashan Rathnayaka, SoC, IIT-Colombo.

2.2 LIMITS AND CONTINUITY

Understanding the concept of a limit was crucial to establish calculus for single-variable functions.
This was essential for grasping continuous functions and defining the derivative. Limits for
functions of two variables are often much more complex, but thankfully, most functions we
encounter are relatively straightforward.
The potential difficulty is mainly due to the fact that there are many ways to approach a point in
the 𝑥𝑦 − plane. If we want to say that
lim 𝑓(𝑥, 𝑦) = 𝐿
($,&)→(),*)

we need to capture the idea that as (𝑥, 𝑦) gets close to (𝑎, 𝑏) then 𝑓(𝑥, 𝑦) gets close to 𝐿.

For functions of one variable, 𝑓(𝑥), there are only two ways that 𝑥 can approach 𝑎: from the left
or right. But there are an infinite number of ways to approach (𝑎, 𝑏): along any one of an infinite
number of lines, or an infinite number of parabolas, or an infinite number of sine curves, and so
on. We might hope that it’s really not so bad—suppose, for example, that along every possible line
through (𝑎, 𝑏) the value of 𝑓(𝑥, 𝑦) gets close to 𝐿; surely this means that 𝑓(𝑥, 𝑦) approaches 𝐿 as
(𝑥, 𝑦) approaches (𝑎, 𝑏). Sadly, no.

Example 2.2.1 Consider

𝑥𝑦 "
𝑓(𝑥, 𝑦) =
𝑥" + 𝑦+

2 Prashan Rathnayaka, SoC, IIT-Colombo.

When 𝑥 = 0 or 𝑦 = 0, 𝑓(𝑥, 𝑦) is 0, so the limit of 𝑓(𝑥, 𝑦) approaching the origin along either the
𝑥 or 𝑦 axis is 0. Moreover, along the line 𝑦 = 𝑚𝑥,
𝑚" 𝑥 ,
𝑓(𝑥, 𝑦) =
𝑥 " + 𝑚+ 𝑥 +
As 𝑥 approaches 0 above expression approaches 0 as well. So along every line through the origin
𝑓(𝑥, 𝑦) approaches 0.
Now suppose we approach the origin along 𝑥 = 𝑦 " . Then
𝑦"𝑦" 𝑦+ 1
𝑓(𝑥, 𝑦) = + +
= +=
𝑦 +𝑦 2𝑦 2
so the limit is 1/2. Thus, there is no limit at (0,0).

2.2.1 WORKING RULE TO FIND THE LIMIT

Step 01: Find the value of 𝑓(𝑥, 𝑦) along 𝑥 = 𝑎.

Step 02: Find the value of 𝑓(𝑥, 𝑦) along 𝑦 = 𝑏.

If the value in step 01 and step 02 remain the same, the limit exists; otherwise not.

Step 03: Find the limit along 𝑦 = 𝑚𝑥 and 𝑦 = 𝑚𝑥 " . If the value of the limit does not contains 𝑚,

then the limit exists; If it contains 𝑚, the limit does not exist.

DEFINITION 2.2.1
If 𝑓(𝑥, 𝑦) is continuous at (𝑎, 𝑏) then,
lim 𝑓(𝑥, 𝑦) = 𝑓(𝑎, 𝑏)
($,&)→(),*)

Example 2.2.2 Test the function 𝑓(𝑥, 𝑦) for continuity. Where,

𝑥, − 𝑦,
; when 𝑥 ≠ 0, 𝑦≠0
𝑓(𝑥, 𝑦) = G 𝑥 " + 𝑦 "
0 ; when 𝑥 = 0, 𝑦=0

3 Prashan Rathnayaka, SoC, IIT-Colombo.

2.3 PARTIAL DERIVATIVES

Let 𝑧 = 𝑓(𝑥, 𝑦) be a function of two variables 𝑥 and 𝑦. If we we keep 𝑦 constant and 𝑥 varies then
𝑧 becomes a function of 𝑥 only. The derivative of 𝑧 with respect to 𝑥, keeping 𝑦 as a constant is
called partial derivative of 𝑧, with respect to 𝑥 is denoted by symbols,

𝜕𝑧 𝜕𝑓
, , 𝑓$ (𝑥, 𝑦)
𝜕𝑥 𝜕𝑥

Then,

𝜕𝑧 𝑓(𝑥 + 𝛿𝑥, 𝑦) − 𝑓(𝑥, 𝑦)

= lim
𝜕𝑥 -$→. 𝛿𝑥

Similarly, the partial derivative of 𝑧, with respect to 𝑦 keeping 𝑥 as a constant is denoted by,

𝜕𝑧 𝜕𝑓
, , 𝑓& (𝑥, 𝑦)
𝜕𝑦 𝜕𝑦

𝜕𝑧 𝑓(𝑥, 𝑦 + 𝛿𝑦) − 𝑓(𝑥, 𝑦)

= lim
𝜕𝑦 -&→. 𝛿𝑦

Example 2.3.1 The partial derivative with respect to 𝑥 of 𝑥 , + 3𝑥𝑦 is 3𝑥 " + 3𝑦.

4 Prashan Rathnayaka, SoC, IIT-Colombo.

Example 2.3.2 The partial derivative with respect to 𝑦 of 𝑓(𝑥, 𝑦) = sin 𝑥𝑦 + 3𝑥𝑦 is:

2.3.1 PARTIAL DERIVATIVES OF HIGHER ORDERS

/0 /0
Let 𝑧 = 𝑓(𝑥, 𝑦) then /$ and /& being the function of 𝑥 and 𝑦 can further be differentiated partially

with respect to 𝑥 and 𝑦.

Symbolically
𝜕 𝜕𝑧 𝜕"𝑧 𝜕"𝑓
Q R = " or or 𝑓$$
𝜕𝑥 𝜕𝑥 𝜕𝑥 𝜕𝑥 "
𝜕 𝜕𝑧 𝜕"𝑧 𝜕"𝑓
Q R= or or 𝑓&$
𝜕𝑦 𝜕𝑥 𝜕𝑦𝜕𝑥 𝜕𝑦𝜕𝑥
𝜕 𝜕𝑧 𝜕"𝑧 𝜕"𝑓
Q R= or or 𝑓$&
𝜕𝑥 𝜕𝑦 𝜕𝑥𝜕𝑦 𝜕𝑥𝜕𝑦
It’s easy to see where some complication is going to come from: with two variables there are four
possible second derivatives. To take a derivative, we must take a partial derivative with respect to
𝑥 or 𝑦, and there are four ways to do it: 𝑥 then 𝑥, 𝑥 then 𝑦, 𝑦 then 𝑥, 𝑦 then 𝑦.

Example 2.3.3 Compute all four-second derivatives of 𝑓(𝑥, 𝑦) = 𝑥 " 𝑦 "

5 Prashan Rathnayaka, SoC, IIT-Colombo.

You will have noticed that two of these are the same, the mixed partials computed by taking
partial derivatives with respect to both variables in the two possible orders. This is not an
accident—as long as the function is reasonably nice, this will always be true.

THEOREM 2.1 Clairaut’s Theorem

If the mixed partial derivatives are continuous, then they are equal.

$&
Example 2.3.4 Compute the mixed partials of 𝑓(𝑥, 𝑦) = $ ! 1& ! .

2.3.2 MAXIMA, MINIMA AND SADDLE POINT

A function 𝑓(𝑥, 𝑦) is said to have a maximum value at 𝑥 = 𝑎, 𝑦 = 𝑏, if there exists a small

neighbourhood of (𝑎, 𝑏) such that,
𝑓(𝑎, 𝑏) > 𝑓(𝑎 + ℎ, 𝑏 + 𝑘)
A function 𝑓(𝑥, 𝑦) is said to have a minimum value at 𝑥 = 𝑎, 𝑦 = 𝑏, if there exists a small
neighbourhood of (𝑎, 𝑏) such that,
𝑓(𝑎, 𝑏) < 𝑓(𝑎 + ℎ, 𝑏 + 𝑘)
The maximum and minimum values of a function are also called extreme or extremum values of
the function.
A Saddle point or a minimax point is a point where a function is neither maximum nor minimum.
Geometrical interpretation: The surface (looks like the leather seat on the back of the horse) forms
a rigid rising in one direction and falling in another direction.

6 Prashan Rathnayaka, SoC, IIT-Colombo.

THEOREM 2.2 Suppose that the second partial derivatives of 𝑓(𝑥, 𝑦) are continuous near
(𝑥. , 𝑦. ), and 𝑓$ (𝑥. , 𝑦. ) = 𝑓& (𝑥. , 𝑦. ) = 0. We denote by D the discriminant:

"
𝐷(𝑥. , 𝑦. ) = 𝑓$$ (𝑥. , 𝑦. )𝑓&& (𝑥. , 𝑦. ) − Y𝑓$& (𝑥. , 𝑦. )Z

If 𝐷 > 0 and if 𝑓$$ (𝑥. , 𝑦. ) < 0: there is a local maximum at (𝑥. , 𝑦. )

If 𝐷 > 0 and if 𝑓$$ (𝑥. , 𝑦. ) > 0: there is a local minimum at (𝑥. , 𝑦. )
If 𝐷 < 0: there is neither a maximum nor a minimum at (𝑥. , 𝑦. )
if 𝐷 = 0: the test fails, and further investigation is needed.

Example 2.3.5 Verify that 𝑓(𝑥, 𝑦) = 𝑥 " + 𝑦 " has a minimum at (0,0).

7 Prashan Rathnayaka, SoC, IIT-Colombo.

Example 2.3.6 Find all local maxima and minima for 𝑓(𝑥, 𝑦) = 𝑥 " − 𝑦 " .

2.4 HESSIAN MATRIX

The Hessian matrix, a square matrix of second-order partial derivatives of a scalar-valued function,
is a fundamental tool in AI and Data Science for optimization and analysis.

For example,
Support Vector Machines (SVMs): In training SVMs, the Hessian matrix is used in the quadratic
programming problem formulation.
Neural Networks: During backpropagation, the Hessian can be used to adjust learning rates and
improve convergence in training deep networks.

The Hessian matrix is a matrix of second-order partial derivatives. Suppose we have a function 𝑓
of 𝑛 variables, i.e.,
𝑓: ℝ! → ℝ
The Hessian of 𝑓 is given by the following matrix and denoted as 𝐻2
𝜕"𝑓 𝜕"𝑓
⎡ ⋯ ⎤
⎢ 𝜕𝑥3" 𝜕𝑥3 𝜕𝑥! ⎥
𝐻2 = ⎢ ⋮ ⋱ ⋮ ⎥
⎢ 𝜕"𝑓 𝜕 𝑓 ⎥⎥
"
⎢ ⋯
⎣𝜕𝑥! 𝜕𝑥3 𝜕𝑥!" ⎦

8 Prashan Rathnayaka, SoC, IIT-Colombo.

Note that: The determinant of the Hessian is also called the discriminant 𝑫 of 𝒇. (Introduced
in theorem 2.2)

Example 2.4.1 Find the Hessian matrix and its determinant for 𝑓(𝑥, 𝑦) = 𝑥 , + 2𝑦 " + 3𝑥𝑦 "

9 Prashan Rathnayaka, SoC, IIT-Colombo.