Filtering and Convolutions

Jack Xin (Lecture) and J. Ernie Esser (Lab)

Abstract
Class notes on filtering, convolutions, eigenvalue/eigenvector, diagonalization, and
z-transform.

Filtering

Filtering refers to linear transforms that change the frequency contents of signals. Depending on whether high (low) frequencies are attenuated, filtering process is called low (high)
pass.

1.1

Low Pass Filtering

Example: two point moving average, recall the linear time invariant system:
y(n) = [s(n) + s(n 1)]/2,

(1.1)

where s(n) is a periodic signal, s(1) = s(N ), N 2. The output y is a smoother signal
with higher frequency component damped and low frequency component maintained. To
see this, let s(n) = sin(2 f n/N ), then:
1
1
s(n) + s(n 1)
2
2
1
1
=
sin(2 f n/N ) + sin(2 f (n 1)/N )
2
2
1
1
=
(1 + cos(2 f /N )) sin(2 f n/N ) sin(2 f /N ) cos(2 f n/N ),
2
2
= A1 sin(2f n/N ) A2 cos(2 f n/N ),

y(n) =

where:

1
1
A1 = (1 + cos(2 f /N )), A2 = sin(2 f /N ).
2
2

Department of Mathematics, UCI, Irvine, CA 92617.

(1.2)

At low frequency, f 0, A1 1, A2 0, so
y(n) sin(2 f n/N ) = s(n).
Low frequency input is almost preserved.
At high frequency, f N/2 (near Nyquist frequency), A1 0, A2 0, so y(n) 0. High
frequency input is almost zeroed out.

fft

magnitute

of

h=[1/2,1/2,zeros(1,126)]

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

20

40

60

80

100

120

140

Figure 1: Illustration of frequency domain response of two point moving average.

1.2

Convolution

A useful way to view filtering is by convolution. The first type of convolution is:
Definition 1.1 (Circular Convolution). Let x and y C N . Circular convolution vector
w = (w1 , w2 , , wN 1 ) C N is:
wm =

N
1
X

xk y(mk) mod N ,

(1.3)

k=0

for 0 m N 1, mod is the remainder of m k divided by N . Notation is w = x  y.

fft

magnitute

of

h=[1/2,1/2,zeros(1,126)]

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

20

40

60

80

100

120

140

Figure 2: Illustration of frequency domain response of two point difference.

In Matlab, the command
mod(m k, N ),
computes m k mod N . For example, mod(7,2)=1, mod(9,4)=2, mod(11,4)=3.
Componentwise, w0 is the inner product:
w0 = [x0 x1 x2 xN 1 ] [y0 yN 1 yN 2 y1 ],
similarly
w1 = [x0 x1 x2 xN 1 ] [y1 y0 yN 1 y2 ],
and
w2 = [x0 x1 x2 xN 1 ] [y2 y1 y0 y3 ],
and so on, until:
wN 1 = [x0 x1 x2 xN 1 ] [yN 1 yN 2 yN 3 y0 ].
Then wm repeats these values, wm = wm modN .
In matrix form,
w = My x,
3

where x = [x0 x1 x2 xN 1 ], and

My =

y0
y1
y2

yN 1

yN 1 yN 2
y0
yN 1
y1
y0

yN 2 yN 3

y1
y2
y3

y0

The matrix My is called circulant matrix, notice that its row entries rotate around. Alternatively, each diagonal is a vector with identitical entries.
In Matlab, the command
diag(y0 ones(N, 1), 0),
produces the N N diagonal matrix with y0 on the diagonal. The command
diag(y1 ones(N 1, 1), 1),
gives the N N matrix with y1 on the first subdiagonal. Similarly,
diag(yN 1 ones(N 1, 1), 1),
gives the N N matrix with yN 1 on the first super-diagonal. Next do
diag(y2 ones(N 2, 1), 2) + diag(yN 2 ones(N 2, 1), 2),
to fill in the next sub(super)-diagonals, and so on. Adding these matrices up generates My .

The two point moving average filter can be written (strictly speaking approximated) as
circular convolution w = x  h, h0 = 1/2, h1 = 1/2, hk = 0, k = 2 : N 1. In Matlab, do:
N = 8;
Mh = diag(1/2 ones(N, 1), 0) + diag(1/2 ones(N 1, 1), 1);
Mh (1, N ) = 1/2,
display shows that Mh has 1/2 on the main diagonal, first subdiagonal, and upper right
corner. Now enter:
x = [1, 1, 1, 1, 1, 1, 1, 1]0 ; w = Mh x,
4

we see w = zeros(8, 1). Oscillation in x is averaged out to zero.

Now re-define x and calculate w:
x = [1, 1, 1, 1, 1, 1, 1, 1]0 ; w = Mh x
gives:
w = [0, 1, 0, 0, 1, 0, 0, 0],
notice that averaging reduces sign changes in x.
Another way to compute circular convolution is using the convolution-multiplication
theorem. Thanks to periodicity of mod N,
DF T (w) = DF T (x) DF T (h).
So we take fft to do regular multiplication in Fourier domain, then ifft back to recover w.
For the above example, enter one line in Matlab:
real(if f t((f f t(x). f f t(h)))),
to get the same results on w.
The effect of convolution with h in Fourier (frequency) domain is componentwise multiplication by vector DFT(h), which is easier to visualize by plotting.
In Matlab,
define h = [1/2, 1/2, zeros(1, 126)],
plot(abs(fft(h))),
we get Figure 1 which shows a low pass filter (attenuating high frequencies).

1.3

High Pass Filtering

Example: two point difference, consider the linear time invariant system:
y(n) = [s(n) s(n 1)]/2,

(1.4)

where s(n) is a periodic signal, s(1) = s(N ), N 2. The output y is a less smooth signal
with low frequency component damped and frequency component maintained.
In Matlab, define h = [1/2, 1/2, zeros(1, 6)], x = [1, 1, 1, 1, 1, 1, 1, 1]; then:
w = real(if f t((f f t(x). f f t(h)))) = [1, 0, 0, 0, 1, 0, 0, 0],
5

notice that two point differencing zeros out identical entries, and maintains entries with
sign change. The first 1 is due to the last -1 in x, the middle -1 stays because the entry in
front of it equals 1.
To visualize the filtering effect in Fourier domain, define h = [1/2, 1/2, zeros(1, 126)],
plot(abs(fft(h))),
we get Figure 2 which shows a high pass filter.

1.4

Eigenvalues of Square Matrix

Eigenvector v 6= 0 is a special vector of a square matrix A such that A v lies in the same
direction of v:
A v = v, v 6= 0,

(1.5)

where is a complex number in general, so called eigenvalue. Rewriting (1.5) as

(A I)v = 0,

(1.6)

shows that A I is degenerate (or its rows are not linearly independent), hence:
determinant|A I| = 0.

(1.7)

Equation (1.7) is a polynomial in .

Example 1:

A=

1 2
1 0

Equation (1.7) is:


 1 2
det 
1



 = 2 2 = 0.


Two roots (eigenvalues) are:

1 = 1, 2 = 2.
When = 1 , v1 1 v2 = 0, so the eigenvector is v = (1, 1) (or any multiple of it). When
= 2 , the corresponding eigenvector is (2, 1).
In Matlab, define:
A = [1 2; 1 0],
command: [V, D] = eig(A) produces a diagonal matrix D of eigenvalues and a full matrix
V whose columns are the corresponding eigenvectors.
[V, D] = eig(A)
6

shows that:


D=

V =

2 0
0 1

0.8944 0.7071
0.4472 0.7071

The first column vector of V is 0.4472*[2,1], an eigenvector corresponding to eigenvalue

2. The second column vector of V is -0.7071*[1, -1], an eigenvector corresponding to
eigenvalue -1, in the same order as 2 and -1 appear on the diagonal of D. Notice that
eigenvectors are determined only up to a constant multiple.
Example 2:

A=

1 1
1 0

The eigenvalue equation is:

2 + 1 = 0,
giving two complex eigenvalues:
1,2 = (1

3 i)/2,

and eigenvectors (1,2 , 1).

In Matlab, we get by the same command [V,D]= eig(A) that:


0.5 + 0.8660 i 0
D=
0
0.5 0.8660 i


0.3536 + 0.6124 i 0.3536 0.6124 i
V =
0.7071
0.7071

1.5

Eigenvectors and Diagonalization

If eigenvalues are distinct, the corresponding eigenvectors are linearly independent. Consider 1 6= 2 , suppose there are two constants c1 and c2 such that:
c1 V1 + c2 V2 = 0,

(1.8)

where V1 and V2 are corresponding eigenvectors. Then multiplying A on both sides of (1.8)
gives:
c1 1 V1 + c2 2 V2 = 0.
(1.9)
7

Equations (1.8)-(1.9) can be combined into:




1 1
c1 V1
= 0,
1 2
c2 V2
where V1 and V2 are row vectors. Because 1 =
6 2 , the left square matrix in invertible
(or has nonzero determinant), hence c1 V1 = 0, c2 V2 = 0, implying that c1 = c2 = 0 since
V1 6= 0 and V2 6= 0. The general case follows in similar manner.
If matrix A is order N N with N distinct eigenvalues, then putting the eigenvectors
together form an invertible matrix V = [V1 V2 VN ], here Vj0 are column vectors. It
follows that
A V = V D,

(1.10)

where D is a diagonal matrix with 1 , , N on its diagonal. Taking inverse of V gives:

V 1 A V = D,

(1.11)

which says that A is similar to a diagonal matrix (diagonalizable). Diagonalization holds

if and only if A has N linearly independent eigenvectors. Having N distinct eigenvalues is
a sufficient condition.

1.6

Diagonalization of Circulant Matrix

Define so called basic waveform EN,m C N as:

EN,m = [e2im0/N , e2im1/N , , e2im(N 1)/N ],

(1.12)

the complex conjugate of the m-th row vector of DFT matrix, m = 0, 1, , N 1. Then
DFT(EN,m ) = N (k m). Let w = h  EN,m , then
W = DF T (w) = DF T (h)(k). N (k m) = N DF T (h)(m) (k m),
ifft back to get:
w = h  EN,m = Mh EN,m = DF T (h)(m) EN,m .

(1.13)

So EN,m is an eigenvector of circulant matrix Mh with eigenvalue DF T (h)(m). Because

vectors EN,m , m = 0, 1, , N 1, are orthogonal to each other, the circulant matrix can
be diagonalized by an orthogonal matrix.
In Matlab, define vm = (k m), k = 0, , N 1; or vm = zeros(1, N ), vm(m) = 1.
EN,m = (vm f f t(eye(N )))0 .
8

Linear Convolution and z-Transform

Given x = x(n), n = 0, , N1 1; y = (y(n)), n = 0, , N2 1,

Definition 2.1 (Linear Convolution).
w(n) =

m2
X

x(m) y(n m),

(2.1)

m=m1

where m1 = max{0, n + 1 N2 }, m2 = min{N1 1, n}. The linear convolution is denoted

by w = x y.
Let us zero-pad x and y to length N = N1 + N2 1. The zero padded versions are xa
and ya respectively. Then (2.1) can be expressed as:
w(n) =

n
X

xa (m) ya (n m), 0 n N 1.

(2.2)

m=0

Zero-padded vectors have the same length. We calculate

xa  ya =
=

N
1
X
m=0
n
X

xa (m) ya (n m modN )
N
1
X

xa (m) ya (n m) +

m=0

xa (m) ya (n m + N ).

(2.3)

m=n+1

For the product term in the second sum to be nonzero, the lengths N1 and N2 of x and y
satisfy the constraints:
n + 1 m N1 1,
0 n m + N,
n m N1 .

(2.4)
(2.5)
(2.6)

The last inequality (2.6) can be seen as follows. If n m > N1 , then n m + N >
N N1 = N2 1, so ya (n m + N ) = 0. The two inequalities (2.5)-(2.6) combine into:
N1 + n m N + n,

(2.7)

which has no overlap with (2.4). It follows that

xa  ya (n) = x y(n), 0 n N 1,
9

(2.8)

which allows one to use fft to calculate linear convolution.

The linear convolution can be computed by multiplying two polynomials with coefficients from x(n) and y(n). Define the z-polynomials:
X(z) =

Y (z) =

N
1 1
X
n=0
N
2 1
X

xn z n ,

(2.9)

yn z n ,

(2.10)

n=0

they are called z-transforms of x and y respectively. The z-transform of linear convolution
w equals X(z)Y (z) which has N1 + N2 1 coefficients.
In Matlab, w = conv(x, y).
Example: compute w = x y, x = [2, 3], y = [1, 4, 5],
w = conv(x, y) = [2, 5, 2, 15].

(2.11)

Here N1 = 2, N2 = 3, N = N1 + N2 1 = 4, so zero-pad as:

xa = [x, 0, 0], ya = [y, 0].
w = xa  ya = real(if f t(f f t(xa ). f f t(ya ))) = [2, 5, 2, 15].
Finally:
X(z) = 2 + 3z 1 ,
Y (z) = 1 4z 1 + 5z 2 ,
X(z)Y (z) = 2 5z 1 2z 2 + 15z 3 ,
giving again (2.11).

2.1

Circular Convolution and z-Transform

Let x, y be N-dimensional vectors with complex components, w = x  y. Let X(z), Y (z),

W (z) be their z-transforms. Then:
W (z) = X(z)Y (z) mod z N .

(2.12)

Here mod z N means that z n and z nN are regarded as the same, for any integer n.

10

Example: x = [1, 1, 1], y = [1/2, 1/2, 0], x  y = [0, 1, 0] (check it in Matlab). On the
other hand, X(z) = 1 + z 1 z 2 , Y (z) = 1/2 + 1/2z 1 , direct multiplication shows:
X(z)Y (z) = 1/2 + z 1 1/2z 3 = z 1 mod z 3 ,
also producing vector [0, 1, 0].
A simple way to see (2.12) other than direct verification by multiplication is as follows.
Let k = e2ik/N , the k-th root of unity. Then by definition of DFT, W (k ) is DFT(w),
similarly for x and y, so:
W (k ) = X(k )Y (k ),
0 k N 1. Define:
P (z) = X(z)Y (z) mod z N .
Because km = kn if m = n mod N ,
P (k ) = X(k )Y (k ),
so P (k ) = W (k ). It follows that the degree N 1 polynomial z N 1 (W (z) P (z))
vanishes at k , k = 0, 1, , N 1, or has N distinct roots, so it must be identically zero,
or W (z) P (z).
Formula (2.12) yields an algorithm for computing circular convolution by linear convolution. The linear convolution u = x y corresponds to coefficients in X(z)Y (z), which
contains powers z 0 , z 1 , , z 2(N 1) . To get circular convolution, we add the coefficients
of z n and z nN , for n = 0, 1, , N 2,
w(n) = u(n) + u(n + N ),
while w(N 1) = u(N 1).

2.2

Bilateral z-Transform and Properties

The bilateral z-transform of a sequence x(n) is:

X(z) =

+
X
n=

for z in the domain of convergence.

Example 1: x(n) = (n), then X(z) = 1.
11

x(n) z n ,

(2.13)

Example 2: Let a be a complex number, x(n) = an if n 0, x(n) = 0 if n < 0. Then:

X(z) =

+
X

a z

n=0

+
X

(az 1 )n =

n=0

1
,
1 a z 1

if |z| > |a|.

Some useful properties are:
Linearity: w(n) = a x(n) + b y(n) = W (z) = a X(z) + b Y (z), for any two complex
numbers a and b.
Time shift: w(n) = x(n m) = W (z) = z m X(z), for any integer m.
X

x(n m) z n =

x(n) z (n+m) = z m X(z).

So z 1 means delay by 1.
Time reversal: w(n) = x(n) = W (z) = X(z 1 ).
Multiplication: w(n) = an x(n) = W (z) = X(z/a), if a 6= 0.
Differentiation: w(n) = nx(n) = W (z) = z dX(z)/dz.
Linear convolution: w(n) = (x y)(n) = W (z) = X(z) Y (z).
Example 3: The moving average linear system:
y(k) = (x(k) + x(k 1))/2,
is z-transformed into:
Y (z) = (1/2 + 1/2 z 1 ) X(z).
The function (1/2 + 1/2 z 1 ) is called transfer function, denoted by H(z).
A more general linear shift invariance filtering system is of the form:
y(k) + a2 y(k 1) + + am+1 y(k m) = b1 x(k) + b2 x(k 1) + + bn+1 x(k n), (2.14)
assuming zero initial condition (causality). So in time:
y(1) = b1 x(1),
12

y(2) = b1 x(2) + b2 x(1) a2 y(1)

y(3) = b1 x(3) + b2 x(2) + b3 x(1) a2 y(2) a3 y(1)
and so on. The transfer function is:
H(z) =

b1 + b2 z 1 + + bn+1 z n
.
1 + a2 z 1 + + am+1 z m

When m = 0 (H(z) is a polynomial in z 1 ), the filter is called Finite Impulse Response

(FIR). When m > 0 (H(z) is a rational function in z 1 ), the filter is called Infinite Impulse
Response (IIR). When n = 0, the filter is called all-pole recursive or autoregressive (AR).
In Matlab, define:
b = [b1 , b2 , , bn+1 ], a = [1, a2 , , am+1 ],
then the output of filtering is given by:
y = filter(b, a, x).
For the 2-point moving average filter: a = 1; b = [1/2, 1/2].

Matlab Projects

1. Write a Matlab function expressing circular convolution by linear convolution command

conv.
function z=cconv(x,y);
% input vectors x, y are two vectors to be convolved
% output vector z: result of circular convolution
N=length(x);
if (length(y) = N);
error(vectors of unequal lengths); end
z=conv(reshape(x,1,N),reshape(y,1,N));
z=z(1:N)+[z(N+1:2*N-1),0];
Test on some vectors x and y, and compare with real(ifft((fft(x).*fft(y)))).
2. Let x be the step function defined as x = zeros(1, 128), x(64 : 128) = 1. Filter it 20
times by the moving average filter as follows:
13

a = 1; b = [1/2, 1/2]; plot(x), hold on;

for i=1:20, y=filter(b,a,x); x=y; end;
plot(y,r).
3. Define one second duration signal with two tones at (3, 40) Hz sampled at 100 Hz:
fs=100;
t=0:1/fs:1;
x=sin(2*pi*t*3) + 0.25*sin(2*pi*t*40);
Next define 10-point moving average filter:
b=ones(1,10)/10;
y=filter(b,1,x);
plot(t,x,t,y,r).
Comparing output y with input x, we notice that 40 Hz tone is removed, however there
is a delay in y (phase distortion). Matlab double filter function filtfilt corrects the phase
distortion, which in the z-domain proceeds as follows:
X(z) X(z)H(z) X(1/z)H(1/z) time reverse,
pass through filter again:
X(1/z)H(1/z)H(z),
time reverse again:
X(z)H(z)H(1/z).
If z = ei , then H(z)H(1/z) = |H(ei )|2 does not alter phase.
yy= filtfilt(b,1,x); plot(t,x,t,y,,t,yy,:);
compare yy and x.

References
[1] S. A. Broughton and K. Bryan, Discrete Fourier Analysis and Wavelets, Wiley, 2009.
[2] R. C. Gonzalez and R. E. Woods, Digital Image Processing, Third Edition, Pearson
Prentice Hall, 2008.
[3] W. Hartmann, Signals, Sound, and Sensation, Springer, AIP Series in Modern Acoustics and Signal Processing, 4th edition, 2000.
[4] B. Porat, A course on digital signal processing, John Wiley and Sons, 1997.

14