MATRIX ALGEBRA

Methods for Dummies

FIL
January 25 2006

Jon Machtynger & Jen Marchant

 Mikkel Walletin’s (Excellent) slides
 John Ashburner (GLM context)
 Slides from SPM courses:
http://www.fil.ion.ucl.ac.uk/spm/course/
 Good Web Guides
 www.sosmath.com
 http://mathworld.wolfram.com/LinearAlgebra.html
 http://ceee.rice.edu/Books/LA/contents.html
 http://archives.math.utk.edu/topics/
linearAlgebra.html

ACKNOWLEDGEMENTS /
INFO
SCALARS, VECTORS AND MATRICES
 Scalar: Variable described by a single
number – e.g. Image intensity (pixel
value)

• Vector: Variable described by magnitude and direction

• Matrix: Rectangular array of scalars

2

Square (3 x 3) Rectangular (3 x 2) d r c : rth row, cth column

MATRICES
 A matrix is defined by the number of Rows and
the number of Columns.
 An mxn matrix has m rows and n columns.

A = 4x3 matrix
Matlab notes ( ;  End of matrix row )
21 2 53 A = [ 21 5 53 ; 5 34 12 ; 6 33 55 ; 74 27 3 ]

5 34 12 To extract data: Matrix name( row, column )

Scalar Data Point A( 1 , 2 ) = 2
6 33 55 Row Vector A( 2 , : ) = [ 5 34 12 ]
Column Vector A( : , 3 ) = [ 53 ; 12 ; 55 ; 3 ]
74 27 3 Smaller Matrix A(2:4,1:2) = [ 5 34 ; 6 33 ; 74
27 ]
Another Matrix A( 2:2:4 , 2:3 ) = [ 34 12 ; 27 3
]

 A square matrix of order n, is an nxn matrix.

MATRIX ADDITION

Addition (matrix of same size)

– Commutative: A+B=B+A
– Associative: (A+B)+C=A+(B+C)

Subtraction consider as the addition of a negative matrix

MATRIX MULTIPLICATION
Constant (or Scalar)
multiplication of a
matrix:
Matrix multiplication rule:
When A is a mxn matrix & B is a kxl matrix, the multiplication of
AB is only viable if n=k. The result will be an mxl matrix.
VISUALISING MULTIPLYING
A matrix = ( m x n )
a11 a12 a13 b11 b12 ? ?
B matrix = ( k x l )
a21 a22 a23 X b21 b22 = ? ?
A x B is only viable if
a31 a32 a33 b31 b32 ? ?
k=n
a41 a42 a43 ? ? width of A = height of B

Result Matrix = ( m x l )

b11 b12

b21 b22
Jen’s way of
b31 b32
visualising the
multiplication
a11 a12 a13 a11b11 + a12b21 + a13b31 a11b12 + a12b22 + a13b32

a21 a22 a23 a21b11 + a22b21 + a23b31 a21b12 + a22b22 + a23b32

a31 a32 a33 a31b11 + a32b21 + a33b31 a31b12 + a32b22 + a33b32

a41 a42 a43 a41b11 + a42b21 + a43b31 a41b12 + a42b22 + a43b32

TRANSPOSITION

column → row row → column

Mrc = Mcr
 Example

Two vectors:

Inner product = Note: (1xn)(nx1)  (1X1)

scalar

Outer product = matrix Note: (nx1)(1xn)  (nXn)

IDENTITY MATRICES
 Is there a matrix which plays a similar role
as the number 1 in number multiplication?
Consider the nxn matrix:

A square nxn matrix A has

one
A In = In A = A
An nxm matrix A has two!!
In A = A & A Im = A

1 2 3 1 0 0 1+0+0 0+2+0 0+0+3

Worked
example 4 5 6 X 0 1 0 = 4+0+0 0+5+0 0+0+6

A In = A 7 8 9 0 0 1 7+0+0 0+8+0 0+0+9

for a 3x3
matrix:
INVERSE MATRICES
 Definition. A matrix A is nonsingular or invertible if
there exists a matrix B such that: worked example:
-1
2 2+1 -1 + 1
1 1 X = = 1 0
3 3 3 3 3
3
-2+ 2
1 1 1+2
-1 2 3 0 1
3 3 3 3
3

• Notation. A common notation for the inverse of a matrix A is A-1.

• The inverse matrix A-1 is unique when it exists.

• If A is invertible, A-1 is also invertible  A is the inverse matrix of A-1.

• If A is an invertible matrix, then (AT)-1 = (A-1)T

 DETERMINANTS
 Determinant is a function:
 Input is nxn matrix
 Output is a real or a complex number
called the determinant

 In MATLAB
 use the command det(A)" to compute
the determinant of a given square matrix
A

 A matrix A has an inverse matrix A-1 if

and only if det(A)≠0.

- - - + + +
MATRIX INVERSE - CALCULATIONS

i.e. Note: det(A)≠0

A general matrix can be inverted using methods such as the Gauss-Jordan

elimination, Gaussian elimination or LU decomposition
SOME APPLICATION AREAS
 Simultaneous Equations
 Simple Neural Network
 GLM

SOME APPLICATION AREAS

SYSTEM OF LINEAR EQUATIONS
Resolving simultaneous equations can be applied using Matrices:

• Multiply a row by a non-zero constant Also known as

• Interchange two rows Gaussian Elimination
• Add a multiple of one row to another row

…
 SIMPLISTIC NEURAL NETWORK
Weights learned in auto associative manner or given random values…

O = output
vector
I = input vector
W = weight
matrix
Given an input, provide an output… η = Learning
rate
d = Desired
output
t = time
variable

Over time, modify weight matrix to more appropriately reflect desired behaviour
Y
da
ta
ve
ct
or
(V

=
=
ox
el
)

DESIGN MATRIX

X
de
s ig
n
m
at
ri
x

×












pa
r am
= et
er
th s

+
+

eb
et
as
er
(h

ro er
r ve
e:
ct
or 1
to
9)
Y
da
ta
ve
ct
or
(V

=
=
ox
el
)

DESIGN MATRIX

X
de
s ig
n
m
at
ri
x

×












pa
r am
= et
er
th s

+
+

eb
et
as
er
(h

ro er
r ve
e:
ct
or 1
to
9)
QUESTIONS?