Why matrix?
Advanced Statistics
W2: Matrix Algebra Basics
This meeting introduces the basic elements of
matrix algebra used in multivariate statistics.
l
l
Titis Wijayanto
Industrial Engineering, UGM
Why matrices? Lots of data and calculation
Why algebra? Allows you to find unknowns.
To do matrix algebra you need to use the matrix manipulations youve
will lear, e.g.
l
l
l
l
Why matrix?
l
Matrix algebra is the fundamental notational technique used in
multiple correlation, MANOVA, factor analysis, structural
equation modeling (SEM), etc.
Although it is possible to use statistical programs without
understanding matrix algebra, it is much harder to do so.
Matrix algebra is a convenient notational system that allows us
to think about data at a higher (broader) level rather than data
point by data point.
Addition and subtraction
Multiplication
Division
Powers
e.g. Inner / outer
e.g. Inverse
e.g. Transpose
Notation and basic definitions:
Matrix
A matrix is a rectangular or square array of
numbers/variables arranged in rows and
columns.
a11 ,, a1n
a 21 ,, a 2n
A=
= {Aij}
am1 ,, amn
Notation and basic definitions:
Column Vector
Notation and basic definitions:
Row Vector
[m x 1] matrix
a1
a 2
a = = {ai}
am
Notation and basic definitions:
Square Matrix
Same number of rows and columns
5 4 7
B = 3 6 1
2 1 3
[1 x n] matrix
a ' = [a1 a 2 , , an ] = {aj}
We use lowercase letter with to
indicate row vector as the transpose of
column vector
Notation and basic definitions:
Equal Matrix
If A = [aij ]mn , B = [bij ]mn
Then A = B if and only if aij = bij 1 i m, 1 j n
Ex 1: (Equal matrix)
1 2
A=
3 4
a b
B=
c d
If A = B
Then a = 1, b = 2, c = 3, d = 4
Notation and basic definitions:
Symmetric Matrix
Notation and basic definitions:
Symmetric Matrix
Symmetric matrix:
Ex:
A square matrix A is symmetric if A = AT
Skew-symmetric matrix:
A square matrix A is skew-symmetric if AT = A
is a skew-symmetric, find a, b, c?
0 1 2
A = a 0 3
b c 0
0 a b
A = 1 0 c
2 3 0
Sol:
Ex:
Sol:
0 1 2
If A = a 0 3
b c 0
1 2 3
If A = a 4 5
b c 6
is symmetric, find a, b, c?
1 2 3
1 a b
T
A = a 4 5 A = 2 4 c
b c 6
3 5 6
A = AT a = 1, b = 2, c = 3
A = AT
a = 2, b = 3, c = 5
Notation and basic definitions:
Diagonal matrix and trace
If a matrix contains zeros in all off diagonal positions, it
is said to be a diagonal matrix.
d1
0
A = diag (d1 , d 2 ,!, d n ) =
"
0
Trace:
If A = [aij ]nn
Then Tr ( A) = a11 + a22 + ! + ann
0
d2
"
0
0
! 0
M nn
# "
! d n
Notation and basic definitions:
Identity Matrix
Square matrix with ones on the
diagonal and zeros elsewhere.
1
0
I =
0
0 0 0
1 0 0
0 1 0
0 0 1
Notation and basic definitions:
Transpose Matrix
Rows become columns and columns become rows
Notation and basic definitions:
Transpose Matrix
Properties of transposes:
a11 a 21 ,, am1
a12 a 22 ,, am 2
A' =
a1n a 2n ,, amn
(1) ( AT )T = A
(2) ( A + B)T = AT + BT
(3) (cA)T = c( AT )
(4) ( AB)T = BT AT
The transpose of a matrix A, denoted by A, is
obtained from A by interchanging rows and
columns.
Matrix operations
Matrix operations
Scalar multiplication:
Matrix addition:
If A = [aij ]mn , B = [bij ]mn
Then A + B = [aij ]mn + [bij ]mn = [aij + bij ]mn
Ex 2: (Matrix addition)
1 2 1 3 1 + 1 2 + 3 0 5
0 1 + 1 2 = 0 1 1 + 2 = 1 3
1 1
3 + 3 =
2 2
1 1 0
3 + 3 = 0
2 + 2 0
If A = [aij ]mn , c : scalar
Then cA = [ca ij ]mn
Matrix subtraction:
A B = A + (1) B
Ex 3: (Scalar multiplication and matrix subtraction)
1 2 4
A = 3 0 1
2 1 2
0 0
2
B = 1 4 3
3 2
1
Find (a) 3A, (b) B, (c) 3A B
Matrix operations
Matrix operations
Ex 4: (Find AB)
Matrix multiplication:
3
1
A = 4 2
0
5
If A = [aij ]mn , B = [bij ]n p
Then AB = [aij ]mn [bij ]n p = [cij ]m p
Size of AB
where
Sol:
cij = aik bkj = ai1b1 j + ai 2b2 j + ! + ainbnj
k =1
a11 a12 ! a1n b ! b
! b1n
11
1j
"
"
"
b
"
b
!
b
21
2j
2n
=
ai1 ai 2 ! ain
"
"
"
" ci1 ci 2 ! cij ! cin
"
"
"
b ! bnj ! bnn
an1 an 2 ! ann n1
Notes: (1) A+B = B+A, (2)
3 2
B=
4 1
AB BA
(1)(3) + (3)(4) (1)(2) + (3)(1)
AB = (4)(3) + (2)(4) (4)(2) + (2)(1)
(5)(2) + (0)(1)
(5)(3) + (0)(4)
9 1
= 4 6
15 10
Matrix operations
Determinants
Matrix form of a system of linear equations:
a11 x1 + a12 x2 + ! + a1n xn = b1
a x + a x +!+ a x = b
21 1 22 2
2n n
2
"
am1 x1 + am 2 x2 + ! + amn xn = bm
m linear equations
The determinant of an nn matrix A is defined as the sum of
all n! possible products of n elements such that
1. each product contains one element from every row and
every column, and
a11 a12 " a1n x1 b1
a
21 a22 " a2 n x2 = b2
!
!
!
! ! !
am1 am 2 " amn xn bm
=
Single matrix equation
Ax =b
m n n 1
m 1
2. the factors in each product are written so that the column
subscripts appear in order of magnitude and each product
is then preceded by a plus or minus sign according to
whether the number of inversions in the row subscripts is
even or odd.
The Inverse of a Matrix
Determinants
Inverse matrix:
The determinant of A is a scalar denoted by |A| or by
det(A).
Consider
A M nn
If there exists a matrix B M nn such that AB = BA = I n ,
Then (1) A is invertible (or nonsingular)
(2) B is the inverse of A
Note:
A matrix that does not have an inverse is called noninvertible (or
Positive
Negative
singular).
The Inverse of a Matrix
Eigenvalues and eigenvectors
The inverse of a matrix is unique)
If B and C are both inverses of the matrix A, then B = C.
Pf:
AB = I
Let A be an nxn matrix. A scalar is called an eigenvalue
of A if there is a nonzero vector x such that
Ax = x.
C ( AB ) = CI
(CA) B = C
IB = C
B=C
such vector x is called a eigenvector A corresponding to
Consequently, the inverse of a matrix is unique.
Notes:
(1) The inverse of A is denoted by
(2) AA1 = A1 A = I
A1
Eigenvalues and eigenvectors
2
3
Show that x = 1 is an eigenvector of A =
Corresponding to
3 2
Eigenvalues and eigenvectors
When A is squared, the eigenvectors stay the same.
The eigenvalues are squared
=4
Is it true that
?
Ax = x
If is an eigenvalue of A and x is an eigenvector
belonging to , any nonzero multiple of x will be an
eigenvector
Eigenvalues and eigenvectors
to find and x, we use the following equation:
(A I)x = 0
The product of the n eigenvalues equals the determinant.
The sum of the n eigenvalues equals the sum of the n diagonal
entries (trace).
Eigenvalues and eigenvectors
Example:
Find the eigenvalues and eigenvectors of
2 1
A =
1 2
Eigenvalues and eigenvectors
Try next:
Find the eigenvalues and eigenvectors of A2, A-1, and (A+4I)
Quiz
1. Find the eigenvalues of these matrices. All powers have the same
eigenvectors.
2. Find the eigenvalues of A and B and AB and BA
Matrix in R
Matrix in R
Now open your R software!
Matrix operation
Making a Matrix in R
A <matrix(data=c(7,18,-2,22,-16,3,55,1,9,-4,0,31),byrow=
TRUE,nrow=3,ncol=4)
#check the dimensions
dim(A)
Making an Identity Matrix (I) in R
id <- function(n)diag(c(1),nrow=n,ncol=n)
# To create an identity matrix of order 12
I12 <- id(12)
Transposition
At <- t(A)
# t() is the transpose function
Diagonals
diag(A)
Addition/Substraction of matrix
A+A
#addition of matrix A and A
Multiplication of matrices
A%*$At
#matrix At multiplied by A
