M Array Manipulation Tips and Tricks: Atlab
M Array Manipulation Tips and Tricks: Atlab
Peter J. Acklam
Statistics Division
Department of Mathematics
University of Oslo
Norway
E-mail: [email protected]
WWW URL: http://www.math.uio.no/~jacklam/
5 May 2000
1
Abstract
This document is intended to be a compilation tips and tricks mainly related to effi-
cient ways of performing low-level array manipulation in M ATLAB. Here, “manipulate”
means replicating and rotating arrays or parts of arrays, inserting, extracting, permut-
ing and shifting elements, generating combinations and permutations of elements, run-
length encoding and decoding, multiplying and dividing arrays and calculating distance
matrics and so forth. A few other issues regarding how to write fast M ATLAB code is
also covered.
This document was produced with -LATEX.
The PS (PostScript) version was created with dvips by Tomas Rokicki.
The PDF (Portable Document Format) version was created with ps2pdf, a part of Aladdin Ghost-
script by Aladdin Enterprises.
The PS and PDF version may be viewed with software available at the Ghostscript, Ghostview and
GSview Home Page at http://www.cs.wisc.edu/~ghost/index.html.
The PDF version may also be viewed with Adobe Acrobat Reader available at
http://www.adobe.com/products/acrobat/readstep.html.
Contents
1 Introduction 3
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Vectorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 About the examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Credit where credit is due . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Errors/Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
4 Shifting 6
4.1 Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.2 Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
6 Reshaping arrays 7
6.1 Subdividing 2D matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
6.1.1 Create 4D array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
6.1.2 Create 3D array (columns first) . . . . . . . . . . . . . . . . . . . . . . . . . 8
6.1.3 Create 3D array (rows first) . . . . . . . . . . . . . . . . . . . . . . . . . . 8
6.1.4 Create 2D matrix (columns first, column output) . . . . . . . . . . . . . . . 9
6.1.5 Create 2D matrix (columns first, row output) . . . . . . . . . . . . . . . . . 9
6.1.6 Create 2D matrix (rows first, column output) . . . . . . . . . . . . . . . . . 10
6.1.7 Create 2D matrix (rows first, row output) . . . . . . . . . . . . . . . . . . . 10
8 Multiply arrays 20
8.1 Multiply each 2D slice with the same matrix (element-by-element) . . . . . . . . . . 20
8.2 Multiply each 2D slice with the same matrix (left) . . . . . . . . . . . . . . . . . . . 20
8.3 Multiply each 2D slice with the same matrix (right) . . . . . . . . . . . . . . . . . . 20
8.4 Multiply matrix with every element of a vector . . . . . . . . . . . . . . . . . . . . 21
8.5 Multiply each 2D slice with corresponding element of a vector . . . . . . . . . . . . 21
8.6 Outer product of all rows in a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 21
8.7 Keeping only diagonal elements of multiplication . . . . . . . . . . . . . . . . . . . 22
9 Divide arrays 22
9.1 Divide each 2D slice with the same matrix (element-by-element) . . . . . . . . . . . 22
9.2 Divide each 2D slice with the same matrix (left) . . . . . . . . . . . . . . . . . . . . 22
9.3 Divide each 2D slice with the same matrix (right) . . . . . . . . . . . . . . . . . . . 22
10 Calculating distances 23
10.1 Euclidean distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
10.2 Distance between two points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
10.3 Euclidean distance vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
10.4 Euclidean distance matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
10.5 Special case when both matrices are identical . . . . . . . . . . . . . . . . . . . . . 24
10.6 Mahalanobis distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
12 Miscellaneous 27
12.1 Creating index vector from index limits . . . . . . . . . . . . . . . . . . . . . . . . 27
12.2 Matrix with different incremental runs . . . . . . . . . . . . . . . . . . . . . . . . . 28
12.3 Finding indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
12.4 Run-length encoding and decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
12.4.1 Run-length encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
12.4.2 Run-length decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
1 Introduction
1.1 Background
Since the early 1990’s I have been following the discussions in the main M ATLAB newsgroup on
Usenet, comp.soft-sys.matlab. I realized that many postings there were about how to ma-
2 OPERATORS, FUNCTIONS AND SPECIAL CHARACTERS 4
nipulate arrays efficiently. I decided to start collecting what I thought was the most interestings
solutions and see if I could compile them into one document. Well, this is it.
1.2 Vectorization
The term “vectorization” is frequently associated with M ATLAB. Strictly speaking, it means to
rewrite code so that, in stead of using a for-loop iterating over each scalar in an array, one takes
advantage of M ATLAB’s vectorization capabilities and does everything in one go. For instance, the
5 lines
x = [ 1 2 3 4 5 ];
y = zeros(size(x));
for i = 1:5
y(i) = x(i)^2;
end
which is faster, most compact, and easier to read. With this rather strict definition of “vectorization”,
vectorized code is always faster than non-vectorized code.
Some people use the term “vectorization” in the sense “removing any for-loop”, but I will stick
to the former, more strict definition.
1.5 Errors/Feedback
If you find errors or have suggestions for improvements or if there is anything you think should be
here but is not, please mail me and I will see what I can do. My address is on the front page of this
document.
at the command prompt and take a look at the list of operators, functions and special characters, and
look at the associated help pages.
When manipulating arrays in M ATLAB there are some operators and functions that are particu-
larely useful.
2.1 Operators
: The colon operator.
Type help colon for more information.
.’ Non-conjugate transpose.
Type help transpose for more information.
’ Complex conjugate transpose.
Type help ctranspose for more information.
If step is not a multiple of the difference upper-lower, the last element of X, X(end), will be
less than upper. So the condition A(end) <= upper is always satisfied.
4 Shifting
4.1 Vectors
To shift and rotate the elements of a vector, use
X([ end 1:end-1 ]); % shift right/down 1 element
X([ end-k+1:end 1:end-k ]); % shift right/down k elements
X([ 2:end 1 ]); % shift left/up 1 element
X([ k+1:end 1:k ]); % shift left/up k elements
Note that these only work if k is non-negative. If k is an arbitrary integer one may use something
like
X( mod((1:end)-k-1, end)+1 ); % shift right/down k elements
X( mod((1:end)+k-1, end)+1 ); % shift left/up k element
4.2 Arrays
To shift and rotate the elements of an array X along dimension dim, first initialize a subscript cell
array with
idx = repmat({’:’}, ndims(X), 1); % initialize subscripts
n = size(X, dim); % length along dimension dim
The repmat solution might in some cases be slighly slower, but it has several advantages. Firstly,
it uses less memory. Seconly, it also works if val is a function returning a scalar value, e.g., if val
is Inf or NaN:
X = NaN(ones(siz)); % won’t work unless NaN is a variable
X = repmat(NaN, siz); % this works
Avoid using
X = val * ones(siz);
since it does unnecessary multiplications and only works if val is of class “double”.
If A is a column-vector, use
B = A(:,ones(1,N)).’;
B = B(:);
but this requires unnecessary arithmetic. The only advantage is that it works regardless of whether
A is a row or column vector.
6 Reshaping arrays
6.1 Subdividing 2D matrix
Assume X is an m-by-n matrix.
Now,
6 RESHAPING ARRAYS 8
into
Y = cat( 3, A, C, B, D );
use
Y = reshape( X, [ p m/p q n/q ] );
Y = permute( Y, [ 1 3 2 4 ] );
Y = reshape( Y, [ p q m*n/(p*q) ] )
Now,
X = [ Y(:,:,1) Y(:,:,m/p+1) ... Y(:,:,(n/q-1)*m/p+1)
Y(:,:,2) Y(:,:,m/p+2) ... Y(:,:,(n/q-1)*m/p+2)
... ... ... ...
Y(:,:,m/p) Y(:,:,2*m/p) ... Y(:,:,n/q*m/p) ];
into
Y = cat( 3, A, B, C, D );
use
Y = reshape( X, [ p m/p n ] );
Y = permute( Y, [ 1 3 2 ] );
Y = reshape( Y, [ p q m*n/(p*q) ] );
6 RESHAPING ARRAYS 9
Now,
X = [ Y(:,:,1) Y(:,:,2) ... Y(:,:,n/q)
Y(:,:,n/q+1) Y(:,:,n/q+2) ... Y(:,:,2*n/q)
... ... ... ...
Y(:,:,(m/p-1)*n/q+1) Y(:,:,(m/p-1)*n/q+2) ... Y(:,:,m/p*n/q) ];
To restore X from Y use
X = reshape( Y, [ p n m/p ] );
X = permute( X, [ 1 3 2 ] );
X = reshape( X, [ m n ] );
into
Y = [ A
B
C
D ];
use
Y = reshape( X, [ p m/p q n/q ] );
Y = permute( Y, [ 1 4 2 3 ] );
Y = reshape( Y, [ m*n/q q ] );
into
Y = [ A B C D ];
use
Y = reshape( X, [ p m/p n ] );
Y = permute( Y, [ 1 3 2 ] );
Y = reshape( Y, [ p m*n/p ] );
Rotating 90 clockwise
s = size(X); % size vector
v = [ 2 1 3:ndims(X) ]; % dimension permutation vector
Y = reshape( X(s(1):-1:1,:), s );
Y = permute( Y, v );
If we want to rotate n*90 degrees counterclockwise, we may merge the three cases above into
% Largest dimension number we have to deal with.
nd = max( [ ndims(A) dim1 dim2 ] );
Inner block rotation is a rotation of the elements within each block, preserving the position of each
block within the array. Outer block rotation rotates the blocks but does not change the position of
the elements within each block.
An example will illustrate: An inner block rotation 90 degrees counterclockwise will have the
following effect
[ A B C [ rot90(A) rot90(B) rot90(C)
D E F => rot90(D) rot90(E) rot90(F)
G H I ] rot90(G) rot90(H) rot90(I) ]
However, an outer block rotation 90 degrees counterclockwise will have the following effect
[ A B C [ C F I
D E F => B E H
G H I ] A D G ]
In all the examples below, it is assumed that X is an m-by-n matrix of p-by-q blocks.
7 ROTATING MATRICES AND ARRAYS 14
use
Y = reshape( X, [ p m/p q n/q ] );
Y = Y(:,:,q:-1:1,:); % or Y = Y(:,:,end:-1:1,:);
Y = permute( Y, [ 3 2 1 4 ] );
Y = reshape( Y, [ q*m/p p*n/q ] );
use
Y = reshape( X, [ p q n/q ] );
Y = Y(:,q:-1:1,:); % or Y = Y(:,end:-1:1,:);
Y = permute( Y, [ 2 1 3 ] );
Y = reshape( Y, [ q m*n/q ] ); % or Y = Y(:,:);
use
Y = X(:,q:-1:1); % or Y = X(:,end:-1:1);
Y = reshape( Y, [ p m/p q ] );
Y = permute( Y, [ 3 2 1 ] );
Y = reshape( Y, [ q*m/p p ] );
7 ROTATING MATRICES AND ARRAYS 15
use
Y = reshape( X, [ p m/p q n/q ] );
Y = Y(p:-1:1,:,q:-1:1,:); % or Y = Y(end:-1:1,:,end:-1:1,:);
Y = reshape( Y, [ m n ] );
use
Y = reshape( X, [ p q n/q ] );
Y = Y(p:-1:1,q:-1:1,:); % or Y = Y(end:-1:1,end:-1:1,:);
Y = reshape( Y, [ m n ] ); % or Y = Y(:,:);
use
Y = reshape( X, [ p m/p q ] );
Y = Y(p:-1:1,:,q:-1:1); % or Y = Y(end:-1:1,:,end:-1:1);
Y = reshape( Y, [ m n ] );
7 ROTATING MATRICES AND ARRAYS 16
use
Y = reshape( X, [ p m/p q n/q ] );
Y = Y(p:-1:1,:,:,:); % or Y = Y(end:-1:1,:,:,:);
Y = permute( Y, [ 3 2 1 4 ] );
Y = reshape( Y, [ q*m/p p*n/q ] );
use
Y = X(p:-1:1,:); % or Y = X(end:-1:1,:);
Y = reshape( Y, [ p q n/q ] );
Y = permute( Y, [ 2 1 3 ] );
Y = reshape( Y, [ q m*n/q ] ); % or Y = Y(:,:);
use
Y = reshape( X, [ p m/p q ] );
Y = Y(p:-1:1,:,:); % or Y = Y(end:-1:1,:,:);
Y = permute( Y, [ 3 2 1 ] );
Y = reshape( Y, [ q*m/p p ] );
7 ROTATING MATRICES AND ARRAYS 17
use
Y = reshape( X, [ p m/p q n/q ] );
Y = Y(:,:,:,n/q:-1:1); % or Y = Y(:,:,:,end:-1:1);
Y = permute( Y, [ 1 4 3 2 ] );
Y = reshape( Y, [ p*n/q q*m/p ] );
use
Y = reshape( X, [ p q n/q ] );
Y = Y(:,:,n/q:-1:1); % or Y = Y(:,:,end:-1:1);
Y = permute( Y, [ 1 3 2 ] );
Y = reshape( Y, [ m*n/q q ] );
use
Y = reshape( X, [ p m/p q ] );
Y = permute( Y, [ 1 3 2 ] );
Y = reshape( Y, [ p n*m/p ] ); % or Y(:,:);
7 ROTATING MATRICES AND ARRAYS 18
use
Y = reshape( X, [ p m/p q n/q ] );
Y = Y(:,m/p:-1:1,:,n/q:-1:1); % or Y = Y(:,end:-1:1,:,end:-1:1);
Y = reshape( Y, [ m n ] );
use
Y = reshape( X, [ p q n/q ] );
Y = Y(:,:,n/q:-1:1); % or Y = Y(:,:,end:-1:1);
Y = reshape( Y, [ m n ] ); % or Y = Y(:,:);
use
Y = reshape( X, [ p m/p q ] );
Y = Y(:,m/p:-1:1,:); % or Y = Y(:,end:-1:1,:);
Y = reshape( Y, [ m n ] );
7 ROTATING MATRICES AND ARRAYS 19
use
Y = reshape( X, [ p m/p q n/q ] );
Y = Y(:,m/p:-1:1,:,:); % or Y = Y(:,end:-1:1,:,:);
Y = permute( Y, [ 1 4 3 2 ] );
Y = reshape( Y, [ p*n/q q*m/p ] );
use
Y = reshape( X, [ p q n/q ] );
Y = permute( Y, [ 1 3 2 ] );
Y = reshape( Y, [ m*n/q q ] );
use
Y = reshape( X, [ p m/p q ] );
Y = Y(:,m/p:-1:1,:); % or Y = Y(:,end:-1:1,:);
Y = permute( Y, [ 1 3 2 ] );
Y = reshape( Y, [ p n*m/p ] );
use
Y = reshape( X, [ p m/p q n/q ] );
Y = permute( Y, [ 3 2 1 4 ] );
Y = reshape( Y, [ q*m/p p*n/q ] );
8 MULTIPLY ARRAYS 20
use
Y = reshape( X, [ p m/p q n/q ] );
Y = permute( Y, [ 1 4 3 2 ] );
Y = reshape( Y, [ p*n/q q*m/p] );
8 Multiply arrays
8.1 Multiply each 2D slice with the same matrix (element-by-element)
Assume X is an m-by-n-by-p-by-q-by-. . . array and Y is an m-by-n matrix and you want to construct
a new m-by-n-by-p-by-q-by-. . . array Z, where
Z(:,:,i,j,...) = X(:,:,i,j,...) .* Y;
for all i=1,...,p, j=1,...,q, etc. This can be done with nested for-loops, or by the following
vectorized code
sx = size(X);
Z = X .* repmat(Y, [1 1 sx(3:end)]);
for all i=1,...,p, j=1,...,q, etc. This can be done with nested for-loops, or by the following
vectorized code
sx = size(X);
sy = size(Y);
Z = reshape(Y * X(:,:), [sy(1) sx(2:end)]);
for all i=1,...,p, j=1,...,q, etc. This can be done with nested for-loops, or by the following
vectorized code
8 MULTIPLY ARRAYS 21
sx = size(X);
sy = size(Y);
dx = ndims(X);
Xt = reshape(permute(X, [1 3:dx 2]), [prod(sx)/sx(2) sx(2)]);
Z2 = Xt * Y;
Z2 = permute(reshape(Z2, [sx([1 3:dx]) sy(2)]), [1 dx 2:dx-1]);
The third line above builds a 2D matrix which is a vertical concatenation (stacking) of all 2D slices
X(:,:,i,j,...). The fourth line does the actual multiplication. The fifth line does the opposite
of the third line.
Solution (1) does a lot of unnecessary work, since we only keep the n diagonal elements of the nˆ2
computed elements. Solution (2) only computes the elements of interest and is significantly faster if
n is large.
9 Divide arrays
9.1 Divide each 2D slice with the same matrix (element-by-element)
Assume X is an m-by-n-by-p-by-q-by-. . . array and Y is an m-by-n matrix and you want to construct
a new m-by-n-by-p-by-q-by-. . . array Z, where
Z(:,:,i,j,...) = X(:,:,i,j,...) ./ Y;
for all i=1,...,p, j=1,...,q, etc. This can be done with nested for-loops, or by the following
vectorized code
sx = size(X);
Z = X./repmat(Y, [1 1 sx(3:end)]);
for all i=1,...,p, j=1,...,q, etc. This can be done with nested for-loops, or by the following
vectorized code
Z = reshape(Y\X(:,:), size(X));
for all i=1,...,p, j=1,...,q, etc. This can be done with nested for-loops, or by the following
vectorized code
10 CALCULATING DISTANCES 23
sx = size(X);
dx = ndims(X);
Xt = reshape(permute(X, [1 3:dx 2]), [prod(sx)/sx(2) sx(2)]);
Z = Xt/Y;
Z = permute(reshape(Z, sx([1 3:dx 2])), [1 dx 2:dx-1]);
The third line above builds a 2D matrix which is a vertical concatenation (stacking) of all 2D slices
X(:,:,i,j,...). The fourth line does the actual division. The fifth line does the opposite of the
third line.
The five lines above might be simplified a little by introducing a dimension permutation vector
sx = size(X);
dx = ndims(X);
v = [1 3:dx 2];
Xt = reshape(permute(X, v), [prod(sx)/sx(2) sx(2)]);
Z = Xt/Y;
Z = ipermute(reshape(Z, sx(v)), v);
If you don’t care about readability, this code may also be written as
sx = size(X);
dx = ndims(X);
v = [1 3:dx 2];
Z = ipermute(reshape(reshape(permute(X, v), ...
[prod(sx)/sx(2) sx(2)])/Y, sx(v)), v);
10 Calculating distances
10.1 Euclidean distance
The Euclidean distance from xi to y j is
di j xi y j xi y j
x1i y1 j
2
x pi y p j
2
where
nx
1 n 1
nx i∑ nx 1 i∑
x¯ xi and Cx xi x¯
xi x¯
1
1
Assume Y is an ny-by-p matrix containing a set of vectors and X is an nx-by-p matrix containing
another set of vectors, then the Mahalanobis distance from each vector Y(j,:) (for j=1,...,ny)
to the set of vectors in X can be calculated with
nx = size(X, 1); % size of set in X
ny = size(Y, 1); % size of set in Y
m = mean(X);
C = cov(X);
d = zeros(ny, 1);
for j = 1:ny
d(j) = (Y(j,:) - m) / C * (Y(j,:) - m)’;
end
11 STATISTICS, PROBABILITY AND COMBINATORICS 25
which is computed more efficiently with the following code which does some inlining of functions
(mean and cov) and vectorization
nx = size(X, 1); % size of set in X
ny = size(Y, 1); % size of set in Y
Special case when both matrices are identical If Y and X are identical in the code above, the
code may be simplified somewhat. The for-loop solution becomes
n = size(X, 1); % size of set in X
m = mean(X);
C = cov(X);
d = zeros(n, 1);
for j = 1:n
d(j) = (Y(j,:) - m) / C * (Y(j,:) - m)’;
end
which is computed more efficiently with
n = size(x, 1);
m = sum(x, 1)/n; % centroid (mean)
c = x - m(ones(n,1),:); % distance to centroid of X
C = (c’ * c)/(n - 1); % variance matrix
d = sum(c/C.*c, 2); % Mahalanobis distances
again, to make it work in the complex case, the last line must be written as
d = real(sum(c/C.*conj(c), 2)); % Mahalanobis distances
Note that the number of times through the loop depends on the number of probabilities and not the
sample size, so it should be quite fast even for large samples.
11.4 Combinations
“Combinations” is what you get when you pick k elements, without replacement, from a sample of
size n, and consider the order of the elements to be irrelevant.
n n! Γ n 1
k
k! n k
! Γ k 1
Γ n k 1
11.5 Permutations
11.5.1 Counting permutations
p = prod(n-k+1:n);
12 Miscellaneous
This section contains things that don’t fit anywhere else.
which unfortunately requires a lot of memory copying since a new x has to be allocated each time
through the loop. A better for-loop solution is one that allocates the required space and then fills in
the elements afterwards. This for-loop solution above may be several times faster than the first one
m = length(lo); % length of input vectors
d = hi - lo + 1; % length of each "run"
n = sum(d); % length of output vector
c = cumsum(d); % last index in each run
for i = 1:m
x(c(i)-d(i)+1:c(i)) = lo(i):hi(i);
end
Neither of the for-loop solutions above can compete with the the solution below which has no for-
loops. It uses cumsum rather than the : to do the incrementing in each run and may be many times
faster than the for-loop solutions above.
m = length(lo); % length of input vectors
d = hi - lo + 1; % length of each "run"
n = sum(d); % length of output vector
x = ones(1, n);
x(1) = lo(1);
x(1+cumsum(d(1:end-1))) = lo(2:m)-hi(1:m-1);
x = cumsum(x);
If fails, however, if lo(i)>hi(i) for any i. Such a case will create an empty vector anyway, so
the problem can be solved by a simple pre-processing step which removing the elements for which
lo(i)>hi(i)
i = lo <= hi;
lo = lo(i);
hi = hi(i);
There also exists a one-line solution which is clearly compact, but not as fast as the no-for-loop
solution above
x = eval([’[’ sprintf(’%d:%d,’, [lo ; hi]) ’]’]);
How does one create the matrix where the ith column contains the vector 1:a(i) possibly padded
with zeros:
b = [ 1 1 1
2 2 2
3 0 3
0 0 4 ];
or
m = max(a);
aa = a(:);
aa = aa(:,ones(m, 1));
bb = 1:m;
bb = bb(ones(length(a), 1),:);
b = bb .* (bb <= aa);
or
m = size(x, 1);
j = zeros(m, 1);
for i = 1:m
k = [ 0 find(x(i,:) ~= 0) ];
j(i) = k(end);
end
12 MISCELLANEOUS 30
To find the index of the last non-zero element in each column, use
i = sum(cumsum((x(end:-1:1,:) ~= 0), 1) ~= 0, 1);