Partition Matrix

A Partition Matrix refers to the process of dividing a matrix into smaller, non-overlapping submatrices or blocks. This concept is pivotal in linear algebra, numerical analysis, and computational science to simplify matrix operations, facilitate parallel computations, and improve algorithm efficiency.

Formally, let A be a matrix of size m × n. Partitioning A involves splitting it into k smaller submatrices A_ij​, such that:

A = \begin{bmatrix} A_{11} & A_{12} & \cdots & A_{1k} \\ A_{21} & A_{22} & \cdots & A_{2k} \\ \vdots & \vdots & \ddots & \vdots \\ A_{k1} & A_{k2} & \cdots & A_{kk} \end{bmatrix}

Here, each A_ij is a submatrix, and the original matrix can be reconstructed by combining these blocks.

Example of Matrix Partition

Consider a 4 × 4 matrix A:

A = \begin{bmatrix} 1 & 2 & 3 & 4 \\ 5 & 6 & 7 & 8 \\ 9 & 10 & 11 & 12 \\ 13 & 14 & 15 & 16 \end{bmatrix}

We can partition A into four 2 × 2 submatrices as:

A = \begin{bmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{bmatrix}

Where:

A_{11} = \begin{bmatrix} 1 & 2 \\ 5 & 6 \end{bmatrix}, \quad A_{12} = \begin{bmatrix} 3 & 4 \\ 7 & 8 \end{bmatrix}, \quad A_{21} = \begin{bmatrix} 9 & 10 \\ 13 & 14 \end{bmatrix}, \quad A_{22} = \begin{bmatrix} 11 & 12 \\ 15 & 16 \end{bmatrix}.

Properties of Partitioned Matrices

Some of the common properties of partitioned matrices are discussed below:

Addition and Subtraction

If two matrices A and B are partitioned conformably (i.e., they have the same partition structure), their addition and subtraction follow element-wise operations on the corresponding blocks.

A + B = \begin{bmatrix} A_{11} + B_{11} & A_{12} + B_{12} \\ A_{21} + B_{21} & A_{22} + B_{22} \end{bmatrix}

Similarly:

A - B = \begin{bmatrix} A_{11} - B_{11} & A_{12} - B_{12} \\ A_{21} - B_{21} & A_{22} - B_{22} \end{bmatrix}

Multiplication

Block multiplication of partitioned matrices follows the same rules as standard matrix multiplication but applied at the block level.

If A and B are partitioned as:

A = \begin{bmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{bmatrix}, \quad B = \begin{bmatrix} B_{11} & B_{12} \\ B_{21} & B_{22} \end{bmatrix},

then their product is:

A \cdot B = \begin{bmatrix} A_{11}B_{11} + A_{12}B_{21} & A_{11}B_{12} + A_{12}B_{22} \\ A_{21}B_{11} + A_{22}B_{21} & A_{21}B_{12} + A_{22}B_{22} \end{bmatrix}.

This property is crucial for algorithms like Strassen's matrix multiplication.

Transpose

The transpose of a partitioned matrix is obtained by transposing each block and swapping its position.

If:

A = \begin{bmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{bmatrix},

then:

A^T = \begin{bmatrix} A_{11}^T & A_{21}^T \\ A_{12}^T & A_{22}^T \end{bmatrix}.

Inverse

For certain 2 × 2 block matrices, the inverse can be computed using block inversion formulas.

If A is partitioned as:

A = \begin{bmatrix} P & Q \\ R & S \end{bmatrix},

and if P, S, and their Schur complements are invertible, the inverse of A is:

A^{-1} = \begin{bmatrix} (P - Q S^{-1} R)^{-1} & -(P - Q S^{-1} R)^{-1} Q S^{-1} \\ -S^{-1} R (P - Q S^{-1} R)^{-1} & S^{-1} + S^{-1} R (P - Q S^{-1} R)^{-1} Q S^{-1} \end{bmatrix}

Determinant

For 2 × 2 block matrices, the determinant can sometimes be expressed in terms of its blocks if certain conditions hold (e.g., invertibility of submatrices).

If A is partitioned as:

A = \begin{bmatrix} P & Q \\ R & S \end{bmatrix}

then: \det(A) = \det(P) \cdot \det(S - R P^{-1} Q),

assuming P is invertible. This is known as the Schur complement determinant formula.

Trace

The trace of a partitioned matrix is the sum of the traces of its diagonal blocks.

If A = \begin{bmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{bmatrix}, then \text{trace}(A) = \text{trace}(A_{11}) + \text{trace}(A_{22}).

Rank

The rank of a partitioned matrix is generally not the sum of the ranks of its individual blocks but depends on the structure and linear dependence of the blocks.

Types of Partitioning

Some common types of partitioning includes:

• Row and Column Partitioning
• Block Partitioning
• Recursive Partitioning

Row Partitioning

The matrix is split into smaller matrices based on its rows. Each submatrix contains a contiguous block of rows.

• Use Case: Frequently used in parallel processing, where each processor operates on a set of rows.

Example: Consider matrix A:

A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix}

Partitioned into rows:

A_1 = \begin{bmatrix} 1 & 2 & 3 \end{bmatrix}, \quad A_2 = \begin{bmatrix} 4 & 5 & 6 \end{bmatrix}, \quad A_3 = \begin{bmatrix} 7 & 8 & 9 \end{bmatrix}

Column Partitioning

The matrix is divided into smaller submatrices based on its columns. Each submatrix contains a contiguous block of columns.

• Use Case: Useful when dealing with column-wise operations, such as column reduction or feature engineering in machine learning.

Example: Using the same A:

A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix}

Partitioned into columns:

A_1 = \begin{bmatrix} 1 \\ 4 \\ 7 \end{bmatrix}, \quad A_2 = \begin{bmatrix} 2 \\ 5 \\ 8 \end{bmatrix}, \quad A_3 = \begin{bmatrix} 3 \\ 6 \\ 9 \end{bmatrix}

Block Partitioning

The matrix is divided into rectangular or square blocks (submatrices) by splitting both rows and columns.

• Use Case: Block partitioning is common in numerical linear algebra, matrix multiplication, and distributed computing.

Example: Consider A:

A = \begin{bmatrix} 1 & 2 & 3 & 4 \\ 5 & 6 & 7 & 8 \\ 9 & 10 & 11 & 12 \\ 13 & 14 & 15 & 16 \end{bmatrix}

Partitioned into 2 × 2 blocks:

A = \begin{bmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{bmatrix}

Where:

A_{11} = \begin{bmatrix} 1 & 2 \\ 5 & 6 \end{bmatrix}, \quad A_{12} = \begin{bmatrix} 3 & 4 \\ 7 & 8 \end{bmatrix}, \quad A_{21} = \begin{bmatrix} 9 & 10 \\ 13 & 14 \end{bmatrix}, \quad A_{22} = \begin{bmatrix} 11 & 12 \\ 15 & 16 \end{bmatrix}

Recursive Partitioning

The matrix is repeatedly partitioned into smaller blocks, often in a hierarchical manner.

Consider a 16 by 16 matrix, then

• Step 1: Partition into two 4 × 4 blocks.
• Step 2: Further partition each 4 × 4 block into 2 × 2 blocks.

Read More,

• Matrices
• Types of Matrices
• Projection Matrices