Appendix B

Tensor products

It is a fundamental operation in physics to join two physical system in order to form a

composite system. In classical physics you might know that systems are combined using
direct products: The position of an object is described as a point in 3 , and the position of
two objects as a point in 3 ⇥ 3 . In quantum physics we use the tensor product instead of
the direct product.

B.0.1 Formal definition of tensor products

We will define tensor products in a very concrete way by using bilinear forms:
Definition B.1 (Tensor product). The tensor product of complex Euclidean spaces HA and
HB is defined as
⇤ ⇤
HA ⌦ HB = {F : HA ⇥ HB ! bilinear}.
To get a better understanding of tensor products, let us consider two vectors |vA i 2 HA
and |wB i 2 HB . We can define the tensor product of these two vectors by

|vA i ⌦ |wB i = F,

where
F (ha|, hb|) = ha|vA ihb|wB i,
for every ha| 2 HA
⇤ and hb| 2 H⇤ . Tensor products |v i ⌦ |w i are also called elementary
B A B
tensors. We can use elementary tensors to form a basis of HA ⌦ HB :
dim(H ) dim(H )
Theorem B.2. For orthonormal bases {|iA i}i=1 A and {|jB i}j=1 B of HA and HB ,
respectively, the elementary tensors {|iA i⌦|jB i}iA ,jA form a basis of HA ⌦HB . In particular,
we have dim(HA ⌦ HB ) = dim(HA ) · dim(HB ).
dim(H ) dim(H )
Proof. Note that {hiA |}i=1 A and {hjB |}j=1 B are orthonormal bases of HA ⇤ and H⇤ ,
B
respectively. It is clear, that a bilinear form F : HA ⇤ ⇥ H⇤ !
B is uniquely determined
by the values fij = F (hiA |, hjB |) where i 2 {1, . . . , dim(HA )} and j 2 {1, . . . , dim(HB )}.
Specifically, we have
0 1
dim(HA ) dim(HB )
X X
F (ha|, hb|) = F @ ai hiA |, bj hjB |A
i=1 j=1
X
= fij ai bj .
i,j

Since we also have X X

fij ai bj = fij ha|iA ihb|jA i,
i,j i,j

211
APPENDIX B. TENSOR PRODUCTS 212

we can conclude that X

F = fij |iA i ⌦ |jB i.
i,j

This expression also shows that

X
fij |iA i ⌦ |jB i = 0,
i,j

implies that fij = 0 for all i, j. Therefore, we conclude that {|iA i ⌦ |jB i}iA ,jA forms a basis
of HA ⌦ HB .

Using bilinearity, it is easy to verify that

(|ai ⌦ (|b1 i + |b2 i)) = |ai ⌦ |b1 i + |ai ⌦ |b2 i

(|a1 i + |a2 i) ⌦ |bi = |a1 i ⌦ |bi + |a2 i ⌦ |bi (B.1)
( |ai) ⌦ |bi = (|ai ⌦ |bi) = |ai ⌦ ( |bi)

Note that the tensor product is not commutative! The tensor product between vectors and
spaces, may also be iterated and it is associative. To obtain a more compact form of higher
order tensor products, we can also set

H1 ⌦ H2 · · · ⌦ Hn = {F : H1⇤ ⇥ H2⇤ ⇥ · · · ⇥ Hn⇤ ! multilinear}.

The theory can then be developed exactly as in the case n = 2. In particular, we may form
higher order tensor products of vectors, e.g., vectors such as |v1 i ⌦ |v2 i ⌦ · · · ⌦ |vn i.
Sometimes, the tensor product HA ⌦ HB is defined as the (formal) span of elementary
tensors |iA i ⌦ |jB i where {|iA i}i and {|jB i}j are orthonormal bases of HA and HB , respec-
tively, and such that the identities in (B.1) are satisfied. If you feel more comfortable using
this definition, then it is fine if you do so during this course.
Up to now, we have not considered any Euclidean structure on the tensor product HA ⌦
HB . To define an inner product, we can define

ha1 ⌦ b1 |a2 ⌦ b2 i := ha1 |a2 ihb1 |b1 i,

for all elementary tensors formed from |a1 i, |a2 i 2 HA and |b1 i, |b2 i 2 HB . This formula can
then be linearly extended to all tensors in HA ⌦ HB . This turns HA ⌦ HB into a complex
Euclidean space, and we will usually mean this space when writing HA ⌦ HB . The same
construction also works for higher order tensor products such that H1 ⌦ H2 · · · ⌦ Hn is a
complex Euclidean space as well.
How do operators act between tensor products? Given operators X 2 B (HA , HC ) and
Y 2 B (HB , HD ) we define their tensor product as the operator X⌦Y 2 B (HA ⌦ HB , HC ⌦ HD )
acting on elementary tensors as

(X ⌦ Y ) (|xi ⌦ |yi) = X|xi ⌦ Y |yi,

and extended linearly. It should be noted that X ⌦Y can also be understood as the elementary
tensor in B (HA , HC ) ⌦ B(HB , HD ) formed from the operators X 2 B (HA , HC ) and Y 2
B(HB , HD )1 . As an exercise, you can make this identification more explicit and show that

B (HA ⌦ HB , HC ⌦ HD ) ' B (HA , HC ) ⌦ B(HB , HD ). (B.2)

We will often make this identification implicitly without making too much fuss about it. An
example can be seen in the following discussion.
1
Note that
⇥ the ⇤spaces B(H, H0 ) are complex Euclidean spaces using the Hilbert-Schmidt inner product
† †
hX, Y i = Tr X Y , where X denotes the adjoint of X. We will sometimes refer to these spaces as Hilbert-
Schmidt inner product spaces.
APPENDIX B. TENSOR PRODUCTS 213

B.0.2 The Kronecker product

It is instructive to write the tensor product X ⌦ Y of operators X 2 B( dA , dC ) and
Y 2 B( dB , dD ) explicitely in the computational basis. We can think of X ⌦ Y as an
operator from dA ⌦ dB to dC ⌦ dD and therefore, we should be able to represent it as
0 0
some kind of matrix. It is very common to identify d ⌦ d and dd by ordering the basis
{|ii ⌦ |ji}i,j . Of course, there are many possible orderings, but it is common to choose the
lexicographic ordering such that (i, j) comes before (i0 , j 0 ) if i < i0 or if i = i0 and j < j 0 . If we
do this for d = d0 = 2 we could for instance construct a basis {|1i, . . . , |4i} of 4 ' 2 ⌦ 2
by setting

|1i = |1i ⌦ |1i

|2i = |1i ⌦ |2i
|3i = |2i ⌦ |1i
|4i = |2i ⌦ |2i.

Consider now matrices X = [xij ]ij and Y = [ykl ]kl and note that
0 1 !
X X
X ⌦Y =@ xij |iihj|A ⌦ ykl |kihl|
ij kl
X
= xij ykl |iihj| ⌦ |kihl|
ijkl
X
= xij ykl |i ⌦ kihj ⌦ l|,
ijkl

where we used (B.2) implicitly in the last line. Writing this operator as a matrix (where we
order tensor product bases lexicographical) shows that for X = [xij ]ij and Y = [xkl ]kl we
have the matrix representation
0 1
0 1 x11 Y x12 Y · · · x1dA Y
x11 y11 x11 y12 · · · B
Bx11 y21 x11 y22 C B x21 Y x22 Y · · · x2dA Y C C
X ⌦Y =@ A = B .. .. .. C.
.. .. @ . . . A
. .
xdC 1 Y x dC 2 Y · · · xdC dA Y

This matrix operation is also called the Kronecker product. As described above, this is simply
the tensor product in the computational basis using a particular convention of how to order
the product basis. With this expression, it is easy to check

Tr [X ⌦ Y ] = Tr [X] Tr [Y ] ,
spec (X ⌦ Y ) = { µ : 2 spec (X) , µ 2 spec (Y )},

for any X 2 B(HA ) and Y 2 B(HB ).

B.1 Vectorization and Schmidt decomposition

To study tensor products of complex Euclidean spaces, it is surprisingly useful to note the
identification
B(HA , HB ) ' HA ⌦ HB .
This identification can be realized by the isomorphism vec : B(HA , HB ) ! HA ⌦ HB defined
as
vec (| ih |) = | i ⌦ | i,
APPENDIX B. TENSOR PRODUCTS 214

for any | i 2 HA and | i 2 HB , and extended linearly. Here, the conjugation on the right-
hand side is applied to the entries of the vector | i with respect to the computational basis
of HA . This makes the isomorphism vec dependent on a choice of basis, but on the upside
it is linear. In the following, we will call this isomorphism the vectorization and we refer to
vec(X) as the vectorization of the operator X 2 B(HA , HB ).
The vectorization does not only define an isomorphism between the vector spaces B(HA , HB )
and HA ⌦ HB , but this isomorphism also preserves the Euclidean structure of both spaces,
where we view B(HA , HB ) as a complex Euclidean space with the Hilbert-Schmidt inner
product h i
hX, Y iHS = Tr X † Y ,

for operators X, Y 2 B(HA , HB ) and where the trace is on B(HA ). We have the following:

Theorem B.3. For any pair of operators X, Y 2 B(HA , HB ), we have

hvec(X), vec(Y )i = hX, Y iHS .

In particular, vec is an isometric isomorphism between the spaces B(HA , HB ) and HA ⌦ HB .

Proof. By linearity it is sufficient to prove the identity on rank-1 operators X = |b1 iha1 | and
Y = |b2 iha2 | with |a1 i, |a2 i 2 HA and |b1 i, |b2 i 2 HB . We can compute that

hvec(|b1 iha1 |), vec(|b2 iha2 |)i = ha1 ⌦ b1 , a2 ⌦ b2 i = ha1 |a2 ihb1 |b2 i = ha2 |a1 ihb1 |b2 i.

This expression coincides with

h i
Tr (|b1 iha1 |)† |b2 iha2 | = Tr [|a1 ihb1 | · |b2 iha2 |] = ha2 |a1 ihb1 |b2 i,

using cyclicity of the trace.

It is helpful to compute the vectorization explicitly in the computational basis for HA =

dA and HB = dB . Here, we have

vec (|iB ihjA |) = |jA i ⌦ |iB i,

for any i 2 {1, . . . , dB } and any j 2 {1, . . . , dA }. Using the lexicographic ordering of the
tensor product basis, we have 0 1
x11
B x21 C
B C
B .. C
B . C
B C
Bx d 1 C
B B C
vec (X) = B x C ,
B 12 C
B . C
B .. C
B C
Bx d 2 C
@ B A
..
.
for any matrix X = [xij ]i,j , i.e., the vectorization operation stacks the rows of X on top of
each other. In Matlab, this would correspond to using the command “reshape” transforming
the matrix X to a column vector.
Using the vectorization operation, we can sometimes transfer properties of operators to
vectors on a tensor product and vice-versa. For example, we can transfer the singular value
decomposition and the notion of rank to vectors, which is known as the Schmidt decomposition
and the Schmidt rank :
APPENDIX B. TENSOR PRODUCTS 215

Theorem B.4 (Schmidt decomposition). Consider Euclidean spaces HA and HB . For any
dim(H ) dim(H )
vector |xAB i 2 HA ⌦ HB there exist orthonormal bases {|ai i}i=1 A and {|bi i}j=1 B of HA
and HB , respectively, such that
min(dA ,dB )
X
|xAB i = i |ai i ⌦ |bi i,
i=1

for numbers i 2 + called the Schmidt coefficients of |xAB i. The number of non-zero
Schmidt coefficients is called the Schmidt rank of |xAB i.

Proof. Define an operator XA!B 2 B(HA , HB ) by

1
XA!B = vec (|vAB i) .

Using the singular value decomposition, we can decompose

min(dA ,dB )
X
0
XA!B = i |bi ihai |,
i=1

dim(HA ) dim(HB )
for singular values i 2 + and orthonormal bases {|ai i}i=1 and {|b0j i}j=1 of HA
and HB , respectively. Finally, we vectorize again to obtain
min(dA ,dB )
X
|xAB i = vec (XA!B ) = i |ai i ⌦ |b0i i.
i=1

The theorem follows by setting |b0i i = |bi i.

The proof of the theorem shows that the singular values of an operator X 2 B(HA , HB )
are the Schmidt coefficients of its vectorization vec(X) 2 HA ⌦ HB . Like the singular values
of an operator, the Schmidt coefficients and Schmidt rank of a vector are unique. We will
denote the Schmidt rank of a vector |xAB i by SR (|xAB i) and we emphasize again that

rk (X) = SR (vec(X)) ,

for any operator X 2 B(HA , HB ).