How to Perform in-place Operations in PyTorch?

In this article, we will see different in-place operations performed on tensors in PyTorch.

Inplace operations are used to directly alter the values of a tensor. The data collected from the user will not be copied. The fundamental benefit of adopting these procedures is that they reduce memory storage while increasing computation, making it easier to deal with high-dimensional data.

In this article, we will see how to do some of the arithmetic operations (addition, subtraction, and multiplication) with and without in-place operations.

Syntax for Addition

Normal Addition: t1.add(t2)
In-place Addition: t1.add_(t2)

Syntax for Subtraction

Normal Subtraction: t1.sub(t2)
In-place Subtraction: t1.sub_(t2)

Syntax for Subtraction

Normal Multiplication: t1.mul(t2)
In-place Multiplication: t1.mul_(t2)

Example 1:

In this example, we are creating two tensors a and b that hold a single value and perform all Addition operations, and then we will display the content of a tensor using the item() method.

# import required library
import torch

# create two tensors a and b with
# single value
a = torch.tensor(2)
b = torch.tensor(6)

print("Addition")
# Normal addition
a.add(b)

# display content in a
print("Normal addition : ", a.item())

# In-place addition
a.add_(b)

# display content in a
print("In-place addition : ", a.item())

Output:

Addition
Normal addition :  2
In-place addition :  8

Example 2:

In this example, we are creating two tensors a and b that hold a single value and perform all subtraction operations, and then we will display the content of a tensor using the item() method.

# import required library
import torch

# create two tensors a and b with 
# single value
a = torch.tensor(2)
b = torch.tensor(6)

print("Subtraction")
# Normal Subtraction
a.sub(b)

# display content in a
print("Normal Subtraction : ", a.item())

# In-place Subtraction
a.sub_(b)

# display content in a
print("In-place Subtraction : ", a.item())

Output:

Subtraction
Normal Subtraction :  2
In-place Subtraction :  -4

Example 3

In this example, we are creating two tensors a and b that hold a single value and perform all multiplication operations, and then we will display the content of a tensor using the item() method.

# import required library
import torch

# create two tensors a and b with 
# single value
a = torch.tensor(2)
b = torch.tensor(6)

print("Multiplication")
# Normal Multiplication
a.mul(b)

# display content in a
print("Normal Subtraction : ", a.item())

# In-place Multiplication
a.mul_(b)

# display content in a
print("In-place Multiplication : ", a.item())

Output:

Multiplication
Normal Subtraction :  2
In-place Multiplication :  12

Example 4

In this example, we are creating two tensors a and b that hold a multiple value and perform all Addition and multiplication operations, and then we will display the content of a tensor using the item() method.

# import required library
import torch

# create two tensors a and b with 
# 4 values each
a = torch.tensor([2, 3, 4, 5])
b = torch.tensor([2, 3, 4, 5])

print("Addition")
# Normal addition
a.add(b)

# display content in a
print("Normal addition : ", a)

# In-place addition
a.add_(b)

# display content in a
print("In-place addition : ", a)


print("\nMultiplication")

# Normal Multiplication
a.mul(b)

# display content in a
print("Normal Subtraction : ", a)

# In-place Multiplication
a.mul_(b)

# display content in a
print("In-place Multiplication : ", a)
print()

Output:

Addition
Normal addition :  tensor([2, 3, 4, 5])
In-place addition :  tensor([ 4,  6,  8, 10])

Multiplication
Normal Subtraction :  tensor([ 4,  6,  8, 10])
In-place Multiplication :  tensor([ 8, 18, 32, 50])