PyTorch is a python library developed by Facebook to run and train deep learning and machine learning algorithms. Tensor is the fundamental data structure of the machine or deep learning algorithms and to deal with them, we perform several operations, for which PyTorch library offers many functionalities.
Tensor operations that handle indexing on some particular row or column for copying, adding, filling values/tensors are said to be index-based developed operation. There are two types of index-based operations in PyTorch, one is in-place operations and the other is out-of-place operations. The basic difference between the two is in-place operation directly changes the values of the tensors without making any copy of that whereas out of place operations don't. Following are the operations:-
- index_add_
- index_add
- index_copy_
- index_copy
- index_fill_
- index_fill
- index_put_
- index_put
- index_select
Now we discuss each of the functions with proper examples.
1. index_add_: Adds the given tensor elements to the self tensor along the order given in the matrix.
Syntax:
index_add_(dim,index,ensor)---> Tensor
Parameters:
- dim: dimension along which index to add. '0' stands for column and '1' stands for row.
- index: indices of the tensor to select from. It can be LongTensor or IntTensor.
- tensor: tensor containing the values to add.
Example 1: We take a zero vector 'x', te tensor of size(3,5) and index tensor. Accumulating the resultant vector along rows we get the output.
Python3
#importing libraries
import torch
x=torch.zeros(5,5)
te=torch.tensor([[1,3,5,7,9],[1,3,5,7,9],[1,3,5,7,9]],dtype=torch.float32)
index0=torch.tensor([0,2,4])
#adding tensor te to x along row of the given order
x.index_add_(0,index0,te)
Output:
tensor([[1., 3., 5., 7., 9.],
[0., 0., 0., 0., 0.],
[1., 3., 5., 7., 9.],
[0., 0., 0., 0., 0.],
[1., 3., 5., 7., 9.]])
Example 2:
Python3
#importing libraries
import torch
y=torch.ones(5,5)#unitvector
index2=torch.tensor([0,1,1,1,2])
ten=torch.randn(1,5)
#adding values to y along the column with given order
y.index_add_(1,index2,ten)
Output:
tensor([[0.9460, 0.4762, 1.2219, 1.0000, 1.0000],
[0.9460, 0.4762, 1.2219, 1.0000, 1.0000],
[0.9460, 0.4762, 1.2219, 1.0000, 1.0000],
[0.9460, 0.4762, 1.2219, 1.0000, 1.0000],
[0.9460, 0.4762, 1.2219, 1.0000, 1.0000]])
2.index_add: It is the out-of place version of the above function. This adds a given tensor to the self tensor temporarily. Parameters and syntax are the out-of-placesame as above.
Example 3:
Python3
import torch
y=torch.ones(5,5)
index2=torch.tensor([0,1,1,1,2])
ten=torch.randn(1,5)
print("Indexed Matrix:\n",y.index_add(1,index2,ten))
print ("Printing Indexed Matrix again:\n",y)
Output:
Indexed Matrix:
tensor([[-0.2811, -1.0776, 2.2697, 1.0000, 1.0000],
[-0.2811, -1.0776, 2.2697, 1.0000, 1.0000],
[-0.2811, -1.0776, 2.2697, 1.0000, 1.0000],
[-0.2811, -1.0776, 2.2697, 1.0000, 1.0000],
[-0.2811, -1.0776, 2.2697, 1.0000, 1.0000]])
Printing Indexed Matrix again:
tensor([[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.]])
3.index_copy_: Copies the elements of a given tensor to the input tensor by selecting the indices in the order given in 'index'.
Syntax:
index_copy_(dim,index,tensor)---> Tensor
Parameters:
- dim: dimension along which index to copy. It is of 'int' format.
- index: indices of the tensor to select from. It can be either IntTensor or LongTensor.
- tensor: tensor containing the values to be copied.
Example 4: Here elements of 'a' is being replaced by 't1' tensor in an order given in the 'index3'.
Python3
#importing libraries
import torch
a=torch.ones(4,4)#unit vector
t1=torch.randn(2,4)
index3=torch.tensor([1,3])
#copying elements of t1 ensor to 'a' in given order of index
a.index_copy_(0,index3,t1)
Output:
tensor([[ 1.0000, 1.0000, 1.0000, 1.0000],
[-0.1918, -1.2089, 0.3229, -0.1831],
[ 1.0000, 1.0000, 1.0000, 1.0000],
[ 0.7521, 0.8424, -0.8698, -0.3908]])
Example 5: In the below example we will receive an error. This is because the dimth dimension of the tensor is not equal to the length of the index. So we have to keep in mind that dimth dimension of the tensor must have the same size as the length of the index.
Python3
#importing libraries
import torch
y=torch.ones(5,5)
index1=torch.tensor([0,1,2,3,4])
te=torch.tensor([[1,3,5,7,9],[1,3,5,7,9],[1,3,5,7,9]],dtype=torch.float32)
y.index_copy_(1,index1,te)
Output:
RuntimeError Traceback (most recent call last)
<ipython-input-8-25e4150d5bd7> in <module>
1 y=torch.ones(5,5)
2 index1=torch.tensor([0,1,2,3,4])
----> 3 y.index_copy_(1,index1,te)
RuntimeError: index_copy_(): Source/destination tensor must have same slice shapes.
Destination slice shape: 5 at dimension 1 and source slice shape: 3 at dimension 0.
Example 6: In this, we will copy the given tensor to the self tensor keeping in mind that dimth dimension of the tensor must have the same size as the length of the index.
Python3
import torch
b=torch.ones(4,4)
t2=torch.randn(4,2)
index4=torch.tensor([0,1])
b.index_copy_(1,index4,t2)
Output:
tensor([[-0.3964, -0.3859, 1.0000, 1.0000],
[ 2.6910, -0.9394, 1.0000, 1.0000],
[ 0.3591, -0.2262, 1.0000, 1.0000],
[ 1.2102, -0.8340, 1.0000, 1.0000]])
4.index_copy: This is the out-of-place index-based operation for replacing elements of input tensor with a given tensor. Syntax, parameters are the same as the above.
5.index_fill_: 'Val' value is filled with the elements of 'x' along with the order of indices given in the vector 'index'.
Syntax:
index_fill_(dim, index, val) → Tensor
Parameters:
- dim: dimension along which to fill. It is of 'int' format.
- index: filling in order of the indices given in this index vector. It can be either IntTensor or LongTensor.
- val(float): value with which to fill in.
Example 7: In this example, we will declare a tensor with random elements and then fill it with '4' along with the given indices.
Python3
#importing libraries
import torch
c=torch.randn(4,4)
index5=torch.tensor([0,2])
#filling 4 within the elements of the tensor 'c' along the indices 0,2
c.index_fill_(0,index5,4)
print(c)
Output:
tensor([[ 4.0000, 4.0000, 4.0000, 4.0000],
[ 0.4762, 0.0056, 0.3258, 1.1345],
[ 4.0000, 4.0000, 4.0000, 4.0000],
[-0.1490, -0.6543, 0.9755, 1.8087]])
Example 8: Similarly we perform a filling operation along with the columns.
Python3
d=torch.randn(5,5)
d.index_fill(1,index5,2)
print(d)
Output:
tensor([[ 0.5978, -1.2461, -0.8794, -1.0175, 0.8938],
[-0.6374, 1.0848, 0.1291, 0.6658, 0.3081],
[-0.9686, -0.8212, -0.5223, -0.3208, -1.7718],
[-0.1153, -1.2552, -0.4119, -1.1293, 0.2266],
[ 1.2610, 0.2618, -1.5528, 0.7805, 1.3730]])
6. index_fill: This is the out of place index-based operation for filling the elements with tensor with 'val'. Syntax, parameters are the same as the above.
7.index_put_: This operation puts the value of 'val' into the self tensor using the indices of the given 'index'.
Syntax:
index_put_(indices, values, accumulate=False) → Tensor
Parameters:
- indices: It is the tuple of LongTensor which is used to index to self.
- values: Tensor with values that need to be put into the target.
- accumulate: whether to accumulate or not.
Example 9: Here we take the target vector and replaces the values of value tensor as mentioned in the
Python3
#importing libraries
import torch
target=torch.zeros([4,4])
indices = torch.LongTensor([[0,1],[1,2],[3,1],[1,0]])#indices to which values to be put
value = torch.ones(indices.shape[0])
#tuple of the index tensor is passed along with the value
target.index_put_(tuple(indices.t()), value)
Output:
tensor([[0., 1., 0., 0.],
[1., 0., 1., 0.],
[0., 0., 0., 0.],
[0., 1., 0., 0.]])
Note: We have to take the transpose of the indices tensor or else an error will occur.
Example 10: In this example, we keep the accumulate function as true which means the elements in value are added to the target.
Python3
e=torch.ones([4,4])
indices2=torch.LongTensor([[0,1],[0,1],[2,1]])
value2=torch.zeros(indices2.shape[0])
e.index_put_(tuple(indices2.t()),value2,accumulate=True)
Output:
tensor([[1., 1., 1., 1.],
[1., 1., 1., 1.],
[1., 1., 1., 1.],
[1., 1., 1., 1.]])
8. index_fill: This is the out of place version of index_fill_. Syntax, parameters are the same as above.
9. index_select: A tensor is returned with indices as mentioned, by selecting from the target tensor.
Syntax:
torch.index_select(input, dim, index, out=None)
Parameters:
- input(Tensor): Tensor from which the indices will be selected.
- dim(int): dimension along which to select.
- index: It contains indices to the index.
Example 11: We pick up 0,1 indices of the tensor 'm' along dim=0 i.e. rows and print the output.
Python3
#importing libraries
import torch
m=torch.randn(3,4)
print('Original matrix:\n',m)
indices=torch.tensor([0,1])
print("Indexed Matrix:\n",torch.index_select(m, 0, indices))
Output:
Original matrix:
tensor([[ 0.2008, -0.2637, 2.1216, -0.2892],
[-0.4059, -1.6054, -2.5022, -0.2912],
[-0.3246, 0.4751, -0.1018, -0.6707]])
Indexed Matrix:
tensor([[ 0.2008, -0.2637, 2.1216, -0.2892],
[-0.4059, -1.6054, -2.5022, -0.2912]])
Reference: https://pytorch.org/docs/stable/tensors.html
Similar Reads
Tensor Operations in PyTorch
In this article, we will discuss tensor operations in PyTorch. PyTorch is a scientific package used to perform operations on the given data like tensor in python. A Tensor is a collection of data like a numpy array. We can create a tensor using the tensor function: Syntax: torch.tensor([[[element1,e
5 min read
Vector Operations in Pytorch
In this article, we are going to discuss vector operations in PyTorch. Vectors are a one-dimensional tensor, which is used to manipulate the data. Vector operations are of different types such as mathematical operation, dot product, and linspace. PyTorch is an optimized tensor library majorly used f
4 min read
Python | Pandas Index.flags
Pandas Index is an immutable ndarray implementing an ordered, sliceable set. It is the basic object which stores the axis labels for all pandas objects. Pandas Index.flags attribute return the status of all the flags for the given Index object. Syntax: Index.flags Parameter : None Returns : status o
2 min read
Python | SymPy Permutation.index() method
Permutation.index() : index() is a sympy Python library function that returns the index value of the permutation in argument. Index of a permutation = Sum of all subscripts j such that permutation[j] is greater than permutation[j+1]. Syntax : sympy.combinatorics.permutations.Permutation.index() Retu
1 min read
Perl | Hash Operations
Prerequisite: Perl Hashes, Perl Hash As most readers likely know, the hash stores data by using a mechanism called Hashing. In hashing, a key is used to determine a value or data. These keys must be unique and are then used as the index at which the data associated with the key is stored. This data
8 min read
Python | Pandas Index.get_loc()
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas Index.get_loc() function return integer location, slice or boolean mask for req
3 min read
Python | Pandas Index.data
Pandas Index is an immutable ndarray implementing an ordered, sliceable set. It is the basic object which stores the axis labels for all pandas objects. Pandas Index.data attribute return the data pointer of the underlying data of the given Index object. Syntax: Index.data Parameter : None Returns :
2 min read
How to Perform in-place Operations in PyTorch?
In this article, we will see different in-place operations performed on tensors in PyTorch. Inplace operations are used to directly alter the values of a tensor. The data collected from the user will not be copied. The fundamental benefit of adopting these procedures is that they reduce memory stora
3 min read
Python | Pandas Index.where
Pandas Index is an immutable ndarray implementing an ordered, sliceable set. It is the basic object which stores the axis labels for all pandas objects. Pandas Index.where function return an Index of same shape as self and whose corresponding entries are from self where cond is True and otherwise ar
2 min read
Python | Pandas Index.shape
Pandas Index is an immutable ndarray implementing an ordered, sliceable set. It is the basic object which stores the axis labels for all pandas objects. Pandas Index.shape attribute return a tuple of the shape of the underlying data in the given Index object. Syntax: Index.shape Parameter : None Ret
2 min read