How to disable GPU in PyTorch (force Pytorch to use CPU instead of GPU)?

PyTorch is a deep learning framework that offers GPU acceleration. This enables the users to utilize the GPU's processing power. The main goal is to accelerate the training and interference processes of deep learning models. PyTorch automatically utilizes the GPU for operations and this leads to quicker computation times.

Using the GPU for PyTorch is recommended, but not required. In case you do not want to use the GPU or if it isn't being supported, the CPU-only version of PyTorch can be installed. However, by doing so, the CPU would be utilized instead of the GPU. This article will guide you through various methods to ensure PyTorch does not use the GPU, providing detailed explanations and code examples.

How to tell PyTorch to not use the GPU?

There are several methods to prevent PyTorch from using the GPU and force it to use the CPU.

Method 1: Setting Environment Variables

One of the simplest ways to prevent PyTorch from using the GPU is by setting the CUDA_VISIBLE_DEVICES environment variable. This variable controls which GPUs are visible to CUDA applications.

By setting CUDA_VISIBLE_DEVICES to an empty string, you can hide all GPUs from PyTorch. This method is effective and easy to implement.

import os
os.environ["CUDA_VISIBLE_DEVICES"] = ""

This approach ensures that PyTorch will not detect any GPUs and will default to using the CPU.

Method 2: Modifying PyTorch Behavior

Here are two ways to modify PyTorch behavior to ensure it uses the CPU:

a) Overriding torch.cuda.is_available:

import torch
torch.cuda.is_available = lambda: False
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("Using device: ", device)

Example:

import torch
device = torch.device(
  "cuda" if torch.cuda.is_available() else "cpu")
print(f"Using device: {device}")

Output:

Using device: cpu

In the above code torch.cuda.is_available checks if GPU is available, if yes, then it sets the device variable to "cuda". This indicates that the GPU is to be used. But if the GPU isn't available, then it sets the device variable to "cpu". This indicates that the CPU is to be used.

b) Setting Default Tensor Type:

This method forces CPU usage by setting the default tensor type

import torch
torch.set_default_tensor_type(torch.FloatTensor)

This method allows you to set device and consistently reference when creating tensors. You can easily switch not only between GPU and CPU, but also between the different GPUs.

import torch

device = torch.device("cpu")
mytensor = torch.rand(5,5, device=device)
print(mytensor)

Output:

tensor(
[[0.3579, 0.5014, 0.6579, 0.2898, 0.5825],         
[0.2527, 0.5916, 0.4194, 0.8879, 0.2101],         
[0.1731, 0.8477, 0.9272, 0.1533, 0.9765],         
[0.7018, 0.9503, 0.1374, 0.5274, 0.4851],         
[0.2010, 0.7723, 0.5672, 0.4249, 0.0093]])

In above code, we are simply setting the device to CPU in the device variable. Then we create tensors on the desired device using the device flag which thereby creates a tensor directly on the device previously specified.

Method 3: Hiding the GPU

You can hide the GPU from PyTorch by setting CUDA_VISIBLE_DEVICES to a comma.

import os
os.environ["CUDA_VISIBLE_DEVICES"] = ","

How to verify only CPU is used in PyTorch?

There are two ways to verify that PyTorch is using the CPU:

1. Verify only CPU is used in PyTorch using Task Manager:

You can open task manager > CPU and verify the CPU and GPU usage next time when you run operations on PyTorch after disabling GPU usage.

2. Check PyTorch environment variable through code:

Check if PyTorch is using CPU or GPU by running below code.

import torch
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Using device: {device}")

Output:

Using device: cpu

(Note: Now, unless tensors and modules are pushed to GPU using the .cuda() or .to("cuda"), then by default, all the computations will be done on the CPU.)

CPU-Only Example in PyTorch

Below is an example that shows CPU only execution, after the above method is followed:

import torch
import torch.nn as nn

device = torch.device("cpu")
# Create a  model
model = nn.Linear(5,1)
# send existing model to device
model = model.to(device)

# Create input data inside CPU
inputs = torch.randn(12, 5, device=device)
# Perform forward pass on the model
output = model(inputs)
print(output)

Output:

tensor([[ 0.9721],         
[ 0.2088],         
[-0.7534],         
[ 0.0765],         
[-1.8325],         
[ 0.3485],         
[-0.3856],         
[-0.1351],         
[ 0.6091],         
[-0.2571],         
[-0.4597],         
[-0.0838]], grad_fn=<AddmmBackward0>)

Here we create a linear model and initialize the model on the CPU. Then we create a tensor of random input data directly on the CPU. And finally, it performs a forward pass of the input data through the model.

When is running PyTorch on CPU Beneficial over GPU?

Older version of PyTorch automatically download CPU-only version. Newer PyTorch versions automatically utilize the GPU.

To implement CPU-Only execution in PyTorch, either download older version of PyTorch which uses CPU-Only version, or use any method as given in above section of this article to disable the GPU or hide the GPU or set device as "cpu".

In deep learning, GPUs are often preferred over CPUs due to their ability to handle large-scale computations more efficiently. However, there are situations where running PyTorch on a CPU is necessary or more convenient. These situations include:

• Hardware Limitations: Not all systems have a compatible GPU.
• Debugging: Debugging on a CPU can be simpler and more straightforward.
• Resource Management: In shared environments, you might want to reserve GPUs for more demanding tasks.

Conclusion

In this article, we have explored how to check if PyTorch is using the GPU, and if it is being used, then how to disable it and use the CPU instead. By following the steps outlined in this article, you can ensure that your PyTorch is using CPU by disabling GPU and you may verify it too.