CA1-Assignment

Graphics Processing Unit

Submitted By

Bhagyashri Sonawane (CS2264)

Aditi Pawse(CS2269)
Gaziya feroztadvi(CS2270)
Bhagyashri Wanare(CS2275)

Deogiri Institute of Engineering & Management Studies

Chhatrapti Sambhajinagar.
 Graphics Processing Unit

1. INTRODUCTION Pg no.

1.1 Introduction 1

1.2 Types of GPU 3

1.3 Features of GPU 4

2. LITERATURE SURVEY
2.1 Literature survey on CPU & GPU 5
2.2 Technical Detail 5
3. COMPARATIVE ANALYSIS
3.2 Comparative Analysis 11

4. FUTURE TRENDS
4.2 Future Trends of GPU 15

5. CONCLUSION
5.1 Conclusion 17
1

1. INTRODUCTION

1.1 Graphics Processing Unit: Graphics processing unit, a specialized processor originally
designed to accelerate graphics rendering. GPUs can process many pieces of data
simultaneously, making them useful for machine learning, video editing, and gaming
applications. GPUs may be integrated into the computer’s CPU or offered as a discrete hardware
unit.

GPUs are also known as video cards or graphics cards. In order to display pictures,
videos, and 2D or 3D animations, each device uses a GPU. A GPU performs fast calculations of
arithmetic and frees up the CPU to do different things. A GPU has lots of smaller cores made for
multi-tasking, while a CPU makes use of some cores primarily based on sequential serial
processing. In the world of computing

What does a GPU Do? The graphics processing unit, or GPU, has become one of the most
important types of computing technology, both for personal and business computing. Designed
for parallel processing, the GPU is used in a wide range of applications, including graphics and
video rendering. Although they’re best known for their capabilities in gaming, GPUs are
becoming more popular for use in creative production and artificial intelligence (AI).
GPUs were originally designed to accelerate the rendering of 3D graphics. Over time, they
became more flexible and programmable, enhancing their capabilities. This allowed graphics
programmers to create more interesting visual effects and realistic scenes with advanced lighting
and shadowing techniques. Other developers also began to tap the power of GPUs to
dramatically accelerate additional workloads in high performance computing (HPC), deep
learning, and more.
2

What Are GPUs Used For? Two decades ago, GPUs were used primarily to accelerate real-
time 3D graphics applications, such as games. However, as the 21st century began, computer
scientists realized that GPUs had the potential to solve some of the world’s most difficult
computing problems. This realization gave rise to the general purpose GPU era. Now, graphics
technology is applied more extensively to an increasingly wide set of problems. Today’s GPUs
are more programmable than ever before, affording them the flexibility to accelerate a broad
range of applications that go well beyond traditional graphics rendering.

1. GPUs for Gaming: Video games have become more computationally intensive, with hyper
realistic graphics and vast, complicated in-game worlds. With advanced display technologies,
such as 4K screens and high refresh rates, along with the rise of virtual reality gaming, demands
on graphics processing are growing fast. GPUs are capable of rendering graphics in both 2D and
3D. With better graphics performance, games can be played at higher resolution, at faster frame
rates, or both.

2. GPUs for Video Editing and Content Creation: For years, video editors, graphic designers,
and other creative professionals have struggled with long rendering times that tied up computing
resources and stifled creative flow. Now, the parallel processing offered by GPUs along with
built-in AI capabilities and advanced acceleration makes it faster and easier to render video and
graphics in higher-definition formats. Uniting fluid gaming experiences with the latest in visual
technologies, Intel Arcgraphics for desktops enables captivating content immersion. Intel Arc
graphics cards include built-in machine learning, graphics acceleration, and ray-tracing hardware
with scalable performance options for laptops, desktops, and professional workstations.

3. GPU for Machine Learning: Some of the most exciting applications for GPU technology
involve AI and machine learning. Because GPUs incorporate an extraordinary amount of
computational capability, they can deliver incredible acceleration in workloads that take
advantage of the highly parallel nature of GPUs, such as image recognition. Many of today’s
deep learning technologies rely on GPUs working with CPUs.
3

1.2 Types of GPU:

1. Integrated GPU:The integrated GPU is a computer where this processor is built on a similar
chip as the CPU. Being built along with the CPU simply provides many benefits like being small
in size, energy efficiency, and not expensive as compared to the opposed discrete GPU. These
types of GPUs use the system RAM instead of having their own RAM similar to discrete GPUs.
The benefit of This GPU is that they don’t consume as much power & generate heat as discrete
GPUs but they also cost less to acquire.

2. Discrete GPU: A dedicated graphics card like a discrete GPU is totally separate from the
CPU. The graphics card encapsulates the graphics processor to separately process instructions &
elements from the CPU. This graphic card has its own form of VRAM or video RAM which
provides the discrete GPU fast access to image data. The main drawback of this dedicated
graphics card is; it generates more heat. In a computer, the GPU is frequently the hottest running
device, which needs a strong power supply.
4

1.3 Features of GPU:

❖ A GPU handles huge amounts of data within various streams.

❖ These processors include a number of cores that handle at a time different threads as
compared to the CPU because it has few cores only with lots of cache memory that
handles only a few software threads.
❖ Some of the best supercomputers around the world mainly depend on GPUs for advanced
scientific discoveries.
❖ These processors are capable of performing floating-point arithmetic & vector operations
with the most recent cards to manipulate double-precision floating-point numbers.
❖ The execution units of GPU can’t deal efficiently with branching and they don’t support
interrupts of hardware equally as CPUs do.
❖ As compared to the CPU, GPU depreciates fast.
5

2. LITERATURE SURVEY
2.1 Literature survey:

This survey provides an overview and insight on the use of GPU computing for large-scale data
mining, focusing on the advantages and limitations of GPUs for big data on state-of-the-art data
mining algorithms, complementing and updating previous surveys on the status of this research
area. First, it discusses how the GPU architecture is organized and its evolution, the
programming model, and the advantages to match the computational requirements of large-scale
data mining algorithms. Second, it analyzes current trends and research works on the
implementation of different popular families of data mining techniques using GPUs, discussing
specific optimizations and noteworthy results. Third, it reviews particular application areas
where GPUs have significantly assisted to speedup real-world problems requiring fast processing
of large data volumes in data mining. Fourth, it discusses novel research works on the integration
of GPU computing with popular MapReduce frameworks for distributed computing from two
perspectives: (1) implementations of the MapReduce programming model within a GPU and (2)
exploitation of GPUs for computing the jobs in the nodes of a distributed cluster using existing
MapReduce frameworks. Finally, it summarizes current GPU limitations, future architectures
advantages, open challenges, and future directions to pursue in order to keep increasing the
scalability of parallel data mining algorithms to the ever-increasing size of data. GPUs are many-
core architectures highly suitable for massively data parallel general-purpose computation.

2.2 Technical Detail:

A graphics processing unit (GPU) is a specialized electronic circuit initially designed to
accelerate computer graphics and image processing (either on a video card or embedded on the
motherboards, mobile phones, personal computers, workstations, and game consoles). After their
initial design, GPUs were found to be useful for non-graphic calculations involving
embarrassingly parallel problems due to their parallel structure.

What are the components are used in GPU:-

Typically referred to as the ‘muscle’ of a computer, the graphics card (GPU) is responsible for
displaying all images and videos on a monitor. Some CPUs will feature integrated graphics, but
for mid to A gaming, it’s recommended you utilize a dedicated GPU.
6

The GeForce RTX™ 3060 lets you take on the latest games using the power of Ampere-
NVIDIA's 2nd generation RTX architecture. Get incredible performance with enhanced Ray
Tracing Cores and Tensor Cores, new streaming multiprocessors, and high-speed G6 memory.

Graphics Card Components: While graphics cards are available in different form factors, they
all feature the same core components. We’ve included a complete breakdown of each of the
different components below.

Graphics Card

1. Graphics Processing Unit: A GPU core can have four to 10 threads per core. A GPU is able
to render images more quickly than a CPU because of its parallel-processing architecture, which
allows it to perform multiple calculations at the same time.The graphics processing unit is where
all of the transistors, CUDA cores, and streaming multiprocessors are located. It is responsible
for accelerating the rate at which your gaming PC can render 3D images.

Graphics Processing Unit

7

2.Video Memory (VRAM): VRAM's purpose is to ensure the even and smooth execution
of graphics display. It is most important in applications that display complex image textures
or render polygon-based three-dimensional (3D) structures. People commonly use VRAM
for
applications such as video games or 3D graphic design programs.
The video memory, or VRAM, serves as a separate cache for the GPU to store any
crucial information it needs for displaying graphics. Compared to standard RAM, video memory
is embedded closer to the graphics processing unit which results in faster data transfer speeds
with minimal input lag.

Video Memory (VRAM)

3. Voltage Regulator Module (VRM): The VRM is responsible for ensuring that the GPU
receives all of the necessary power at a constant steady voltage. Graphics cards can come with
varying power consumptions. Whilst some GPUs will draw power from the PCIe slot, some will
require an additional connector from the power supply (PSU).

Voltage Regulator Module (VRM)

8

4. PCIe Interface: PCIe slots allow your motherboard to connect with your PC's most critical
components and provide key functionality. They also supply you with numerous customization
and upgrade options when you're ready to venture beyond preloaded features like graphics and
storage. The PCIe interface or slot is how the graphics card is plugged into the motherboard. For
the graphics card, you will typically utilize either the PCIe x 8 or x 16 lane.

Peripheral interconnect component Express

5. Cooling: The cooling system serves three important functions. First, it removes excess heat
from the engine; second, it maintains the engine operating temperature where it works most
efficiently; and finally, it brings the engine up to the right operating temperature as quickly as
possible.
All graphics cards will come with an integrated thermal system that will consist of a
heatsink and/or several fans. This thermal system is responsible for drawing away heat from the
core components of the graphics card to ensure it consistently performs at optimum
temperatures.

6. Interconnection interface: The interconnection bus allows you to connect multiple graphics
cards on a single motherboard for enhanced capabilities and scaling . On AMD graphics cards,
this technology is called CrossFireX. Whereas on all NVIDIA GPUs, this is called NVIDIA
NVLink.

7. Video BIOS: A video BIOS or VBIOS is the Basic Input Output System (BIOS) of a graphics
card or the integrated graphics controller in a computer. The VBIOS provides a set of video-
related functions that are used by programs to access the video hardware. The video BIOS
9

serves as a program hub where you can make adjustments to specific settings, such as the clock
frequencies, fan speeds, and voltages.

Video BIOS

8. Output interface: These are built onto the graphics card itself and are how you connect the
GPU to a monitor. On most GPUs, you will have a DisplayPort, DVI and HDMI port. An Output
Interface can define any device that processes output such as a printer, fax, and email. An Output
Interface can be assigned to one or more Output Devices.

The different GPU power connectors are:

❖ 6-pin (ideal for GPUs that need 150W)
❖ 8-pin (suited for GPUs that require more than 150W. You may have just one
8-pin or two 6- pin connectors)
❖ 12-pin (designed for high-end/professional GPUs) .
With the latest NVIDIA RTX 4090 and 4080 graphics cards requiring a lot more power, a new
standard of PSU, ATX 3.0, has been created with a 16-pin 12VHPWR connector. This power
connector has been designed to deliver up to 600W from one singular cable. If you do opt for a
4090 or 4080, your chosen GPU should come with a 12VHPWR adapter.
1
0

Which journals,books,websites are used in GPU:-

NVIDIA GeForce RTX 3090 Ti is one of the best GPUs for deep learning if you are a data
scientist that performs deep learning tasks on your machine. Its incredible performance and
features make it ideal for powering the most advanced neural networks than other GPUs.
Powered by NVIDIA ampere architecture, it provides the fastest speeds possible. With this
NVIDIA Geforce RTX GPU, gaming enthusiasts may experience a maximum setting of 4K, ray-
traced games at the fastest possible rates, and even 8K NVIDIA DLSS-accelerated gaming on
monitors that support 8K 60Hz, as stated in HDMI.
11

3. COMPARATIVE ANALYSIS.

3.1 Comparative analysis :

Machine Learning is a field that makes the computer system learn without being explicitly
programmed. It requires a massive amount of data to generate meaningful information and make
decisions. The formal and scientific definition of Machine Learning is ”A computer program is
said to learn from experience E with respect to some class of task T and performance P. If its
performance at tasks in T, as measured by P, improves with experience E.” Deep Learning is a
sub branch of machine learning which tends to use neural networks for data processing and
decision making. The working of artificial neural network models mimics the human brain.

Central Processing Unit (CPU) and Graphical Processing Unit (GPU) are two processing units
that are extensively used to process ML and DL models. GPU is specially designed for parallel
computation while CPU is not used for the same. Pytorch and Tensorflow are two frameworks
that provide an abstraction for complex mathematical calculations in the field of deep learning.

The need for GPUs arises for training the deep learning network when a vast amount of data is
generated. The normal CPUs typically have 4-5 cores and only a limited number of threads can
be handled. While GPUs have thousands of small cores that can handle several computation
threads in parallel as shown in Figure 1. For example, the NVIDIA A6000 has 10752 CUDA
cores. A deep learning system contains millions of calculations to train and infer which needs a
GPU for fast processing.

1. PROBLEM STATEMENT: In this research, we will focus on how the mathematical

operations are executed on CPU and GPU and analyze their time and memory. Analyzing time
and memory at runtime helps to optimize the network operations which helps in faster execution
and inference. Using Pytorch Profiler [4] API and NVIDIA commands, we can visualize the
network graphs and find out the execution time of each operation in addition to the memory
consumption status.

2. SCOPE: The main motivation behind this research is to find how CPU and GPU operations
happen while training machine learning and deep learning models. The main focus will be on
CPU and GPU time and memory profiling part, but not on the deep learning models. The
1
2

profiling will be generated using a deep learning model using Pytorch [4] profiler and
Tensorboard . Tracing is done at each training step to get the data about time and memory. Later,
the resource utilization is compared between CPU and GPU.

3. BACKGROUND :

The following terminology is used in the paper.

1. Neural Network: It is the group of interconnected nodes where the data flow and
processing happens to produce an output.
2. CPU: CPU stands for Central Processing Unit which processes the task inside a
computer. It consists of RAM, a cache, a control unit, and an arithmetic unit.
3. GPU: GPU stands for Graphical Processing Unit that consists of multiple components
similar of CPU designed for parallel computations.
4. CUDA: CUDA stands for Computer Unified Device Architecture which is a parallel
computing platform and model used to make programs compatible to work with GPU.
5. Profiler: It is a tool to analyze and measure the performance of the program, and usage
of resources like time and memory.
6. Memory Allocation: The process of allocating computer memory to execute programs
and processes.
7. Pytorch: It is a deep learning framework developed by Meta which helps us to build
neural network pipelines.
13

Advantages of GPU:
❖ The graphic processing unit is used in video editing, gaming & machine learning
applications.
❖ These are integrated into the CPU of the computer.
❖ These are capable of graphics rendering in both 2D & 3D.
❖ These processors can perform parallel operations.
❖ GPUs have high throughput.
❖ These processors are excellent for parallel processing.

Disadvantages of GPU:

❖ These processors require lots of regularity, data reuse, and data parallelism.
❖ As compared to CPUs. GPUs are very hard to program & tune.
❖ GPUs perform a single task at a massive scale, however, they cannot execute
general-purpose computing tasks.
❖ Individual GPUs processors are at present very expensive as compared to CPUs.
❖ Special large-scale GPUs can cost hundreds to thousands of dollars.
❖ Graphic processing units execute only some kinds of code.
❖ GPUs can perform a single task at a huge scale, however, they cannot perform normal
computing tasks.
1
4

4. FUTURE TRENDS
4.1 Future trends

1. Increased Performance: GPU manufacturers are constantly striving to improve performance.

This includes increasing the number of cores, optimizing architecture, and refining
manufacturing processes. As a result, we can expect GPUs to continue offering higher levels of
performance for a wide range of applications, including gaming, artificial intelligence, and
scientific computing.

2. Ray Tracing: Ray tracing is a rendering technique that simulates the way light interacts with
objects in a 3D environment, resulting in more realistic and visually stunning graphics. GPUs are
expected to continue to support and accelerate ray tracing, making it more accessible for gaming
and other graphics-intensive applications.

3. AI Integration: GPUs have become a crucial component for AI and machine learning
applications due to their parallel processing capabilities. Future GPUs are likely to have
enhanced AI-specific features and optimizations to accelerate deep learning tasks and enable
more advanced AI applications.

4. Energy Efficiency: With an increasing focus on energy efficiency and sustainability, GPU
manufacturers are expected to develop more power-efficient GPUs. This can be achieved
through advanced manufacturing processes, architectural optimizations, and better power
management techniques.

5. Hybrid Computing: GPUs are not limited to graphics processing anymore. They are
increasingly used for general-purpose computing tasks. In the future, we can expect GPUs to
become more versatile and integrated into a wider range of applications, including data analytics,
scientific simulations, and more.

6. Connectivity and Memory: Faster and more efficient memory technologies like HBM (High
Bandwidth Memory) and GDDR6 have been introduced. Future GPUs may continue to adopt
these technologies, leading to better memory bandwidth and overall performance. Additionally,
advancements in GPU interconnects, such as NVLink, will enable GPUs to work together
seamlessly in multi-GPU setups.
15

5. CONCLUSION:
In conclusion, several steps of the machine learning process require CPUs and GPUs.
While GPUs are used to train big deep learning models, CPUs are beneficial for data preparation,
feature extraction, and small-scale models. For inference and hyperparameter tweaking, CPUs
and GPUs may both be utilized. Hence both the Processing units have their own need specific
usage, it’s the user who needs to be aware of which can be used when. I hope this article made
you aware of CPU vs GPU and a smart user of these processors.
The GPU is an essential component of every computer system. They display everything
on our screens. They can be either integrated into a CPU, or stored in separate forms as different
GPUs. There are currently two major GPU manufacturers: Nvidia and AMD. Both have
excellent options, and you can choose the best GPU based on your use. You can also use our
factors to help you make an informed decision. Graphics Processing Units (GPUs) continue to be
a dynamic and influential component in the world of computing and technology.