What is the Difference Between Serial and Parallel Processing in Computer

Architecture
The main difference between serial and parallel processing in computer architecture is that serial
processing performs a single task at a time while parallel processing performs multiple tasks at a
time.
Computer architecture defines the functionality, organization, and implementation of a computer
system. It explains how the computer system is designed and the technologies it is compatible
with. The processor is one of the most essential components in the computer system. It executes
instructions and completes the tasks assigned to it. There are two main types of processing as
serial and parallel processing.

Key Terms
Computer Architecture, Parallel Processing, Serial processing
 What is Serial Processing in Computer Architecture
In serial processing, the processor completes one task at a time. After completing that, it executes
the other tasks in a sequential manner. An operating system executes many programs and each of
them has multiple tasks. The processor has to complete all these tasks, but it completes one task
at a time. The other tasks wait in the queue until the processor completes the current task. In
other words, all tasks are processed sequentially. Hence, this type of processing is called serial
processing or sequential processing. Machines such as Pentium 3 and Pentium 4 perform serial
processing.
We can understand the functionality of serial processing using the following analogy. Assume a
supermarket with multiple queues and only one cashier. The cashier finishes billing the products
of one customer and then moves on to another customer. He performs billing one after the other.

What is Parallel Processing in Computer Architecture

There are multiple processors in parallel processing. Each processor executes the tasks assigned
to them simultaneously. The processors use the bus to communicate with each other and to
access the main memory.  Each processor operates on its local data. As the processors work
independently, failure in one processor does not affect the functionality of another processor.
Therefore, parallel processing increases the throughput as well as improves reliability. Most
modern computers support parallel processing to increase performance.

We can understand the functionality of parallel processing using the following example. In a
supermarket, there are multiple queues, and there is a cashier for each queue. Each cashier bills
the products of the customers in his own queue.

Difference Between Serial and Parallel Processing in Computer

Architecture
Definition
Serial processing is a type of processing in which one task is completed at a time and all the
tasks are executed by the processor in a sequence. Parallel processing is a type of processing in
which multiple tasks are completed at a time by different processors. Thus, this is the
fundamental difference between Serial and Parallel Processing.

Number of processors
A major difference between serial and parallel processing is that there is a single processor in
serial processing, but there are multiple processors in parallel processing.

Performance
Therefore, the performance of parallel processing is higher than in serial processing.
 Work Load
In serial processing, the workload of the processor is higher. However, in parallel processing, the
workload per processor is lower. Thus, this is an important difference between serial and parallel
processing.

Data Transferring
Moreover, in serial processing, data transfers are in bit-by-bit format. However, in parallel
processing, data transfers are in byte form (8 bits).

Required time
Time taken is also a difference between serial and parallel processing. That is; serial processing
requires more time than parallel processing to complete a task.

Cost
Furthermore, parallel processing is more costly than serial processing as it uses multiple
processors.

Conclusion
There are two types of processing as serial and parallel processing in a computer system. The
main difference between serial and parallel processing in computer architecture is that serial
processing performs a single task at a time while the parallel processing performs multiple tasks
at a time. In brief, the performance of parallel processing is higher than the serial processing.

Parallel & Distributed Computing: Fall-2022

lec#1

Course Introduction
With advances in computer architecture, high performance
multiprocessor computers have become readily available & affordable.
As a result supercomputing is accessible to a large segment of
industry that was once restricted to military research & large
corporations.
The course is comprised of architecture & programming of Parallel
computing systems; design concepts, principles, paradigms, models,
performance evaluation & applications.

Parallel Architecture Models:

Implicit: Pipelining, Superscalar & VLIW(Very long instruction word ).
Explicit: SIMD(single-instruction multiple-data),MIMD(Multiple
Instruction and Multiple Data), multicore SMP(Symmetrical
multiprocessing), vector/array processor, MPP(massively parallel
processing) & Clusters
Parallel Memory Models:
SM (shared-memory), DM(Distributed Memory), DSM(Distributed
Shared Memory), UMA(Uniform Memory Access) / NUMA(Non-
Uniform Memory Access)
Cache Coherence protocols
Interconnection Networks (IN):
Static vs Dynamic network topologies
LANs vs SANs (System Area Networks)

Parallel and Distributed Computing

Computing is the process to complete a given goal-oriented task by

using by using computer technology.

Computing may include the design and development of software and

hardware systems for a broad range of purposes, often consist of
structuring , processing and managing any kind of information.
Parallel
In mathematics: parallel means two line that never intersect – Think of an equal sign.

Introduction
Parallel computing system are the simultaneous execution of the single task (split
up and adapted) on multiple processors in order to obtain results faster.
The idea is based on the fact that the process of solving problem usually can be
divided into smaller tasks (divide and conquer), which may be carried out
simultaneously with some coordination.
The term parallel computing architecture sometimes used for a computer with
more than one processor ( few to thousands), available for processing. The recent
multicore processor (Chips with more than one processor core) are some
commercial examples which bring parallel computing to the desktop.
Parallel Computing
Multi-Core & Multi- Processor

Parallel computing is an evolution of serial computing

Parallel computing is a type of computing architecture in which several processors
execute or process an application or computation simultaneously.

Parallel computing is also known as parallel processing.

Why Use Parallel Computing?

Save time and/or money:

Throwing more resources at a task shortens its time to completion
Solving problems in shorter time results in saving big Money in many practical
situations.
Parallel computers can be built from cheap, commodity components.
A computer which consists of a number of inter-connected computers which
cooperatively execute a single program to solve a problem is called a parallel
Why Use Parallel Computing?
Provide concurrent Working environment:
A single compute resource can only do one thing at a time. Multiple computing
resources can do many things simultaneously.
For example, Access Grid www.accessgrid.org provides global collaboration
network where people around the world can meet & conduct work "virtually"

Why Use Parallel Computing?

Integrating Remote Resources usage:
Using compute resources on a wide area network, or even the Internet when local
compute resources are scarce.
For example:
SETI@home satiathome.berkeley.edu uses over 330,000 computers for a
compute power over 528 TeraFLOPS
Folding@home folding.stanforg.edu uses over 340,000 computers for a compute
power of 4.2 PetaFLOPS

Who and What ?

 Why Use Parallel Computing?
Solve larger problems:
Many problems are so large and/or complex that it is impractical
or impossible to solve them on a single computer.
For example:
"Grand Challenge“ en.wikipedia.org/wiki/Grand_Challenge
Ones that cannot be solved in a reasonable amount of time with
today’s computers. Obviously, an execution time of 10 years is
always unreasonable
Such problems require PetaFLOPS & PetaBytes of computing
resources
Some of these are:
Numerical simulation to predict the behaviour of physical
systems.
High performance graphics—particularly visualization, and
animation.
Big data analytics for strategic decision making.
Synthesis of molecules for designing medicines.

Distributed Systems
We define a distributed system as one in which hardware or
software located at networked computers communicate and
coordinate their actions only by passing messages.
The simple definition covers the entire range of systems in which
networked computers can usefully be deployed.
Online meeting
 Parallel & Distributed Computing: Fall-2022

lec#2
Why Use Parallel Computing?
In whether forecasting, a mathematical model of the behavior of the earth’s
atmosphere is developed in which the most important variables are
the wind speed, air temperature, humidity and atmospheric pressure.
The objective of numerical weather modeling is to predict the status of the atmosphere
at a particular region at a specified future time based on the current and past
observations of the values of atmospheric variables.

Why Use Parallel Computing?

In one model, the regions are demarcated by using 180 latitudes and 360
longitudes equally spaced around the globe.
In the vertical direction 12 layers are used to describe the atmosphere.
For each region one point is taken as representing the region and this is called a
grid point.
At each grid point the values of the 5 variables (namely air velocity, temperature,
pressure, humidity, and time) are computed.
The simultaneous algebraic equations are normally solved using an iterative
method.

Why Use Parallel Computing?

In an iterative method several iterations (100 to 1000) are needed for each grid
point before
the results converge. The calculation of each trial value normally requires around
100 to 500 floating point arithmetic operations
Thus, the total number of floating point operations required for each simulation is
approximately given by:
Number of floating point operations per simulation = Number of grid points ×
Number of values per grid point × Number of trials × Number of operations per
trial.
In this example we have:
Number of grid points = 180 × 360 × 12 = 777600
Number of values per grid point = 5, Number of trials = 500
Number of operations per trial = 400
Thus, the total number of floating point operations required per simulation =
777600 × 5 × 500 × 400 = 7.77600 × 1011.

Visualization and Animation

In visualization and animation, the results of computation are to be realistically
rendered on a high resolution terminal.
Assume that
G is the number of area elements where the picture is to be rendered
R is the number of picture elements (called pixels) to be processed in each area
element.
N is the no. of instructions which are required to process a pixel
P is the no. of times the computation should be repeated in a second for
animation.
Thus, computational complexity in this case is: G × R × P × N
Typically, for animation
The computation should be repeated at least 60 times a second i.e P= 60
 G = 105, R = 107
Thus, a computer must process 105 x 107 x 60 x N = 6 x N x 1013 instructions/sec
for animation.
If N=1 and each instruction takes 100 ns then
Processing for Animation will be complotted in 11.57days

The Future:
During the past 20 years, the trends indicated by ever faster networks,
distributed systems, & multi-processor computer architectures
(even at the desktop level) clearly show that
parallelism is the future of computing

Parallel Computing
In simple terms, parallel computing is breaking up a task into smaller pieces and
executing those pieces at the same time, each on their own processor or on a set
of computers that have been networked together. Let's look at a simple example.
Say we have this equation:
Y = (4 x 5) + (1 x 6) + (5 x 3)
On a single processor, the steps needed to calculate a value for Y might look like:
Step 1: Y = 20 + (1 x 6) + (5 x 3)
Step 2: Y = 20 + 6 + (5 x 3)
Step 3: Y = 20 + 6 + 15
Step 4: Y = 41
But in a parallel computing scenario, with three processors or computers, the
steps look something like:
Step 1: Y = 20 + 6 + 15
Step 2: Y = 41

Sequential computing
program that process images and reports how many are
mages ← [ "pet1.jpg", "pet2.jpg", "pet3.jpg", "pet4.jpg"]
numCats ← 0
FOR EACH image IN images
{
foundCat ← detectCat(image)
IF foundCat
{
numCats ← numCats + 1
}
}

The program begins with a list of filenames and a variable to store the number of
images of cats, initialized to 0. A loop iterates through the list, repeatedly calling a
function that detects whether the image has a cat and updating the variable.

Now let's analyze the program further, by annotating each operation with how
many seconds it takes and how many times it's called:

Time Calls Operation

images ← [ "pet1.jpg", "pet2.jpg", "pet3.jpg", "pet4.jp
3 1
]

1 1 numCats ← 0
1 4 FOR EACH image IN images
10 4 foundCat ← detectCat(image)
1 4 IF foundCat
2 4 numCats ← numCats + 1

Note: the values in the time column are made up, the actual time would vary
based on the implementation of this pseudocode and the hardware running it.

We can calculate the total time taken by multiplying the number of seconds for
each operation by the number of calls and summing the results. For this program,
that'd be:

(3×1)+(1×1)+(1×4)+(10×4)+(1×4)+(2×4)=60
This timeline visualizes the time taken by the computer:

Parallel computing

Multi-core processors can take advantage of parallel computing, a computational

model that breaks programs into smaller sequential operations and performs
those smaller operations in parallel.
Can we modify the cat detection program so that some of its operations can be
executed in parallel? Only if there are operations that are not dependent on each
other.
Take a look at the program again and see if you can identify operations that don't
need to happen sequentially.
For this program, we could parallelize the operations inside the loop, since
the detectCat function does not depend on the results from other calls
to detectCat.
That means telling the computer that these lines of code can be executed in
parallel:

foundCat ← detectCat(image) IF foundCat { numCats ← numCats + 1 }

The exact mechanisms for parallelizing a program depend on the particular programming
language and computing environment, which we won't dive into here.

Time Calls Operation

 images ← [ "pet1.jpg", "pet2.jpg", "pet3.jpg"
3 1
]

1 1 numCats ← 0

1 4 FOR EACH image IN images

10 4 foundCat ← detectCat(image)

1 4 IF foundCat

2 4 numCats ← numCats + 1

The non-parallelized operations take the same amount of time as before:

(3 \times 1) + (1 \times 1) = 4(3×1)+(1×1)=4left parenthesis, 3, times, 1, right

parenthesis, plus, left parenthesis, 1, times, 1, right parenthesis, equals, 4

The time taken by the parallelized operations depends on the number of parallel
processors executing operations at once. If I run this program on a computer with
4 cores, then each core can process one of the 4 images, so the parallel portion
will only take as long as a single images takes:

1 + 10 + 1 + 2 = 141+10+1+2=141, plus, 10, plus, 1, plus, 2, equals, 14

This timeline visualizes the time taken by a computer that splits the program
across 4 processes: