Parallel Processing

Introduction
Playing a simple video game on a computer uses audio, images, calculations, rendering, etc.
If a single CPU handles all these things, it will take time to process them. On the other hand,
using multiple CPUs to handle various jobs will work much more smoothly. This process of
executing multiple tasks concurrently is called parallel processing.
Parallel processing is used to enhance the performance of a computer by performing multiple
instructions simultaneously. Unlike sequential processing, parallel architecture can divide a
job into parts and multi-task them. That means concurrent execution of instructions is
possible. Thus, the computational speed of the computer will increase.
One of the significant components of the CPU is ALU (Arithmetic Logic Unit). It is helpful
to execute any instruction on the computer. ALU is divided into several functional units:

All these units will work concurrently and produce the required output.
For Example:
1. The system may have two or more ALUs and can execute multiple instructions simultaneously.
2. The system may have multiple processors operating simultaneously.
3. The following instruction can be fetched while the current instruction is being executed in ALU.
The primary purpose of parallel processing is to increase the computer processing capability
and its throughput. We can achieve parallel processing by using multiple functional units that
can perform identical or different operations concurrently to distribute the data among these
functional units.
 With and without parallel processing

Need of Parallel Processing

1. Application demands: With increasing technology, modern applications require more computing
cycles. Examples include videos, high graphic games, databases, science models, etc.
2. Technology Trends: With the increasing number of transistors on a chip, clock rates are
expected to go up but slowly.
3. Economics: Instead of costly components used in traditional supercomputers, today's
microprocessors offer high performance and have multiprocessor support.
Advantages of Parallel Processing
1. It increases the speed and efficiency of computers. Sequential computing forces fast processors
to do things inefficiently.
 2. Computers can be used to solve more complex and more extensive problems. A single web app
may have to process millions of requests every second with so much data.
3. For a system that has to support billions of operations (for example, bank software), parallel
processing makes things cheaper.
4. Parallel computing is more suited for hardware since serial computing wastes the computing
power of processors.
Disadvantages of Parallel Processing
1. Increases the cost of computers since more hardware is required.
2. Multicore architectures consume higher power.
3. Parallel architectures are difficult to achieve.
4. A parallel computing system needs different code tweaking depending on the target architecture.
5. It increases the overhead cost due to synchronization and data transfers.
Also See, Shift Registers in Digital Electronics
FAQs
1. What is parallel processing?
Ans Parallel processing divides a big task into smaller tasks and executes these smaller tasks
concurrently to get the output. It increases the speed and efficiency of computers.
2. What is the use of parallel processing?
Ans Parallel processing increases the speed and efficiency of computers. Computers can
perform much more computations within the given time; thus, user experience is enhanced.

Computer Network Architecture

Computer Network Architecture is defined as the physical and logical design of the
software, hardware, protocols, and media of the transmission of data. Simply we can
say that how computers are organized and how tasks are allocated to the computer.

The two types of network architectures are used:

o Peer-To-Peer network
o Client/Server network

Peer-To-Peer network
o Peer-To-Peer network is a network in which all the computers are linked together
with equal privilege and responsibilities for processing the data.
 o Peer-To-Peer network is useful for small environments, usually up to 10 computers.
o Peer-To-Peer network has no dedicated server.
o Special permissions are assigned to each computer for sharing the resources, but this
can lead to a problem if the computer with the resource is down.

Advantages Of Peer-To-Peer Network:

o It is less costly as it does not contain any dedicated server.
o If one computer stops working but, other computers will not stop working.
o It is easy to set up and maintain as each computer manages itself.

Disadvantages Of Peer-To-Peer Network:

o In the case of Peer-To-Peer network, it does not contain the centralized system .
Therefore, it cannot back up the data as the data is different in different locations.
o It has a security issue as the device is managed itself.

Client/Server Network
o Client/Server network is a network model designed for the end users called clients, to
access the resources such as songs, video, etc. from a central computer known as
Server.
o The central controller is known as a server while all other computers in the network
are called clients.
o A server performs all the major operations such as security and network
management.
 o A server is responsible for managing all the resources such as files, directories,
printer, etc.
o All the clients communicate with each other through a server. For example, if client1
wants to send some data to client 2, then it first sends the request to the server for
the permission. The server sends the response to the client 1 to initiate its
communication with the client 2.

Advantages Of Client/Server network:

o A Client/Server network contains the centralized system. Therefore we can back up
the data easily.
o A Client/Server network has a dedicated server that improves the overall performance
of the whole system.
o Security is better in Client/Server network as a single server administers the shared
resources.
o It also increases the speed of the sharing resources.

Disadvantages Of Client/Server network:

o Client/Server network is expensive as it requires the server with large memory.
o A server has a Network Operating System(NOS) to provide the resources to the
clients, but the cost of NOS is very high.
o It requires a dedicated network administrator to manage all the resources.

Cache Coherence
The cache coherence protocol is discussed in this article as a solution to the
multicache inconsistency issues.

Cache Coherence
A cache coherence issue results from the concurrent operation of several processors
and the possibility that various caches may hold different versions of the identical
memory block. The practice of cache coherence makes sure that alterations in the
contents of associated operands are quickly transmitted across the system.

The cache coherence problem is the issue that arises when several copies of the
same data are kept at various levels of memory.
Cache coherence has three different levels:
o Each writing operation seems to happen instantly.
o Each operand's value changes are seen in every processor in precisely the same
order.
o Non-coherent behavior results from many processors interpreting the same action in
various ways.
Methods to resolve Cache Coherence
The two methods listed below can be used to resolve the cache coherence issue:

o Write Through
o Write Back

Write Through
The easiest and most popular method is to write through. Every memory write
operation updates the main memory. If the word is present in the cache memory at
the requested address, the cache memory is also updated simultaneously with the
main memory.

PauseNext

Unmute

Current Time 9:21

Duration 18:10

Loaded: 57.24%

Fullscreen

The benefit of this approach is that the RAM and cache always hold the same
information. In systems with direct memory access transfer, this quality is crucial. It
makes sure the information in the main memory is up-to-date at all times so that a
device interacting over DNA can access the most recent information.

Advantage - It provides the highest level of consistency.

Disadvantage - It requires a greater number of memory access.

Write Back
Only the catch location is changed during a write operation in this approach. When
the word is withdrawn from the cache, the place is flagged, so it is replicated in the
main memory. The right-back approach was developed because words may be
updated numerous times while they are in the cache. However, as long as they are
still there, it doesn't matter whether the copy that is stored in the main memory is
outdated because requests for words are fulfilled from the cache.

An accurate copy must only be transferred back to the main memory when the word
is separated from the cache. According to the analytical findings, between 10% and
30% of all memory references in a normal program are written into memory.

Advantage - A very small number of memory accesses and write operations.

Disadvantage - Inconsistency may occur in this approach.

The important terms related to the data or information stored in the cache as well as
in the main memory are as follows:

o Modified - The modified term signifies that the data stored in the cache and main
memory are different. This means the data in the cache has been modified, and the
changes need to be reflected in the main memory.
o Exclusive - The exclusive term signifies that the data is clean, i.e., the cache and the
main memory hold identical data.
o Shared - Shared refers to the fact that the cache value contains the most current data
copy, which is then shared across the whole cache as well as main memory.
o Owned - The owned term indicates that the block is currently held by the cache and
that it has acquired ownership of it, i.e., complete privileges to that specific block.
o Invalid - When a cache block is marked as invalid, it means that it needs to be
fetched from another cache or main memory.

Below is a list of the different Cache Coherence Protocols used in multiprocessor

systems:

o MSI protocol (Modified, Shared, Invalid)

o MOSI protocol (Modified, Owned, Shared, Invalid)
o MESI protocol (Modified, Exclusive, Shared, Invalid)
o MOESI protocol (Modified, Owned, Exclusive, Shared, Invalid)

These protocols are discussed below:

1. MSI Protocol
This is a fundamental cache coherence mechanism that is utilized in multiprocessor
systems. A cache may be in any state indicated by the protocol name's letters.
Therefore, for MSI, each block may be in one of the following states:

o Modified - In other words, the data in the cache is incompatible with the main
memory, and this status denotes that the block has been updated in the cache.
Therefore, when the data from the cache block is removed and is in the Modified (M)
state, the cache is responsible for writing the block to the main memory.
o Shared - At least one cache has at least one copy of this block, which has not been
updated. The cache might be removed without writing the data to the backup store.
o Invalid - If this block is going to be stored in this cache, it must be obtained from
RAM or from a different cache because it is invalid.

2. MOSI Protocol

It has one extra state than the MSI protocol, which is discussed below:

Owned - It is used to signify the ownership of the current processor to this block and
will respond to inquiries if another processor wants this block.

3. MESI Protocol

The protocol for cache coherence that is utilized the most is this one. Each cache line
bears a status indicating one of the following:

o Modified - As mentioned above, this term signifies that the data stored in the cache
and main memory are different. This means the data in the cache has been modified,
and the changes need to be reflected in the main memory.
o Exclusive - The exclusive term signifies that the data is clean, i.e., the cache and the
main memory hold identical data.
o Shared - This signifies that other caches on the computer may also hold this cache
line.
o Invalid - This indicates that this cache line is marked as invalid by the word "invalid."

4. MOESI Protocol

This protocol provides comprehensive cache coherence, covering all potential states
that are frequently utilized in other protocols. There are one of the following statuses
for each cache line:
 o Modified - While the copy in main memory is inaccurate and no other processors are
holding copies, a cache line in this condition contains the most recent, accurate copy
of the data.
o Owned - The most current, accurate copy of the data is stored in a cache line in this
state. In that other processors can store copies of the most recent, accurate data
comparable to the shared state; unlike the shared state, copies in main memory can
be inaccurate. One processor can only own the data at a time, and the remaining
processor can have the data in the shared state.
o Exclusive - The most current, accurate copy of the data is stored on a cache line in
this state. Since no other storage location has a copy of the data, the ram copy is also
the most recent and accurate copy.
o Shared - The most current, accurate copy of the data is stored on a cache line in this
condition. Additional system processors may also store data copies in the shared
state. If no other processor has ownership of the data, the copy in primary memory
also represents the most recent and accurate version of the data.
o Invalid - In this situation, a cache line doesn't contain a reliable copy of the data.
Still, valid data can be found in primary memory or another processor's cache.

Types of Coherence:
There exist three varieties of coherence referred to the coherency mechanisms, which
are listed below:

1. Directory Based - A directory-based system keeps the coherence amongst caches by

storing shared data in a single directory. In order to load an entry from primary
memory into its cache, the processor must request permission through the directory,
which serves as a filter. The directory either upgrades or devalues the other caches
that contain that record when a record is modified.
2. Snooping - Individual caches watch address lines during the snooping process to
look for accesses to memory locations that they have cached. A write invalidate
protocol is what it is known as. When a write activity is seen to a memory address for
which a cache maintains a copy, the cache controller invalidates its own copy of the
snooped memory location.
3. Snarfing - A cache controller uses this approach to try and update its own copy of a
memory location when a second master alters a place in the main memory by
keeping an eye on both the address and the contents. The cache controller updates
its own copy of the underlying memory location with the new data when a write
action is detected to a place of which a cache holds a copy.