Concept of Pipelining

Basic
• A means of introducing parallelism into the essentially sequential
nature of a machine-instruction program. Examples are instruction
pipelining and vector processing.
• To improve the performance of a CPU we have two options:
1) Improve the hardware by introducing faster circuits.
2) Arrange the hardware such that more than one operation can be
performed at the same time. Since there is a limit on the speed of
hardware and the cost of faster circuits is quite high.
Types of Pipeline
• It is divided into 2 categories:
1. Arithmetic Pipeline
2. Instruction Pipeline
Arithmetic Pipeline
• Arithmetic pipelines are usually found in most of the computers. They
are used for floating point operations, multiplication of fixed point
numbers etc
Instruction Pipeline
• In this a stream of instructions can be executed by overlapping fetch,
decode and execute phases of an instruction cycle. This type of
technique is used to increase the throughput of the computer system.

• An instruction pipeline reads instruction from the memory while

previous instructions are being executed in other segments of the
pipeline. Thus we can execute multiple instructions simultaneously.
The pipeline will be more efficient if the instruction cycle is divided
into segments of equal duration.
Pipelining 
• Pipelining is a process of arrangement of hardware elements of the CPU such that its overall
performance is increased.
• Simultaneous execution of more than one instruction takes place in a pipelined processor.
• Let us see a real-life example that works on the concept of pipelined operation.
• Consider a water bottle packaging plant.
• Let there be 3 stages that a bottle should pass through,
• Inserting the bottle(I), Filling water in the bottle(F), and Sealing the bottle(S).
• Let us consider these stages as stage 1, stage 2, and stage 3 respectively.
• Let each stage take 1 minute to complete its operation.
• Now, in a non-pipelined operation, a bottle is first inserted in the plant, after 1 minute it is
moved to stage 2 where water is filled. Now, in stage 1 nothing is happening. Similarly, when the
bottle moves to stage 3, both stage 1 and stage 2 are idle. But in pipelined operation, when the
bottle is in stage 2, another bottle can be loaded at stage 1. Similarly, when the bottle is in stage
3, there can be one bottle each in stage 1 and stage 2. So, after each minute, we get a new bottle
at the end of stage 3. Hence, the average time taken to manufacture 1 bottle is: 
Design of a basic pipeline

• In a pipelined processor, a pipeline has two ends, the input end and
the output end. Between these ends, there are multiple
stages/segments such that the output of one stage is connected to
the input of the next stage and each stage performs a specific
operation.
• Interface registers are used to hold the intermediate output between
two stages. These interface registers are also called latch or buffer.
• All the stages in the pipeline along with the interface registers are
controlled by a common clock.
Execution in a pipelined processor
•  Execution sequence of instructions in a pipelined processor can be
visualized using a space-time diagram. For example, consider a
processor having 4 stages and let there be 2 instructions to be
executed. We can visualize the execution sequence through the
following space-time diagrams: 
• 5 steps for instruction execution
• 5 stages pipeline
• 1. IF – Instruction fetch
• 2. ID – Instruction decode and register fetch
• 3. EXE – Execution or address calculation
• 4. MEM – Data memory access
• 5. WB – Write back
Pipeline Conflicts
• There are some factors that cause the pipeline to deviate its normal
performance. Some of these factors are given below:

1. Timing Variations :All stages cannot take same amount of time. This
problem generally occurs in instruction processing where different
instructions have different operand requirements and thus different
processing time.
2. Data Hazards :When several instructions are in partial execution, and if
they reference same data then the problem arises. We must ensure that
next instruction does not attempt to access data before the current
instruction, because this will lead to incorrect results.
3. Branching: In order to fetch and execute the next instruction, we
must know what that instruction is. If the present instruction is a
conditional branch, and its result will lead us to the next instruction,
then the next instruction may not be known until the current one is
processed.
4. Interrupts : set unwanted instruction into the instruction stream.
Interrupts effect the execution of instruction.
5. Data Dependency: It arises when an instruction depends upon the
result of a previous instruction but this result is not yet available.
Advantages of Pipelining
1. The cycle time of the processor is reduced.
2. It increases the throughput of the system
3. It makes the system reliable
• A Scalar processor is a normal processor, which works on simple instruction at a time, which operates on
single data items. But in today's world, this technique will prove to be highly inefficient, as the overall
processing of instructions will be very slow.
• What is Vector(Array) Processing? There is a class of computational problems that are beyond the
capabilities of a conventional computer. These problems require vast number of computations on
multiple data items, that will take a conventional computer(with scalar processor) days or even weeks to
complete. Such complex instructions, which operates on multiple data at the same time, requires a better
way of instruction execution, which was achieved by Vector processors.
• Scalar CPUs can manipulate one or two data items at a time, which is not very efficient. Also, simple
instructions like ADD A to B, and store into C are not practically efficient. Addresses are used to point to
the memory location where the data to be operated will be found, which leads to added overhead of
data lookup. So until the data is found, the CPU would be sitting ideal, which is a big performance issue.
Hence, the concept of Instruction Pipeline comes into picture, in which the instruction passes through
several sub-units in turn. These sub-units perform various independent functions, for example: the first
one decodes the instruction, the second sub-unit fetches the data and the third sub-unit performs the
math itself. Therefore, while the data is fetched for one instruction, CPU does not sit idle, it rather works
on decoding the next instruction set, ending up working like an assembly line.
• Vector processor, not only use Instruction pipeline, but it also pipelines the data, working on multiple
data at the same time. A normal scalar processor instruction would be ADD A, B, which leads to addition
of two operands, but what if we can instruct the processor to ADD a group of numbers(from 0 to n
memory location) to another group of numbers(lets say, n to k memory location). This can be achieved by
vector processors. In vector processor a single instruction, can ask for multiple data operations, which
saves time, as instruction is decoded once, and then it keeps on operating on different data items.
RISC(Reduced instruction set computing) PIPELINING
• Instruction pipelining is often used to enhance performance.
• Let us reconsider this in the context of a RISC architecture. Most instructions
• are register to register, and an instruction cycle has the following two stages:
• • I: Instruction fetch.
• • E: Execute. Performs an ALU operation with register input and output.
• For load and store operations, three stages are required:
• • I: Instruction fetch.
• • E: Execute. Calculates memory address
• • D: Memory. Register-to-memory or memory-to-register operation.