Scribd
Upload
48 views

MapReduce BigData 09

MapReduce is a distributed processing technique and programming model for processing large datasets across clusters of computers. It contains two main functions: Map, which filters and distributes work, and Reduce, which aggregates the results into a final answer. MapReduce allows developers to write applications that process vast amounts of data in parallel on large clusters of commodity hardware reliably.

Uploaded by

Seikh Sadi
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
48 views

MapReduce BigData 09

MapReduce is a distributed processing technique and programming model for processing large datasets across clusters of computers. It contains two main functions: Map, which filters and distributes work, and Reduce, which aggregates the results into a final answer. MapReduce allows developers to write applications that process vast amounts of data in parallel on large clusters of commodity hardware reliably.

Uploaded by

Seikh Sadi
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 9

MapReduce

Presented By:
Aashraful Islam Souraov
(ID 2025351009)
What is MapReduce? ①

• MapReduce is a processing technique and a program model for


distributed computing based on java.
• The MapReduce algorithm contains two important tasks, namely
Map and Reduce.
• Developer: Google ①
Why do we use MapReduce?

• MapReduce is a framework using which we can write applications


to process huge amounts of data, in parallel, on large clusters of
commodity hardware in a reliable manner.
What is the purpose of MapReduce?

• MapReduce serves two essential functions:


1. It filters and parcels out work to various nodes within the cluster or
map, a function sometimes referred to as the mapper,
2. It organizes and reduces the results from each node into a cohesive
answer to a query, referred to as the reducer.
The Algorithm of MapReduce
• Generally MapReduce paradigm is based on sending the computer to where
the data resides!
• MapReduce program executes in three stages, namely map stage, shuffle
stage, and reduce stage.
• Map stage − The map or mapper’s job is to process the input data. Generally the input
data is in the form of file or directory and is stored in the Hadoop file system (HDFS).
The input file is passed to the mapper function line by line. The mapper processes the
data and creates several small chunks of data.
• Reduce stage − This stage is the combination of the Shuffle stage and
the Reduce stage. The Reducer’s job is to process the data that comes from the
mapper. After processing, it produces a new set of output, which will be stored in the
HDFS.
The Algorithm of MapReduce

• During a MapReduce job, Hadoop sends the Map and Reduce tasks to the
appropriate servers in the cluster.
• The framework manages all the details of data-passing such as issuing tasks,
verifying task completion, and copying data around the cluster between the
nodes.
• Most of the computing takes place on nodes with data on local disks that
reduces the network traffic.
• After completion of the given tasks, the cluster collects and reduces the data to
form an appropriate result, and sends it back to the Hadoop server.
What are the Advantages & Disadvantages of
MapReduce?

Advantages:
• Scalable (due to simple design)
• Runs on cheap commodity hardware
• Procedural control i.e. we can control of the execution of every step
What are the Advantages & Disadvantages of
MapReduce?

Disadvantages:
• It is not flexible i.e. the MapReduce framework is rigid.
• A lot of manual coding is required, even for common operations such as
join, filter, projection, aggregates, sorting, distinct.
• Semantics are hidden inside the map and reduce functions, so it is difficult
to maintain, extend and optimize them

You might also like