INTRODUCTION
TO BIG DATA
ANALYTICS
Author : FU
Date : Mar-2022
Objectives
After studying this chapter, the student should be able to
understand the key concepts:
Clarify what is meant by Big Data?
Why advanced analytics are needed?
How Data Science differs from Business Intelligence (BI)?
What new roles are needed for the new Big Data ecosystem?
Content
1. Big Data Overview
2. State of the Practice in Analytics
3. Key Roles for the New Big Data Ecosystem
4. Examples of Big Data Analytics
1. Big Data Overview – Where Big Data comes?
Data created constantly, and at an ever-increasing rate. Mobile phones,
social media, Imaging technologies … create new data, stored some
where for some purpose. Devices and sensors automatically generate
diagnostic information that needs to be stored and processed in real time.
Merely keeping up with this huge influx of data is difficult, more
challenging in analyzing vast amount of data, especially non-conform
traditional data structure
Challenges of data deluge present the opportunity to transform business,
government, science, and everyday life
Several industries led the way in the ability to gather and exploit data:
o Credit card companies
o Mobile phone companies
o For companies such as LinkedIn and Facebook
1. Big Data Overview – What is Big data?
Three attributes stand out as defining Big Data characteristics:
o Huge volume of data
o Complexity of data types and structures (variety of new data sources, formats,
and structures)
o Speed of new data creation and growth
Big Data is sometimes described as having 3 Vs:
o volume,
o variety,
o and velocity
Another definition of Big Data comes from the McKinsey Global
report from 2011:
o Big Data is data whose scale, distribution, diversity, and/or timeliness
require the use of new technical architectures and analytics to enable
insights that unlock new sources of business value.
1. Big Data Overview – What methods and
tools used for Big data analysis?
Cannot be efficiently analyzed using only traditional databases
or methods.
Require new tools and technologies to store, manage, and
realize the benefits
New tools and technologies enable creation, manipulation, and
management of large datasets and the storage environments
McKinsey’s definition of Big Data implies that organizations will
need new data architectures and analytic sandboxes, new
tools, new analytical methods, and an integration of multiple
skills into the new role of the data scientist
1. Big Data Overview – What’s Driving Data
Deluge?
Several sources of the Big Data deluge. The rate of data
creation is accelerating, driven by many of the items in Figure
1-1.
1. Big Data Overview - Fastest-growing sources of
Big Data?
Social media and genetic sequencing are
among the fastest-growing sources of Big Data
o Social media data
2012 Facebook users posted 700 status updates per second
worldwide
Facebook construct social graphs to analyze users data
o Genetic sequencing
Genetic sequencing and human genome mapping provide a detailed
understanding of genetic makeup and lineage
Health care industry is looking toward these advances to help
predict which illnesses a person is likely to get in his lifetime and
take steps to avoid these maladies or reduce their impact through
the use of personalized medicine and treatment
1.1 Data Structures
Big data forms:
structured and non-
structured data
(financial data, text files,
multimedia files, and
genetic mappings)
Most of the Big Data
is unstructured or
semi-structured,
requires different
techniques and tools
to process and
analyze
1.1 Data Structures – Data types
Structured data: Data containing a defined data type,
format, and structure
Semi-structured data: Textual data files with a discernible
pattern that enables parsing (such as Extensible Markup
Language [XML] data files that are self-describing and defined
by an XML schema)
Quasi-structured data: Textual data with erratic data formats
that can be formatted with effort, tools, and time (for instance,
web clickstream data that may contain inconsistencies in data
values and formats)
Unstructured data: Data that has no inherent structure,
which may include text documents, PDFs, images, and video
1.2 Analyst Perspective on Data Repositories
2. State of the Practice in Analytics
Business problems provide many opportunities for
organizations to become more analytical and data driven, as
shown in Table 1-2
2.1 BI versus Data Science - What is BI?
Several ways to compare these groups of analytical techniques
BI tends to provide reports, dashboards, and queries on business
questions (closed-ended and explain current or past behavioral) for the
current period or in the past
BI systems used to answer questions related to quarter-to-date
revenue, progress toward quarterly targets, and understand how much
of a given product was sold in a prior quarter or year
BI provides hindsight and some insight and generally answers
questions related to “when” and “where” events occurred
BI problems tend to require highly structured data organized in rows
and columns for accurate reporting
2.1 BI versus Data Science - What is Data Science?
Use disaggregated data in a more forward-looking,
exploratory way, focusing on analyzing the present
and enabling informed decisions about the future.
Be more exploratory in nature and may use scenario
optimization to deal with more open-ended questions,
focusing on questions related to “how” and “why”
events occur.
Data Science projects tend to use many types of data
sources, including large or unconventional datasets
2.1 BI versus Data Science - Summary
2.2 Analytical Architecture
FIGURE 1-9 Typical analytic architecture
2.3 Drivers of Big Data
Data now comes
from multiple
sources:
o Medical information
o Photos and video
o Video surveillance
o Mobile devices
o Smart devices
o Nontraditional IT
devices
FIGURE 1-10 Data evolution and the rise of Big Data sources
2.4 Emerging Big Data Ecosystem and a New
Approach to Analytics
New ecosystem
takes shape,
there are four
main groups of
players within
this
interconnected
web
o Data devices
o Data collectors
o Data
aggregators
o Data users and FIGURE 1-11 Emerging Big Data ecosystem
3. Key Roles for the New Big Data Ecosystem (1)
FIGURE 1-12 Key
roles of the new
Big Data
ecosystem
3. Key Roles for the New Big Data Ecosystem (2)
Three recurring sets of activities for data scientists
o Reframe business challenges as analytics challenges
o Design, implement, and deploy statistical models and data mining
techniques on Big Data
o Develop insights that lead to actionable recommendations
Required main skills for data scientists
o Quantitative skill
o Technical aptitude
o Skeptical mind-set and critical thinking
o Curious and creative
o Communicative and collaborative:
FIGURE 1-13 Profile of a Data Scientist
4. Examples of Big Data Analytics
An example of this is the U.S. retailer Target. Charles Duhigg’s book The Power
of Habit discusses how Target used Big Data and advanced analytical
methods to drive new revenue. After analyzing consumer purchasing
behavior, Target’s statisticians determined that the retailer made a great deal
of money from three main life-event situations.
o Marriage, when people tend to buy many new products
o Divorce, when people buy new products and change their spending habits
o Pregnancy, when people have many new things to buy and have an urgency to buy them
Hadoop represents another example of Big Data innovation on the IT
infrastructure. Apache Hadoop is an open source framework that allows
companies to process vast amounts of information in a highly parallelized way
Social media e.g. LinkedIn (250 million user accounts) represents a
tremendous opportunity to leverage social and professional interactions to
derive new insights
Summary
Big Data comes from myriad sources, including social media,
sensors, the Internet of Things, video surveillance, and many
sources of data
Organizations evolve their processes and see the opportunities from
Big Data. Move beyond traditional BI activities, such as using data to
populate reports and dashboards, and move toward Data Science-
driven projects that attempt to answer more open-ended and
complex questions.
Big Data presents requires new data architectures, including analytic
sandboxes, new ways of working, and people with new skill sets
Need to build Data Science team. Growing talent gap that makes
finding and hiring data scientists in a timely manner difficult
