Using Big Data Lite Virtual Machine

Course Road Map

Lesson 2: Big Data and the Information

Module 1: Big Data Management System Management System

Module 2: Data Acquisition and Storage Lesson 3: Introduction to the Big Data
Lite VM and MoviePlex

Module 3: Data Access and Processing Lesson 4: Introduction to the Big Data
Ecosystem

Module 4: Data Unification and Analysis

Module 5: Using and Managing Big Data

Appliance

3-2
 Objectives

After completing this lesson, you should be able to use:

• Oracle Big Data Lite Virtual Machine
• The MoviePlex application

3-3
 Lesson Agenda

• Overview of Big Data Lite Virtual Machine

– Environment details
– Software installed
• Deep dive into the MoviePlex case study

3-4
 Oracle Big Data Lite Virtual Machine:
Introduction
Provides an integrated environment to help you get started with
the Oracle Big Data platform

3-5
 Oracle Big Data Lite 4.0.1 VM Components

Cloudera’s Distribution
Oracle Enterprise Oracle Database 12c Release 1
Including Apache Hadoop
Linux 6.4 Enterprise Edition (12.1.0.2)
(CDH 5.1.2)

Oracle JDeveloper Cloudera

Oracle Big Data Connectors 4.0
12c (12.1.3) Manager (5.1.2)
• Oracle SQL Connector for HDFS 3.1.0
Oracle SQL • Oracle Loader for Hadoop 3.2.0
Oracle R
Developer and • Oracle Data Integrator 12c
Distribution 3.1.1
Data Modeler 4.0.3 • Oracle R Advanced Analytics for Hadoop
2.4.1
• Oracle XQuery for Hadoop 4.0.1 Oracle Perfect
Oracle Big Data Balance 2.2
SQL

Oracle NoSQL Database Oracle Data Integrator

Oracle GolderGate 12c
Enterprise Edition 12cR1 (3.0.14) 12cR1 (12.1.3)

3-6
 Oracle Big Data Lite 4.0.1 Services

3-7
 Initializing the Environment for
the Big Data Lite VM
Steps Commands
1. Start VM and log in. id/password: welcome1

2. Download and install RStudio cd /home/oracle/scripts

(perform this task only once). ./install_rstudio.sh
(uncomment proxy setting in shell script if behind Oracle
firewall)
3. Open a Terminal window. Right-click on the desktop and select Open Terminal.
4. Go to the moviedemo home. cd movie/moviedemo/scripts

5. Start Oracle NoSQL DB and the First time (or every time you want to initialize Oracle
Movie Application. NoSQL DB). This will take several minutes:
./1_start_movieapp.sh

3-8
 Initializing the Environment

Steps Commands
6. Start Hue. ./2_start_hue.sh

7. Start Flume. Minimize the Flume Window. Show ./3_flume_tail_movielog_master

the App Log. .sh
8. Start the Oracle NoSQL DB Console App. ./4_nosqldb_console.sh

9. Start Firefox by clicking the web browser icon.

10. Start the movie application by going to the Oracle

MoviePlex bookmark (guest1/welcome1).
• In a second tab, connect to Hue by using
the bookmark (oracle/oracle).
• In a third tab, go to the Flume master by
using the bookmark under Hadoop Links.
• In a fourth tab, go to RStudio by using the
bookmark.

3-9
 Lesson Agenda

• Overview of Big Data Lite Virtual Machine

– Environment details
– Software installed
• Deep dive into the MoviePlex case study

3 - 10
 Oracle MoviePlex Case Study: Introduction

• Oracle MoviePlex is an application that is based on a

fictitious online movie streaming rental company.
• With this web-based application, you can do the following:
– Browse a catalog of movies.
– Watch movie trailers.
– Rent movies.
– Review and rank movies.

3 - 11
 Introduction

• Like many other online stores, they

needed a cost-effective approach to
tackle their “big data” challenges.
• They recently implemented Oracle Big
Data Platform to better manage their
business, identify key opportunities, and
enhance customer satisfaction.

3 - 12
 Big Data Challenge

• Applications are generating massive volumes of unstructured

data that describes user behavior and application performance.
• How can companies fully capitalize on this valuable information
due to cost and complexity?
• How do you use this raw data to gain better insights into your
customers, enhance their user experience, and ultimately
improve profitability?
{"custId":1185972,"movieId":null,"genreId":null,"time":"2012-07-01:00:00:07","recommended":null,"activity":8}
{"custId":1354924,"movieId":1948,"genreId":9,"time":"2012-07-01:00:00:22","recommended":"N","activity":7}
{"custId":1083711,"movieId":null,"genreId":null,"time":"2012-07-01:00:00:26","recommended":null,"activity":9}
{"custId":1234182,"movieId":11547,"genreId":44,"time":"2012-07-01:00:00:32","recommended":"Y","activity":7}
{"custId":1010220,"movieId":11547,"genreId":44,"time":"2012-07-01:00:00:42","recommended":"Y","activity":6}
{"custId":1143971,"movieId":null,"genreId":null,"time":"2012-07-01:00:00:43","recommended":null,"activity":8}
{"custId":1253676,"movieId":null,"genreId":null,"time":"2012-07-01:00:00:50","recommended":null,"activity":9}
{"custId":1351777,"movieId":608,"genreId":6,"time":"2012-07-01:00:01:03","recommended":"N","activity":7}
{"custId":1143971,"movieId":null,"genreId":null,"time":"2012-07-01:00:01:07","recommended":null,"activity":9}
{"custId":1363545,"movieId":27205,"genreId":9,"time":"2012-07-01:00:01:18","recommended":"Y","activity":7}
Capture users' clicks {"custId":1067283,"movieId":1124,"genreId":9,"time":"2012-07-01:00:01:26","recommended":"Y","activity":7}
{"custId":1126174,"movieId":16309,"genreId":9,"time":"2012-07-01:00:01:35","recommended":"N","activity":7}
{"custId":1234182,"movieId":11547,"genreId":44,"time":"2012-07-01:00:01:39","recommended":"Y","activity":7}
{"custId":1067283,"movieId":null,"genreId":null,"time":"2012-07-01:00:01:55","recommended":null,"activity":9}
{"custId":1377537,"movieId":null,"genreId":null,"time":"2012-07-01:00:01:58","recommended":null,"activity":9}
{"custId":1347836,"movieId":null,"genreId":null,"time":"2012-07-01:00:02:03","recommended":null,"activity":8}
{"custId":1137285,"movieId":null,"genreId":null,"time":"2012-07-01:00:03:39","recommended":null,"activity":8}
{"custId":1354924,"movieId":null,"genreId":null,"time":"2012-07-01:00:03:51","recommended":null,"activity":9}
{"custId":1036191,"movieId":null,"genreId":null,"time":"2012-07-01:00:03:55","recommended":null,"activity":8}
{"custId":1143971,"movieId":1017161,"genreId":44,"time":"2012-07-01:00:04:00","recommended":"Y","activity":7}
{"custId":1363545,"movieId":27205,"genreId":9,"time":"2012-07-01:00:04:03","recommended":"Y","activity":5}
{"custId":1273464,"movieId":null,"genreId":null,"time":"2012-07-01:00:04:39","recommended":null,"activity":9}
{"custId":1346299,"movieId":424,"genreId":1,"time":"2012-07-01:00:05:02","recommended":"Y","activity":4}

3 - 13
 Derive Value from Big Data

How can you:

• Make the right movie offers at the right time?
• Better understand the viewing trends of various customer
segments?
• Optimize marketing spend by targeting customers with
optimal promotional offers?
• Minimize infrastructure spend by understanding bandwidth
usage over time?
• Prepare to answer questions that you haven’t thought of
yet!

3 - 14
 Oracle MoviePlex: Goal

• To deliver a personalized movie-watching experience by

collecting and storing:
– User profiles
– Movie listings
– Ratings
– User’s viewing location within
paused movie
• All customer information and
session details are completely
fictitious.

3 - 15
 Oracle MoviePlex: Big Data Challenges

• The application generates a huge volume of unstructured

data.
• Requests require fast response times (measured in
milliseconds).
• Data point: Latency matters.

3 - 16
 MoviePlex: Architecture

Endeca Oracle
Log of all activity Information Business
Application on site Discovery Intelligence EE
Log
Customer Profile Oracle Exalytics
Capture activity required (e.g., recommended
for MoviePlex site movies)

Streamed
into HDFS Clustering/Market Basket
using Flume “Mood” Oracle Advanced
Oracle NoSQL DB
Recommendations Analytics
Load Recommendations Oracle Exadata

Load Session and

Activity Data
Oracle Big
HDFS Data
Connectors
MapReduce MapReduce MapReduce
ORCH - CF Recs. Pig - Sessionize Hive - Activities

Oracle Big Data Appliance

3 - 17
 MoviePlex: Data Generation

{"custId":1185972,"movieId":null,"genreId":null,"time":"2012-07-01:00:00:07","recommended":null,"activity":8}
{"custId":1354924,"movieId":1948,"genreId":9,"time":"2012-07-01:00:00:22","recommended":"N","activity":7}
{"custId":1083711,"movieId":null,"genreId":null,"time":"2012-07-01:00:00:26","recommended":null,"activity":9}
{"custId":1234182,"movieId":11547,"genreId":44,"time":"2012-07-01:00:00:32","recommended":"Y","activity":7}
{"custId":1010220,"movieId":11547,"genreId":44,"time":"2012-07-01:00:00:42","recommended":"Y","activity":6}
{"custId":1143971,"movieId":null,"genreId":null,"time":"2012-07-01:00:00:43","recommended":null,"activity":8}
{"custId":1253676,"movieId":null,"genreId":null,"time":"2012-07-01:00:00:50","recommended":null,"activity":9}
{"custId":1351777,"movieId":608,"genreId":6,"time":"2012-07-01:00:01:03","recommended":"N","activity":7}
{"custId":1143971,"movieId":null,"genreId":null,"time":"2012-07-01:00:01:07","recommended":null,"activity":9}
{"custId":1363545,"movieId":27205,"genreId":9,"time":"2012-07-01:00:01:18","recommended":"Y","activity":7}
{"custId":1067283,"movieId":1124,"genreId":9,"time":"2012-07-01:00:01:26","recommended":"Y","activity":7}
{"custId":1126174,"movieId":16309,"genreId":9,"time":"2012-07-01:00:01:35","recommended":"N","activity":7}
{"custId":1234182,"movieId":11547,"genreId":44,"time":"2012-07-01:00:01:39","recommended":"Y","activity":7}
{"custId":1067283,"movieId":null,"genreId":null,"time":"2012-07-01:00:01:55","recommended":null,"activity":9}
{"custId":1377537,"movieId":null,"genreId":null,"time":"2012-07-01:00:01:58","recommended":null,"activity":9}
{"custId":1347836,"movieId":null,"genreId":null,"time":"2012-07-01:00:02:03","recommended":null,"activity":8}
{"custId":1137285,"movieId":null,"genreId":null,"time":"2012-07-01:00:03:39","recommended":null,"activity":8}
{"custId":1354924,"movieId":null,"genreId":null,"time":"2012-07-01:00:03:51","recommended":null,"activity":9}
{"custId":1036191,"movieId":null,"genreId":null,"time":"2012-07-01:00:03:55","recommended":null,"activity":8}
{"custId":1143971,"movieId":1017161,"genreId":44,"time":"2012-07-01:00:04:00","recommended":"Y","activity":7}
{"custId":1363545,"movieId":27205,"genreId":9,"time":"2012-07-01:00:04:03","recommended":"Y","activity":5}
{"custId":1273464,"movieId":null,"genreId":null,"time":"2012-07-01:00:04:39","recommended":null,"activity":9}
{"custId":1346299,"movieId":424,"genreId":1,"time":"2012-07-01:00:05:02","recommended":"Y","activity":4}

3 - 18
 MoviePlex: Data Generation Format
Fields Description
custID The customer's ID
movieId The ID of the selected movie
genreId The genre of the selected movie
Time The timestamp when the customer watched the movie
recommended? Whether or not the selected movie recommended, Y or N
Activity • 1: Rate movie
• 2: Completed movie
• 3: Not completed
• 4: Started movie
• 5: Browsed movie
• 6: List movies
• 7: Search

{"custId":1354924,"movieId":1948,"genreId":9,"time":"201
2-07-01:00:00:22","recommended":"N","activity":7}

3 - 19
 MoviePlex Application
Advanced analytics: movies
Simple profile updates 2 based on mood
1

3
4 Advanced profile attributes
What is a key-value store?

3 - 20
 Summary

In this lesson, you should have learned how to use:

• Oracle Big Data Lite Virtual Machine
• The Oracle MoviePlex application

3 - 21