NoSQL

1
 Big Data (some old numbers)
• Facebook:
 130TB/day: user logs
 200-400TB/day: 83 million pictures

• Google: > 25 PB/day processed data

• Gene sequencing: 100M kilobases

per day per machine
 Sequence 1 human cell costs Illumina $1k
 Sequence 1 cell for every infant
 10 trillion cells / human body

• Total data created in 2010: 1.ZettaByte

(1,000,000 PB)/year
 ~60% increase every year
2
 Big data is not only databases
• Big data is more about data analytics and
on-line querying

Many components:
• Storage systems
• Database systems
• Data mining and statistical algorithms
• Visualization

3
 What is NoSQL?
• An emerging “movement” around
non-relational software for Big Data
• Roots are in the Google and Amazon homegrown
software stacks

• Wikipedia: “A NoSQL database provides a mechanism for storage

and retrieval of data that use looser consistency models than
traditional relational databases in order to achieve
horizontal scaling and higher availability.
• Some authors refer to them as "Not only SQL" to emphasize that
some NoSQL systems do allow SQL-like query language to be
used.”
Some NoSQL Components

Analytics Interface Imperative Lang

(Pig, Hive, …) (RoR, Java,Scala, …)

Data Parallel Processing

(MapReduce/Hadoop)
Distributed Key/Value or Column
Store
(Cassandra, Hbase, Redis, …)
Scalable File System
(GFS, HDFS, …)

5
NoSQL features
• Scalability is crucial!
 load increased rapidly for many applications
• Large servers are expensive

• Solution: use clusters of small

commodity machines
 need to partition the data and use
replication (sharding)
 cheap (usually open source!)
 cloud-based storage

6
 NoSQL features
• Sometimes not a well defined schema

• Allow for semi-structured data

 still need to provide ways to query
efficiently
(use of index methods)
 need to express specific types of queries
easily

7
 Flavors of NoSQL

Four main types:

• key-value stores
• document databases
• column-family (big-table) stores
• graph databases

=>Here we will talk more about

“Document” databases (MongoDB)

10
 Key-Value Stores

There are many systems like that: Redis,

MemcacheDB, Amazon's DynamoDB,
Voldemort

• Simple data model: key/value pairs

• the DBMS does not attempt to interpret the
value

• Queries are limited to query by key

• get/put/update/delete a key/value pair
• iterate over key/value pairs

11
 Document Databases
Examples include: MongoDB, CouchDB, Terrastore

• Also store key/value pairs

- However, the value is a document.
• expressed using some sort of semi-structured data model
• XML
• more often: JSON or BSON (JSON's binary counterpart)
• the value can be examined and used by the DBMS (unlike
in key/ data stores)
• Queries can be based on the key (as in key/value
stores), but more often they are based on the
contents of the document.

• Here again, there is support for sharding and

replication.
• the sharding can be based on values within the
document 12
 The Structure Spectrum

Structured Semi- Unstructure

(schema- Structured d (schema-
first) (schema-later) never)

Relational Documents Plain Text

Database XML
Media
Formatted Tagged
Messages Text/Media
MongoDB (An example of a
Document Database)
-Data are organized in collections. A collection
stores a set of documents.
- Collection like table and document like
record
- but: each document can have a different set
of attributes even in the same collection
- Semi-structured schema!
- Only requirement: every document should
have an “_id” field

14
 Example mongodb

{ "_id”:ObjectId("4efa8d2b7d284dad101e4bc9"),
"Last Name": ” Cousteau",
"First Name": ” Jacques-Yves",
"Date of Birth": ”06-1-1910" },

{ "_id": ObjectId("4efa8d2b7d284dad101e4bc7"),
"Last Name": "PELLERIN",
"First Name": "Franck",
"Date of Birth": "09-19-1983",
"Address": "1 chemin des Loges",
"City": "VERSAILLES" }

15
 Example Document Database:
MongoDB
Key features include:
• JSON-style documents
• actually uses BSON (JSON's binary
format)
• replication for high availability
• auto-sharding for scalability
• document-based queries
• can create an index on any attribute
• for faster reads

16
 MongoDB Terminology
relational term <== >MongoDB equivalent
----------------------------------------------------------
database <== > database
table <== > collection
row <== > document
attributes <== > fields (field-name:value pairs)
primary key <== > the _id field, which is the key
associated with the document

17
 JSON
• JSON is an alternative data model for
semi-structured data.
• JavaScript Object Notation

• Built on two key structures:

• an object, which is a sequence of name/value pairs
{ ”_id": "1000",
"name": "Sanders Theatre",
"capacity": 1000 }
• an array of values [ "123", "222", "333" ]
• A value can be:
• an atomic value: string, number, true,
false, null
• an object
• an array
18
 The _id Field
Every MongoDB document must have an _id
field.
• its value must be unique within the
collection
• acts as the primary key of the collection
• it is the key in the key/value pair
• If you create a document without an _id field:
• MongoDB adds the field for you
• assigns it a unique BSON ObjectID
• example from the MongoDB shell:
> db.test.save({ rating: "PG-13" })
> db.test.find()
{ "_id" :ObjectId("528bf38ce6d3df97b49a0569"), "rating" : "PG-
13" }
19
 Data Modeling in MongoDB
Need to determine how to map
entities and relationships => collections of
documents
• Could in theory give each type of entity:
• its own (flexibly formatted) type of document
• those documents would be stored in the same
collection
• However, it can make sense to group different
types of entities together.
• create an aggregate containing data
that tends to be accessed together

20
 Capturing Relationships in
MongoDB
• Two options:
 1. store references to other documents
using their _id values

 2. embed documents within other

documents

21
 Example relationships
Consider the following documents examples:
{ {
"_id":ObjectId("52ffc33cd85242f436000001"), "_id":ObjectId("52ffc4a5d85242602e000000"),
"name": "Tom Hanks", "building": "22 A, Indiana Apt",
"contact": "987654321", "pincode": 123456,
"dob": "01-01-1991" "city": "Los Angeles",
} "state": "California"
}

Here is an example of embedded relationship:

{
"_id":ObjectId("52ffc33cd85242f436000001"),
"contact": "987654321",
"dob": "01-01-1991",
"name": "Tom Benzamin", And here an example of reference based
"address": [
{ {
"building": "22 A, Indiana Apt",
"_id":ObjectId("52ffc33cd85242f436000001"),
"pincode": 123456,
"city": "Los Angeles", "contact": "987654321",
"state": "California" "dob": "01-01-1991",
}, "name": "Tom Benzamin",
{ "address_ids": [
"building": "170 A, Acropolis Apt", ObjectId("52ffc4a5d85242602e000000"),
"pincode": 456789, ObjectId("52ffc4a5d85242602e000001")
"city": "Chicago",
]
"state": "Illinois"
} }
]
}
22
 Other Structure Issues
• NoSQL: a) Tables are unnatural, b) “joins” are
evil, c) need to be able to “grep” my data

• DB: a) Tables are a natural/neutral structure,

b) data independence lets you precompute
joins under the covers, c) this is a price of all
the DBMS goodness you get

This is an Old Debate – Object-oriented

databases, XML DBs, Hierarchical, …

23