Scribd
Upload
28 views

02-17 Object Storage Overview

Block storage stores and manages data in blocks which can be directly accessed via low-level protocols, providing similar performance to local disk access. File storage organizes data into a hierarchical structure of files and folders. Object storage stores flexible sized objects with custom metadata in a flat structure across multiple locations at massive scale for low cost.

Uploaded by

Natalia Pardo
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
28 views

02-17 Object Storage Overview

Block storage stores and manages data in blocks which can be directly accessed via low-level protocols, providing similar performance to local disk access. File storage organizes data into a hierarchical structure of files and folders. Object storage stores flexible sized objects with custom metadata in a flat structure across multiple locations at massive scale for low cost.

Uploaded by

Natalia Pardo
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 26

Block Storage

Block storage stores and manages data in blocks which are accessed
via low-level storage protocols using SCSI and NVMe commands.
SAN protocols provide block access over a network. The experience for
the user and applications is similar to accessing a local disk.
Block Storage
The direct access to the data reduces overhead by minimizing
abstraction layers.
Higher level tasks such as multi-user access, sharing, locking and
security are usually handled by the operating system.
Block Storage Metadata
There is no storage-side metadata associated with the block, only its
address.
The block is simply a chunk of data that has no description, no
association and no owner.
Block Storage Use Cases
Block storage is considered the best solution for performance
sensitive, transactional, and database oriented applications.
It is mostly used for primary storage and with the client and storage
system both located in the same physical location. Adding distance
between the application and storage harms performance.
File Storage
File storage, used by NAS protocols, stores data as a file hierarchy in a
file system.
The hierarchy is similar to a physical file cabinet with folders (or
‘directories’) and subfolders.
The user or application connects to the file system through a share
(CIFS/SMB) or by mounting an export (NFS).
File Storage Metadata
File system metadata is recorded separately from the file itself and
records basic file attributes such as file name, creation date, creator,
file type, most recent change and last access.
The metadata is fixed and standardized to the file system.
Adding custom metadata (extended attributes) requires a custom
application and database.
File Storage Use Cases
File storage is well suited to general purpose data, especially data
which is edited frequently and concurrently by multiple users or
applications.
It is designed to be accessed over both the local network and
remotely.
Block and File Storage Limitations
Block and file storage systems can be scaled out by adding more disks
and nodes but they are typically limited in scale to a single geographic
location physically.
NAS file index tables (inodes) have a maximum size and can affect
performance if they grow too large.
Block and file stores need to be backed up offsite for resiliency.
Object Storage
Object storage organizes information into containers of flexible sizes,
referred to as objects.
Objects are stored and managed in a flat organisation of flexibly sized
containers called buckets or containers.
Buckets can span multiple nodes and geographic locations.
Object Storage Scalability
Object based storage architectures can be scaled out and managed
simply by adding additional nodes which can be across multiple
locations.
It is very commonly offered by cloud providers, but on-premises and
hybrid solutions (where objects can be stored both on-premises and in
the cloud) are also available.
Object Storage

Every object contains three things:


The data itself.
Customisable metadata.
A globally unique identifier.
Object Storage Metadata
Customisable metadata links directly with the object and contains
additional descriptive properties.
This can be used for better indexing, for example by including ‘black’
and ‘cat’ tags in a video object, or the patient’s name and injury type
in an x-ray.
Object Storage Metadata (Cont.)
It can also be used for better management, for example indicating
replication instructions, when to move to a different tier of storage or
location, and when to delete.
Globally Unique Identifier
A globally unique identifier is used instead of a file name and path as in NAS.
This address is unique across the namespace, and is used to find the object
over the distributed system without having to know the physical location of
the data.
This removes the complexity and scalability challenges of a NAS hierarchical
file system based on complex file paths.
Data Protection
Object Stores provide resiliency through replication and/or erasure
coding.
Replication is used to store objects on multiple nodes which can be
geographically dispersed. This is most suitable for small files.
Data Protection (Cont.)
With erasure coding the object is broken up into smaller distributed
parts. Parity information allows the data to be reconstructed if there is
node failure. This is suitable for larger files.
If one or more nodes fail, the data can still be made available without
the application or the end user being impacted.
Object Storage Versioning
Object Storage supports versioning, where the old version of the object
can be automatically saved if it is changed.
It does not typically support file locking and files can’t be updated in place
(this makes the simple data protection possible).
If multiple users update the same object concurrently, the system will
simply write different versions of the object.
Object Storage Versioning (Cont.)
It is designed for Puts and Gets, not for data which will have multiple
edits from multiple users such as transactional databases or Word
documents.
This has traditionally made it more suitable for secondary (backup and
archive) rather than primary data.
Cost
Object storage is typically used as secondary storage where
performance is not a priority.
It is usually one of the lowest cost storage options from a cloud provider.
On-premises Object Storage platforms can typically be bought as an
appliance or software only.
Lower cost commodity hardware can be used for the underlying
infrastructure on a software only purchase.
Object Storage Protocols
Object Storage uses RESTful APIs that use HTTP requests to GET, PUT,
POST and DELETE data, and HEAD to request metadata information.
Open Standard API examples:
CDMI, the Cloud Data Management Interface from the Storage
Networking Industry Association (SNIA)
Amazon Web Services Simple Storage Service (S3)
OpenStack Swift
Object Storage Protocols
End users can access the storage directly via a web interface or
application such as Cyberduck.
Support for access via NAS protocols is typically also included, either
natively or by using a cloud gateway.
The cloud gateway can be a physical appliance or virtual machine.
Object Storage Benefits Summary
Single namespace with almost infinite scale
Scales across multiple physical locations
Performance does not degrade with scale
Customizable metadata for better indexing and management
Supports data management functions such as replication at object-level
granularity
Typically low cost
Object Storage Limitations Summary
Generally lower performance, so not suitable for databases or other
applications which require high performance.
Does not have locking and file-sharing facilities so not suitable for data
which may be accessed concurrently and changed by multiple users or
applications.
Object Storage Use Cases
Object storage is best suited as a massively scalable store for unstructured data
that’s updated infrequently.
It can be used as an additional storage tier beyond transactional storage for
inactive data, or as archival storage.
As object storage evolves it may become more suited to primary data.
It’s well suited for file content in the cloud space, especially images and videos.
Object Storage Use Case Examples

Media
Medical
Oil and Gas
Object Storage Examples
Cloud Providers:
Amazon Web Services Simple Storage Service (S3)
Microsoft Azure Blob Storage
Proprietary:
Facebook Haystack
On-Premises/Hybrid:
NetApp StorageGRID

You might also like