Big Data Storage IGATE

36 Pages·2014·2.47 MB·English

Big Data Storage
Apurva Vaidya
Anshul Kosarwal
IGATE

2014 Storage Developer Conference. © IGATE. All Rights Reserved.
Agenda
 Abstract
 Big Data and its characteristics
 Key requirements of Big Data Storage
 Triage and Analytics Framework
 Big Data Storage Choices
 Hyper scale, Scale Out NAS, Object Storage
 Impact of Flash memory and tiered storage technologies
 Big Data Storage Architecture
 Performance Optimization
 Conclusion
2014 Storage Developer Conference. © IGATE. All Rights Reserved.
Abstract
Big Data is a key buzzword in IT.
And for this,
proper storage
options should be
in place to handle
BIG DATA
The amount of data to be stored and
processed has exploded to a mind boggling
degree.
Although "Big Data Analytics" has evolved to
get a handle on this information content, the
data must be appropriately stored for easy

and efficient retrieval.

2014 Storage Developer Conference. © IGATE. All Rights Reserved.
“The pessimist – complains about the wind
The optimist – expects it to change
The leader – adjusts the sails”
John Maxwell

2014 Storage Developer Conference. © IGATE. All Rights Reserved.
Big Data & Its Characteristics
2014 Storage Developer Conference. © IGATE. All Rights Reserved.
Storage Capacity
LARGE
ANALOG
19 Exabytes

Y DIGITAL
T 280 Exabytes
I
C
A
P Beginning of the
A
C Digital age
E Global
G
A
R Information
O
T
S Storage Capacity
N
O ANALOG
I
T 94%
A 2.6 Exabytes
M 2017
R
O DIGITAL
7,235 Exabytes
F
N 0.02 Exabytes 50%
I
25%

3%
% Digital
SMALL 1%
1986 1993 2000 2002 2007
Source: Business Computing World
(http://www.businesscomputingworld.co.uk/storage-predictions-for-2014/)
2014 Storage Developer Conference. © IGATE. All Rights Reserved.
Key Characteristics of Big Data
It’s estimated that
2.5 TRILLION GIGABYTES
Of data are created each day
The
Variety
Four V’s
Volume
DIFFERENT
SCALE OF DATA of Big FORMS OF
Terabytes DATA
Records
Data
Transactions
Structured
Tables, files
Unstructured
Semistructured
By 2016 there will be
18.9 Billion

Sensor Data
Network Connections ?
Veracity
Velocity
UNCERTAINTY
ANALYSIS
OF DATA
Batch 1 IN 3 BUSINESS
Near time
LEADERS
Real time
Don’t trust the information
Streams
they use to make decisions
Source: http://www.ibmbigdatahub.com/infographic/four-vs-big-data
2014 Storage Developer Conference. © IGATE. All Rights Reserved.
Prioritizing Things

s • Data growth s • Enterprise growth s • Analytics/business
n e e
Intelligence
r • System i • Attracting & c
t
e n
performance and i retaining new • Mobile
r
c e
o
n scalability customers r technologies
o ri e
• Network • Reducing • Cloud computing
P f
C
e
congestion and enterprise costs
s r
connectivity s P
e

y
n
g
i
s o
u
l
o
B
n
h
c
e
T
2014 Storage Developer Conference. © IGATE. All Rights Reserved.
Requirements of Big Data Storage
2014 Storage Developer Conference. © IGATE. All Rights Reserved.
Key Requirements of Big Data Storage

Provide tiered
Scalable
storage
Self-Managing Highly Available
Requirements
Widely Accessible Security
Integrate with
Self Healing
legacy applications
2014 Storage Developer Conference. © IGATE. All Rights Reserved.