Challenges in modeling enterprise storage systems

46 Pages·2006·0.91 MB·Danish

by  Arif Merchant

Challenges in
modeling enterprise
storage systems
Arif Merchant
Hewlett-Packard Laboratories
Contact: [email protected]
©2006 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice
An enterprise storage system
Each app has
Application
App1 App2 App3
Multiple
its own data
data
applications
objects
data1 data2 data3 data4 data5
objects
Primary site
Data storage
Running on Potentially using
system
resources at secondary site
Host Host
primary site resources
Storage-area Secondary site(s)
network
Disk
array
Disk Sites may serve
Tape lib
array
as primary /
secondary peers
WAN link(s)
Tape transport
QEST, September 2006 Arif Merchant: Challenges in modeling enterprise storage systems 2
Storage System models
Why should you care?
Storage is a significant part of IT costs
•
− Over 35% for enterprises with 5000+ employees (IDC 2002)
− Large fraction of storage cost is management
− Environment is complex: 100s of applications sharing petabytes of
data
− Stringent application requirements – failures can be catastrophic
(Survey: $89K-6.4M/hr of downtime)
Good storage models are crucial for managing storage
•
− Systematic, accurate models needed for informed choices
Uses of models
•
− Capacity planning – green field & consolidation
− Storage design
− On-going management
QEST, September 2006 Arif Merchant: Challenges in modeling enterprise storage systems 3
Storage management automation
Where storage models fit in [Anderson2002,Alvarez2001]
Understand needs
Determine solution
•Workload
•Select devices + configurations
characterization
•Model to verify config. Is good
•System components
(Changing) business •Assign load to devices
requirements
Design
Design
Iterative feedback
loop
Analyze Implement
Analyze Implement
Monitor usage
Construct solution
•Measure workload,
re-characterize
•Configure devices, LVM, …
•System response
•Migrate data
QEST, September 2006 Arif Merchant: Challenges in modeling enterprise storage systems 4
Components of storage models
Application
App1 App2 App3
data
data1 data2 data3 data4 data5
objects
Primary site
Data storage
system
Workloads Host Host
Storage-area Secondary site(s)
network
Disk
array
Disk
Tape lib
array
WAN link(s)
Tape transport
QEST, September 2006 Arif Merchant: Challenges in modeling enterprise storage systems 5
Workload characterization
Need
•
− Compact characterization
− Adequate for predictive storage system models
Minimal parameter set:
•
− Size of data
− IO rate, IO size distribution
− Reads vs. writes
− Sequentiality, spatial locality
− Temporal locality (e.g., frequency of repeated access)
− Concurrency (Simultaneously outstanding requests)
QEST, September 2006 Arif Merchant: Challenges in modeling enterprise storage systems 6
Measuring workloads
How do we acquire workload parameters?
•
− Users do not know workload details
− Measurements on production system may add load
− Storage devices have limited, vendor-dependent measurement points
(although industry standards SMI-S are helpful!)
− Workload behavior may depend on hardware & configuration
− Open problems:
• Minimum perturbation methods for measuring workload
parameters
• Tradeoffs between inaccuracies of workload parameters and
model accuracy
IO trace
workload
workload
description
analysis
workload analysis
running system
QEST, September 2006 Arif Merchant: Challenges in modeling enterprise storage systems 7
Workload attributes – access variability
How do you compactly capture this complex behavior?
Sample results for a single run
i3125om5, RC = 8GB, WC = 8GB; cache outcome percentages by I/O address range
100%
Coalesced write
average ccaacchhee hhiittss
over
90%
full ffoorr aa
trace
Warm write
80% average ttrraacceedd
for writes
70% second ooppeennmmaaiill
half Cold write
wwoorrkkllooaadd
60%
I/O's Delayed write
50%
40% Compulsory read miss
reads
30%
Replacement read miss
20%
read
Read hit -reread
10% hits
0%
Read hit from write cache
0 400 800 1200 1600 2000 2400 2800 3200 3600 ave steady
trace time [seconds] state GGrraapphh bbyy tthhee HHPPLL SSoonnoorraa tteeaamm
QEST, September 2006 Arif Merchant: Challenges in modeling enterprise storage systems 8
Workload specification challenges
What is an adequate workload
•
specification?
−Complex request arrival patterns
• Self-similar? Dependent on response?
• Seasonality, trends
−Complex access patterns
−Locality (spatial and temporal)
−Concurrency variation
−Correlation/interference between workloads
Synthesis of workloads accurately
•
representing real workloads
QEST, September 2006 Arif Merchant: Challenges in modeling enterprise storage systems 9
Components of storage systems
The basic storage device
Application
App1 App2 App3
data
data1 data2 data3 data4 data5
objects
Primary site
Data storage
system
Host Host
Disk drives
Storage-area Secondary site(s)
network
Disk
array
Disk
Tape lib
array
WAN link(s)
Tape transport
QEST, September 2006 Arif Merchant: Challenges in modeling enterprise storage systems 10