flexible storage for hpc clouds with archipelago and ceph vangelis koukis archipelago and ceph ...

30 Pages·2013·4.41 MB·English

VHPC’13
[email protected]
FLEXIBLE STORAGE FOR HPC CLOUDS WITH
AARRCCHHIIPPEELLAAGGOO AANNDD CCEEPPHH. VVAANNGGEELLIISS KKOOUUKKIISS
TECHNICAL LEAD, SYNNEFO
1
Outline VHPC’13
[email protected]
Archipelago overview
Storagge resources,, clones/snappshots
HPC workflow with Archipelago
RReessoouurrccee CCoommppoossiittiioonn
Archipelago Implementation
Flexible I/O pipelines
Integgration with Syynnefo
Future directions
2
Archippelaggo overview VHPC’13
[email protected]
Distributed Storage System
- Poweringg storagge in clouds
Decouples storage resources from storage backends
- FFiilleess // IImmaaggeess // VVoolluummeess // SSnnaappsshhoottss
Unified way to provision, handle, and present resources
Decouples logic from actual physical storage
- Software-Defined Storagge
3
Archippelaggo loggic VHPC’13
[email protected]
Thin provisioning, with clones and snapshots
- Independent from the underlying storage technology
Hash-based data deduplication
PPlluuggggaabbllee aarrcchhiitteeccttuurree
- Multiple endpoint (northbound) drivers
- MMullttiiplle bbackkendd ((soutthhbboundd)) ddriivers
Multiple storage backends
- Unified management
- wwiitthh ssttoorraaggee mmiiggrraattiioonnss
4
Unified view of resources VHPC’13
[email protected]
Files
- User files, with Dropbox-like syncing
Images
- TTeemmppllaatteess ffoorr VVMM ccrreeaattiioonn
Volumes
- Live disks, as seen from VMs
Snapshots
- PPoiintt-iin-ttiime snapshhotts off VVollumes
5
VHPC’13
[email protected]
Spawn
mmyy oowwnn UUbbuunnttuu
Freeze
6
VHPC’13
[email protected]
Clone
UUbbuunnttuu ++ uusseerr ddaattaa
Snapshot
7
The bigg ppicture VHPC’13
[email protected]
Archipelago Core
Storagge backend 1 Storagge backend 2 Storagge backend 3
(e.g., Ceph cluster 1) (e.g., Ceph cluster 2) (e.g., NFS over NAS)
8
VHPC’13
[email protected]
9
End-to-end workflow with unified storagge VHPC’13
[email protected]
10