Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014
Download as PPTX, PDF2 likes1,682 views
The document discusses Ceph, an open-source storage platform, and its evolution in addressing the scaling challenges of data storage as global data volume is projected to exceed 15 zettabytes by 2020. It highlights the transition from traditional storage solutions to the scalable 'scale-out' architecture of Ceph, alongside its integration with various technologies like OpenStack. Additionally, it outlines upcoming community initiatives, products, and support services related to Ceph.
Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014
- 1. Ceph, Open Source, and the Path to Ubiquity in Storage AACS – 2014 Apr 02
- 2. Copyright © 2014 by Inktank | Private and Confidential WHO? 2
- 3. Copyright © 2014 by Inktank | Private and Confidential AGENDA 3
- 4. THE FORECAST By 2020 over 15 ZB of data will be stored. 1 .5 ZB are stored today. 4 Copyright © 2014 by Inktank | Private and Confidential
- 5. THE PROBLEM Existing systems don’t scale Increasing cost and complexity Need to invest in new platforms ahead of time2010 2020 IT Storage Budget Growth of data 5 Copyright © 2014 by Inktank | Private and Confidential
- 6. THE SOLUTION PAST: SCALE UP FUTURE: SCALE OUT 6 Copyright © 2014 by Inktank | Private and Confidential
- 8. HISTORICAL TIMELINE Copyright © 2013 by Inktank | Private and Confidential 8 RHEL-OSP & RHEV Support FEB 2014 MAY 2012 Launch of Inktank OpenStack Integration 2011 2010 Mainline Linux Kernel Open Source 2006 2004 Project Starts at UCSC Production Ready Ceph SEPT 2012 2012 CloudStack Integration OCT 2013 Inktank Ceph Enterprise Launch Xen Integration 2013
- 9. ARCHITECTURAL COMPONENTS 9 Copyright © 2014 by Inktank | Private and Confidential APP HOST/VM CLIENT
- 10. CEPH UNIFIED STORAGE FILE SYSTEM BLOCK STORAGE OBJECT STORAGE Keystone Geo-Replication Native API 10 Multi-tenant S3 & Swift OpenStack Linux Kernel iSCSI Clones Snapshots CIFS/NFS HDFS Distributed Metadata Linux Kernel POSIX Copyright © 2014 by Inktank | Private and Confidential
- 11. TRADITIONAL STORAGE VS. CEPH TRADITIONAL ENTERPRISE STORAGE 11 Copyright © 2014 by Inktank | Private and Confidential
- 12. CEPH WITH OPENSTACK 12 Copyright © 2014 by Inktank | Private and Confidential
- 13. WEB APPLICATION STORAGE 13 Copyright © 2014 by Inktank | Private and Confidential S3/Swift S3/Swift S3/Swift S3/Swift
- 14. MULTI-SITE OBJECT STORAGE 14 Copyright © 2014 by Inktank | Private and Confidential
- 15. Copyright © 2014 by Inktank | Private and Confidential ARCHITECTURE
- 16. ARCHITECTURAL COMPONENTS 16 Copyright © 2014 by Inktank | Private and Confidential APP HOST/VM CLIENT
- 17. 1 DISK FS DISK DISK OSD DISK DISK OSD OSD OSD OSD FS FS FSFS btrfs xfs ext4 MMM 17
- 18. 1 M M M HUMAN 18
- 19. 1 Monitors: Maintain cluster map Provide consensus for distributed decision-making Must have an odd number These do not serve stored objects to clients M OSDs: •One per disk (recommended) •At least three in a cluster •Serve stored objects to clients •Intelligently peer to perform replication tasks •Supports object classes 19
- 21. 2 Metadata Server Manages metadata for a POSIX- compliant shared filesystem Directory hierarchy File metadata (owner, timestamps, mode, etc.) Stores metadata in RADOS Does not serve file data to clients Only required for shared filesystem 21
- 22. CRUSH Pseudo-random placement algorithm Ensures even distribution Repeatable, deterministic Rule-based configuration Replica count Infrastructure topology Weighting CRUSH 22
- 23. 2 10 10 01 01 10 10 01 11 01 10 10 10 01 01 10 10 01 11 01 10 hash(object name) % num pg CRUSH(pg, cluster state, rule set) 23
- 24. 2 CLIENT ?? 24
- 25. 2 25
- 26. 2 26
- 27. 2 CLIENT ?? 27
- 29. Ceph Developer Summit 29 • Recent: “Giant” • March 04-05 • wiki.ceph.com • Virtual (irc, hangout, pad, blueprint, youtube) • 2 days (soon to be 3?) • Discuss all work • Recruit for your projects!
- 30. New Contribute Page 30 • http://ceph.com/ community/ Contribute • Source tree • Issues • Share experiences • Standups • One-stop shop
- 31. New Ceph Wiki 31
- 32. Accepted as a mentoring organization 8 mentors from Inktank & Community http://ceph.com/gsoc2014/ Student applications being evaluated Hope to turn this into academic outreach Google Summer of Code 2014 32
- 33. Ceph Days 33 • inktank.com/ cephdays • Recently: London, Frankfurt, NYC, Santa Clara • Aggressive program • Upcoming: Sunnyvale, Austin, Boston, Kuala Lumpur
- 34. Meetups 34 • Community organized • World wide • Wiki • Ceph-community • Goodies available • Logistical support • Drinkup to tradeshow
- 35. We haven’t forgotten! Looking for potential founding members Especially important to keep the IP clean Ceph Foundation 35
- 36. Coordinated Efforts 36 • Always need help • CentOS SIG • OCP • Xen • Hadoop • OpenStack • CloudStack • Ganetti • Many more!
- 37. http://metrics.ceph.com 37 Copyright © 2014 by Inktank | Private and Confidential
- 38. THE PRODUCT
- 39. 39 INKTANK CEPH ENTERPRISE WHAT’S INSIDE? Ceph Object and Ceph Block Calamari Enterprise Plugins (2014) Support Services Copyright © 2014 by Inktank | Private and Confidential
- 41. Ceph Inktank Ceph Enterprise Open-Source Ceph ✔ ✔ Object Storage ✔ ✔ Block Storage ✔ ✔ File System ✔ Management API ✔ ✔ Management GUI ✔ Hyper-V Support Q3 2014 SNMP Support Q3 2014 24x7 Support ✔ Bug Prioritization ✔ FEATURE COMPARISON 41 Copyright © 2014 by Inktank | Private and Confidential
- 42. ROADMAP INKTANK CEPH ENTERPRISE 42 Copyright © 2014 by Inktank | Private and Confidential April 2014 September 2014 2015
- 43. RELEASE SCHEDULE Copyright © 2014 by Inktank | Private and Confidential 43 2013 2014 2015 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2
- 44. Copyright © 2014 by Inktank | Private and Confidential QUESTIONS? 44
- 45. THANK YOU! Patrick McGarry Director, Community [email protected] m irc.oftc.net/ceph @scuttlemonkey @ceph
Editor's Notes
- #9: Make it a horizontal timeline