SWITCHdrive

Experience with running Owncloud on top of Openstack/Ceph

Christian Schnidrig
christian.schnidrig@switch.ch

Zürich, Jan 19, 2016

 https://www.flickr.com/photos/jimculp/865599033/in/album-72157594310567262/

© 2016 SWITCH 2
 • Openstack (Juno) • 118 servers
• Ceph (Hammer) • 2008 cores
• 17 TiB RAM
• 2.25 PiB storage

Paid service since 1.1.2016

© 2016 SWITCH 3
• Owncloud 7.0 (8.1 in Feb)
• Users: 14’000
• Quota: 25 GiB / user
• Number of files: 32M
© 2016 SWITCH
• Storage: 33 TiB
– 22 TiB live data
– 3 TiB trash
– 2.5 TiB versions
– 1 TiB thumbnails
– 4.5 TiB cache
Paid service since 1.1.2016
4
Starting Architecture (PoC)

https://commons.wikimedia.org/wiki/File:Flickr_-_stringer_bel_-_Overloaded_car_in_west_Africa.jpg

© 2016 SWITCH 5
 Internet

nginx/php5-fpm
lb PostgreSQL
haproxy

web sync
web sync

db-
nfs db ldap1
slave ldap

OpenStack

data backup db db
volume volume volume volume

ceph

© 2016 SWITCH 6
Why it worked so well …

https://en.wikipedia.org/wiki/Three_wise_monkeys#/media/File:Three_Wise_Monkeys_(2010).jpg

© 2016 SWITCH 7
Collectd, Graphite & Grafana

© 2016 SWITCH 8
© 2016 SWITCH 9
Elasticsearch, Logstash & Kibana (ELK)

© 2016 SWITCH 10
© 2016 SWITCH 11
© 2016 SWITCH 12
What changed

Drop Queue
Requests Requests

One Large Many small

Volume Volumes

DB on DB on
Ceph local SSD

© 2016 SWITCH 13
Current Architecture

© 2016 SWITCH 14
Desired Architecture

https://commons.wikimedia.org/wiki/File:Road_Train_Australia.jpg

© 2016 SWITCH 15
What is still needed?

© 2016 SWITCH 16
 christian.schnidrig@switch.ch

oc@lists.geant.org

© 2016 SWITCH 17