Introducing Cloudian HyperStore 6.0

23
Cloudian HyperStore 6.0 Changing the economics of Storage Release : April 12 th 2016

Transcript of Introducing Cloudian HyperStore 6.0

Page 1: Introducing Cloudian HyperStore 6.0

Cloudian HyperStore 6.0Changing the economics of StorageRelease : April 12 th 2016

Page 2: Introducing Cloudian HyperStore 6.0

FINALIST

History of Innovation

3

2011 2012 2013 2014 2015 2016

HyperStore 1.5• Software• Scale-out• Peer-to-peer• Multi Tenancy• Quality of Service

(QoS)

HyperStore 2.3• Software• S3 API• Multi Region Storage

HyperStore 3.0• Software• Compression• Virtual Appliance

HyperStore 2.4• Software• Server Side

Encryption• Billing & Chargeback• Citrix Cloud Platform

HyperStore 4.0• Software• Erasure Coding• OpenStack• NFS• Cloud Archive

HyperStore 5.0• Software &

Appliances• Server Side

Encryption• Billing & Chargeback• Citrix Cloud Platform

HyperStore 5.1• Smart Data• Hadoop• Smart Support

HyperStore 5.2• Smart Data Policies• Faster Repair

HyperStore ‘forever live’ 3000• Hot plug everything• Seamless Scale• Petabyte Scale

HyperStore 6.0• Operations @scale• Durable @scale• Tuning @scale

Page 3: Introducing Cloudian HyperStore 6.0

New Economics of Storage

Hardware/Software (62%)$1245

Operations FTE (26%)$511

• In 2016 estimated that 1 FTE can manage 344TB• Software Defined Storage has dramatically lowered storage costs to $122/TB/Year• Operational Costs now significantly outweigh the Acquisition related costs

Other $253 (12%)$2009/TB/Yr TCO

Hardware/Software (14%) $122

Operations FTE (58%)$511

Other $253 (28%)$886/TB/Yr TCO

Source : Gartner 2016 IT Key Metrics Data

Cloudian HyperStore 1c per GB/month

6.0 Focus

Page 4: Introducing Cloudian HyperStore 6.0

Introducing @scale Storage

5

Operations @scale New Operations Console for one click management Fully automated add/remove of nodes Non disruptive rolling upgrades

Durable @scale Always repaired, Always verified Dynamic Object Routing for automated failure avoidance Simple Disaster Recovery with Cross Region Replication

Tuning @scale Visual Storage Analytics reports to automatically identify hot spots Object ‘GPS’ to locate objects

Key announcement messages1. Double the management capacity for your Storage Administrators2. Continuous and automated failure resolution for data durability3. Proactive low-cost management for Petabyte Scale storage

NEW

NEW

April 12th

NEW

Page 5: Introducing Cloudian HyperStore 6.0

Operations @scale : New UI

Page 6: Introducing Cloudian HyperStore 6.0

Operations @scale : New UI

• Instant 360 View• Check Health• View Sys Info

Page 7: Introducing Cloudian HyperStore 6.0

Operations @scale : Datacenters

• One screen for hundreds of nodes• Instantly view health of nodes• Add nodes with one click• Cluster dynamically rebalances

Page 8: Introducing Cloudian HyperStore 6.0

Operations @scale : Usage

• View capacity consumption• View system performance

Page 9: Introducing Cloudian HyperStore 6.0

Operations @scale : Capacity Explorer

• Analyze your data uses across globe• Plan for future based on your demand

Page 10: Introducing Cloudian HyperStore 6.0

Operations @scale : Place data based on your Business Req

Page 11: Introducing Cloudian HyperStore 6.0

Operations @scale : Turnkey Appliance View

• Identify Nodes types• Blink Suspect Servers• Blink Suspect Drives

Page 12: Introducing Cloudian HyperStore 6.0

Tuning @scale : “Object GPS”

• Locate any object parts• Object digests and timestamps validation

Page 13: Introducing Cloudian HyperStore 6.0

Operations @scale : Rolling Upgrade

• Automates Upgrade/Patch installation - touchless

• Uses Puppet framework for distribution and management

• No downtime – node-by-node rolling upgrade

✓ Peer-to-peer system = no SPOF

Distributed Everything – Data, MetaData, Configuration

✓New version- Old version

Node 1 Node 2 Node 3t0

t1

t2

t3

Node 4 Node 5 Node 6

Node 1 Node 2 Node 3 Node 4 Node 5 Node 6

Node 1 Node 2 Node 3 Node 4 Node 5 Node 6

Node 1 Node 2 Node 3 Node 4 Node 5 Node 6

Page 14: Introducing Cloudian HyperStore 6.0

Durable @scale : Proactive Repair

Problem If the node/disk/network failed

today and IOs that were incomplete will have to be identified by scanning the whole system and making a lot of Disk IOs

VERY slow process

Exposes customer for days to data loss in event of double failures

Solution When an IO fails due to

network/node or disk failure we keep hints as to what failed

Now when we need to run repair we know what exactly needs repair

Very FAST process

Drastically Reduce exposure window from days to hours

Page 15: Introducing Cloudian HyperStore 6.0

Durable @scale : Rebuild Analytics

• View Data Rebuild info• View Cluster Rebalance info

Page 16: Introducing Cloudian HyperStore 6.0

To protect against unplanned outagesDurable @scale : Smart Redirect

Introducing Local cache copy

ReplicationClient

Node 1 Node 2 Node 3 Node 4

Cassandra Local Cache

1- req In

2- Node1 Down

3- Node 2 makes Extra Copy stored in local cache4- Node 2 keeps hint in Cassandra

5- req Success

Copy2 Copy1

Page 17: Introducing Cloudian HyperStore 6.0

To protect against unplanned outages

Introducing Hot Spares (k+m+L)

No Local Cache, only hints Repair handled by proactive

repair If customer has 20 nodes it can

only fail 2 nodes - (k+m+L) fixes this

Erasure Coding

Node 1

Node 4 Node 2

Node 3

Node 5 Node 6`

Node 7 Node 8

Client

Durable @scale : Smart Redirect

Page 18: Introducing Cloudian HyperStore 6.0

Durable @scale : Smart Disk Balancing

1. Scenario 1 :Disk Imbalance If we notice an imbalance it will change the tokens pointing from “highly used disk” to “low used

disk”

2. Scenario 2 : Disk failure New data automatically routes to newly assigned resources

t0

t1

t0

t1

Auto Dynamic MappingStatic Mapping

Page 19: Introducing Cloudian HyperStore 6.0

Durable @scale : Repair-On-Read

NODE 1 NODE 2 NODE N+1

Replica Replica

Read��

��

🔧on Read

• All replicas of the object are checked on a read request• Missing/out-of-date replicas are automatically replaced or updated

Object hasChanged

Out-of-Date replicas are replaced or updated

��

🔧on Read

Page 20: Introducing Cloudian HyperStore 6.0

Durable @scale S3 Cross Region Replication

• Restore Script restores object in a new bucket

• Restore scripts generates source list as a CSV file

• No CMC support in 6.0

Page 21: Introducing Cloudian HyperStore 6.0

S3 Compatibility

Ceph S3 testsTotal tests: 416OPEN 4 (3 bugs open: POST filename subst., minor ACL)PASS 386PASS* 26 (Illegal character case handled at Jetty)

Page 22: Introducing Cloudian HyperStore 6.0

Introducing @scale Storage

23

Operations @scale New Operations Console for one click management Fully automated add/remove of nodes Non disruptive rolling upgrades

Durable @scale Always repaired, Always verified Dynamic Object Routing for automated failure avoidance Simple Disaster Recovery with Cross Region Replication

Tuning @scale Visual Storage Analytics reports to automatically identify hot spots Object ‘GPS’ to locate objects

Key announcement messages1. Double the management capacity for your Storage Administrators2. Continuous and automated failure resolution for data durability3. Proactive low-cost management for Petabyte Scale storage

NEW

NEW

April 12th

NEW

Page 23: Introducing Cloudian HyperStore 6.0

Thank Youwww.cloudian.com

Cloud Storage for Everyone