Software Defined Storage - Open Framework and Intel® Architecture Technologies
-
Upload
stanislas-odinot -
Category
Technology
-
view
1.544 -
download
5
description
Transcript of Software Defined Storage - Open Framework and Intel® Architecture Technologies
Software Defined Storage - Open Framework and Intel® Architecture Technologies
Anjaneya “Reddy” ChagamPrincipal Engineer, Intel Data Center Group
Shayne HuddlestonInfrastructure Architect, Information Services, Oregon State University
DATS001
2
Agenda
• The problem
• Software Defined Storage (SDS) vision
• Why SDS matters?
• Customer perspective
• SDS gaps and Intel response
• Intel role in SDS
• Summary
3
Agenda
• The problem
• Software Defined Storage (SDS) vision
• Why SDS matters?
• Customer perspective
• SDS gaps and Intel response
• Intel role in SDS
• Summary
4
The Problem:From 2013 to 2020, the digital universe will grow by a factor of 10, from 4.4 ZB to 44 ZB
It more than doubles every two years.
Data needs are growing at a rate unsustainable with today’s infrastructure and labor costs
Source: IDC – The Digital Universe of Opportunities: Rich Data and the Increasing Value of the Internet of Things - April 2014
…cost challenges continue to grow…
….data complexity is increasing
…. and storage accounts for 40% of the data center budget.
5
The Problem with Storage Infrastructure
Storage Silos (Traditional)
• Application mapped to specific appliance
• Storage resources optimized to run specific workload
• Isolated storage resources
Challenges
• Cost of managing diverse storage solutions
- Data Growth
- Maintenance, Operations & Support
- Infrastructure
• Vendor lock-in
- Limited scalability
- Flexibility to innovate
• Need for massively shared data
- But not yet cloud ready
Traditional storage management is too complex and inefficient
6
Agenda
• The problem
• Software Defined Storage (SDS) vision
• Why SDS matters?
• Customer perspective
• SDS gaps and Intel response
• Intel role in SDS
• Summary
7
Software Defined Infrastructure (SDI)
PROVISIONING MANAGEMENTOrchestration provisions, manages and optimally
allocates resources based on the unique requirements of an application
POOLED RESOURCESNetwork, Storage and Compute elements are
abstracted into resource poolsStorage Network Compute
Services Delivery
Resource Pool
Orchestration Software
Infrastructure Attributes
Application A
Application B
Application C
Application D
Power Performance Security Thermals Utilization Location
SERVICE ASSURANCEPolicies and intelligent monitoring trigger dynamic
provisioning and service assurance as applications are automatically deployed and maintained
8
SDS – A Key Component of SDIDynamic, policy-driven storage resource management
Services Delivery
Software Defined Storage Controller
Orchestration Software
Infrastructure Attributes
App #1 App #2 App #3 App #4
Power Performance Thermals Utilization Location
Storage Systems
Latency Durability
SDS is a framework that delivers a scalable, cost-effective solution to serve the needs of tomorrow’s Data Center
Abstracting Software from Hardware, providing flexibility & scalability
Aggregating diverse provider solutions, increasing flexibility and drive down costs
Provisioning resources dynamically (pay-as-you-grow) increasing efficiency
Orchestrating application access to diverse storage systems through Service Level Agreements (SLAs), increasing flexibility and handle data complexity
9
SDS Architecture
Applications
SDS Controller
Node Node Node
Orchestrator
Storage System[Capacity]
Storage System[Performance]
Data Services
Storage System[SAN]
Storage System[NAS]
SDS Controller• Visibility and Control of ALL
storage resources• Communication between
Apps, Orchestrator and Storage Systems
• Allocates storage resources to meet SLA’s
Node NodeJBOD
Data Services• Application that runs in
data plane to optimize storage
• Ex: Predictive Analytics• Ex: De-Duplication• Ex: Tiering
……….. ………..
SDS : Consolidated Management of Scale-Out and Scale-Up Storage and plug into SDI
Compute ControllerSDN Controller
NASSAN
Northbound API
Southbound API
10
Agenda
• The problem
• Software Defined Storage (SDS) vision
• Why SDS matters?
• Customer perspective
• SDS gaps and Intel response
• Intel role in SDS
• Summary
11
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
Example: Application provides storage pool
1
2
Gold
App 1
(OLTP)App 2
(Backup)
App 3
(Sharing)
Bronze Silver
Discover
Compose
3 Assign
12
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
Example: Application provides storage pool
1
2
App 1
(OLTP)
App 2 (Backup)
App 3
(Sharing)
Bronze Silver
Discover
Compose
3 Assign
13
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
Example: Application provides storage pool
1
2
App 1
(OLTP)
App 2 (Backup)
App 3
(Sharing)
Silver
Discover
Compose
3 Assign
14
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
Example: Application provides storage pool
1
2
App 1
(OLTP)
App 2 (Backup)
App 3
(Sharing)
Discover
Compose
3 Assign
15
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
Example: Application provides SLO attributes (performance)
1
2
IOPS=7000
App 1 App 2 App 3
Tpt - 2 Mbps Tpt – 30Mbps
Discover
Compose
3 Assign
16
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
Example: Application provides SLO attributes (performance)
1
2
App 1
App 2 App 3
Tpt - 2 Mbps Tpt – 30Mbps
Discover
Compose
3 Assign
17
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
Example: Application provides SLO attributes (performance)
1
2
App 1 App 2
App 3
Tpt – 30Mbps
Discover
Compose
3 Assign
18
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
Example: Application provides SLO attributes (performance)
1
2
App 1 App 2App 3
Discover
Compose
3 Assign
19
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
Example: Application provides SLO attributes (performance, client caching)
1
2
IOPS=7000App 1
Discover
Compose
3 Assign
Client Cache = 200GBApp1
20
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
Example: Application provides SLO attributes (performance, client caching)
1
2
App 1
Discover
Compose
3 Assign
Client Cache = 200GB
App1
21
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
1
2
IOPS=170K (R/W)App 1
Discover
Compose
3 Assign
Example: Application provides SLO attributes (ephemeral)
Ephemeral
22
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
1
2
IOPS=170K (R/W)
Discover
Compose
3 Assign
App1
Example: Application provides SLO attributes (ephemeral)
23
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
1
2
IOPS=170K (R/W)App 1
Discover
Compose
3 Assign
Example: Application provides SLO attributes (ephemeral, local protection)
Persistent, Local Protection
24
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
1
2
IOPS=170K (R/W)
Discover
Compose
3 Assign
App1
Example: Application provides SLO attributes (ephemeral, local protection)
25
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
1
2
IOPS=9000App 1
Discover
Compose
3 Assign
Example: Application provides SLO attributes (IOPS, noisy neighbor policy)
Compute Controller
26
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
1
2
IOPS=9000
App 1
Discover
Compose
3 Assign
App1
Example: Application provides SLO attributes (IOPS, noisy neighbor policy)
Compute Controller
Qemu QoS Max IOPS=9000
27
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
1
2
Tpt – 2 MbpsApp 1
Discover
Compose
3 Assign
Example: Application provides SLO attributes (Throughput, noisy neighbor policy)
SDN
28
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
1
2
Tpt – 2 Mbps
App 1
Discover
Compose
3 Assign
App1
Example: Application provides SLO attributes (Throughput, noisy neighbor policy)
SDN
tc ul rate 2048kbit
29
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
1
2
Tpt – 2 MbpsApp 1
Discover
Compose
3 Assign
Example: Application provides SLO attributes (Performance but storage is full)
Free Pool
2
Policies
Policy #XIf 80% capacityThen mark system unavailable
81% utilization
30
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
1
2
App 1
Discover
Compose
3 Assign
Example: Application provides SLO attributes (Performance but storage is full)
Free Pool
2
Policies
Policy #XIf 80% capacityThen mark system unavailable
81% utilization
31
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
1
2
App 1
Discover
Compose
3 Assign
Example: Application provides SLO attributes (Performance but storage is full)
Free Pool
2
Policies
Policy #XIf 80% capacityThen mark system unavailable
32
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High PerfIOPS – 10K
(DC1)
High PerfIOPS – 5K
(DC2)
Med PerfTpt – 100Mbps
(DC2)
Med PerfTpt – 50 Mbps
(DC1)
CapacityTpt – 5 Mbps
(DC1)
$$$ Gold $$ Silver $ Bronze
1
2
Discover
Compose
3 Assign
Example: Application provides SLO attributes (Performance but storage is full)
Free Pool
2
Policies
Policy #XIf 80% capacityThen mark system unavailable
App 1
33
Agenda
• The problem
• Software Defined Storage (SDS) vision
• Why SDS matters?
• Customer perspective
• SDS gaps and Intel response
• Intel role in SDS
• Summary
34
Perspectives from Oregon State University (OSU) IT on SDS
20
35
Oregon State University - Introduction
A top research university
• 28,000 Students, 3,500 faculty, 5 datacenters
• Top Carnegie Foundation class of research universities
• One of only two universities in the U.S. to be a Land, Sea, Space and Sun Grant university
• Statewide economic footprint of $2.06 Billion
• 15 Agricultural Experiment Stations; two remote campuses, Hatfield Marine Sciences Center in Newport,, and OSU-Cascades in Bend
21
36
Oregon State University - Information Services
• Central IT provider for OSU (Infrastructure as a Service - storage, compute, networking, facilities)
• Mission is to enable and support Students, Faculty & Science
• Multitude of different workloads that we must support
- Long term digital archive
durable, replicated, WORM
- Virtual Machines
- Virtual Desktops (VDI)
- Research workloads
Genome sequencers generate 20TB/hour sustained high bandwidth write
- OLTP - Oracle, MSSQL, mySQL
- Business analytics (Hadoop, etc.)
22
37
Oregon State University – DC Infrastructure
• 3 central data centers on campus
• Infrastructure
- Heavily virtualized
- High availability metro cluster architecture
- Home grown orchestration layer
- Puppet configuration management
- Active/active replicated block storage
- SAN, NAS storage islands
- Distributed storage (piloting)
23
38
Oregon State University – IT Challenges
24
• Storage demand is growing faster
- Growing at over 40%/year
• Storage is not optimized for diverse workloads
• Traditional storage management is costly and hard to scale
• Lot of touch points to administer, provision, and consume storage resources
• Allocation of storage resources to user workloads is 1 to 1
• IT budget is relatively flat
• Majority of storage cost is hardware and operations
39
Oregon State University – Focus Areas & Plans
25
• Storage growth explosion
- Shift to cost optimized, “pay as you grow” storage architecture
- Distributed storage on standard high volume servers (open source focus)
• Storage optimized for diverse workloads
- SLA and policy driven storage resource allocation
- Storage allocation based on workloads, performance, cost and resiliency requirements
• Storage management
- All storage operations managed by single controller in the DC
Administrator should not have to go beyond the controller except in extraordinary circumstances
- Recognition and classification of performance problems in central dashboard
- Fully automated provisioning and simplified maintenance
New capacity automatically added to appropriate service and controlled by policies
OSU is collaborating with Intel to address storage challenges via open source SDS controller initiative
40
Agenda
• The problem
• Software Defined Storage (SDS) vision
• Why SDS matters?
• Customer perspective
• SDS gaps and Intel response
• Intel role in SDS
• Summary
41
SDS Gaps
SDS Controller
Clear Standards
Standards exist, but they are not focused on SDS, not organized together, and
are not complete
Industry wide focus needed to develop standards by either enhancing or developing new standards (e.g., SNIA, TOSCA, DMTF)
Node
Applications
Northbound API
Southbound API
Node Node
SDS Controller SDS Controller
Storage System(Capacity)
Storage System(Performance)
Node Arrays
Orchestrator
42
Storage Control Plane Standards – State & Gaps
Intel plans to work with standard bodies to address SDS gaps
Node
Applications
Northbound API
Southbound API
Node Node
SDS Controller SDS Controller
Storage System(Capacity)
Storage System(Performance)
Node Arrays
Orchestrator
Standard Gap
OASIS TOSCA
Applications provide storage requirements using SLOs
IETF Network focus - storage requirements (e.g. policies) not comprehended
CDMI Object storage focus – need extensions for block, file, etc.
SMI-S Appliance focus - requires changes for distributed storage usage
43
SDS Gaps
Clear Standards
SDS Controller
Open, federated “control plane” with pluggable architecture is needed for ecosystem innovation
Node
Applications
Northbound API
Southbound API
Node Node
SDS Controller SDS Controller
Storage System(Capacity)
Storage System(Performance)
Node Arrays
Orchestrator
Open SDS Controller does not
exist
44
SDS Controller
Storage
Orchestration
SDN Controller
Storage Systems
Distributed Storage
SLOs File/BlockObject
Service/ Tenant Portal
REST APIs
Users
Storage Hardware,
Media
Unprovisioned
GoldLatency – 5msIOPS – 20000Access - block
SilverLatency – 10msIOPS – 1000Access - block
BronzeLatency – 50msThroughput – 100MbpsAccess - object
Storage Arrays
File/Block
MongoDB* eCommerce
Media Store
SLOs SLOs
Distributed Storage
Compute Controller
45
SDS Controller
Storage
Orchestration
SDN Controller
Storage Systems
Distributed Storage
SLOs File/BlockObject
SLAs
Applica-tions
Service/ Tenant Portal
REST APIs
Users
Storage Hardware,
Media
Unprovisioned
GoldLatency – 5msIOPS – 20000Access - block
SilverLatency – 10msIOPS – 1000Access - block
BronzeLatency – 50msThroughput – 100MbpsAccess - object
Storage Arrays
File/Block
MongoDB* eCommerce
Media Store
SLOs SLOs
Distributed Storage
Compute Controller
46
SDS Controller
Storage
Orchestration
SDN Controller
Storage Systems
Distributed Storage
SLOs File/BlockObject
SLAs
Applica-tions
SLA SLO
Service/ Tenant Portal
REST APIs
Users
Storage Hardware,
Media
Unprovisioned
GoldLatency – 5msIOPS – 20000Access - block
SilverLatency – 10msIOPS – 1000Access - block
BronzeLatency – 50msThroughput – 100MbpsAccess - object
Storage Arrays
File/Block
MongoDB* eCommerce
Media Store
SLOs SLOs
Distributed Storage
Compute Controller
47
SDS Controller
Storage
Orchestration
SDN Controller
Storage Systems
Distributed Storage
SLOs File/BlockObject
SLAs
Applica-tions
SLA SLO
Service/ Tenant Portal
REST APIs
Users
Storage Hardware,
Media
Unprovisioned
GoldLatency – 5msIOPS – 20000Access - block
SilverLatency – 10msIOPS – 1000Access - block
BronzeLatency – 50msThroughput – 100MbpsAccess - object
Storage Arrays
File/Block
MongoDB* eCommerce
Media Store
Provisioning
QoS
SDS Controller
Control Plane
Policy MgmtSLA Storage
SLOMonitoring
REST APIs
SLOs SLOs
Distributed Storage
Compute Controller
48
SDS Controller
Storage
Orchestration
SDN Controller
Storage Systems
Distributed Storage
SLOs File/BlockObject
DataPlane
SLAs
Applica-tions
SLA SLO
Service/ Tenant Portal
REST APIs
Users
Storage Hardware,
Media
Unprovisioned
GoldLatency – 5msIOPS – 20000Access - block
SilverLatency – 10msIOPS – 1000Access - block
BronzeLatency – 50msThroughput – 100MbpsAccess - object
Storage Arrays
File/Block
MongoDB* eCommerce
Media Store
Provisioning
QoS
SDS Controller
Control Plane
Policy MgmtSLA Storage
SLOMonitoring
REST APIs
SLOs SLOs
Distributed Storage
Compute Controller
49
SDS Controller
Storage
Orchestration
SDN Controller
Storage Systems
Distributed Storage
SLOs File/BlockObject
DataPlane
SLAs
Applica-tions
SLA SLO
Service/ Tenant Portal
REST APIs
Users
Storage Hardware,
Media
Unprovisioned
Data Services
(optional)
GoldLatency – 5msIOPS – 20000Access - block
SilverLatency – 10msIOPS – 1000Access - block
BronzeLatency – 50msThroughput – 100MbpsAccess - object
Storage Arrays
File/Block
MongoDB* eCommerce
Media Store
Provisioning
QoS
SDS Controller
Control Plane
Policy MgmtSLA Storage
SLOMonitoring
REST APIs
SLOs SLOs
Distributed Storage
Compute Controller
50
Open Source SDS Controller - PrototypeHorizon
Dashboard
SDS Controller
Cinder Manila GlanceObject
Storage
Command Line Interface
Neutron
Nova
Ke
yst
on
e
Ceilometer
Distributed Storage
SAN Distributed Storage
NAS Distributed Storage
Distributed Storage
2
1
1 SDS controller discovers storage systems and capabilities (e.g., perf, capacity, tiers, etc.)
Admin composes storage pools (e.g., gold, silver, bronze)
Application requests storage service using SLOs.
Controller allocates storage volume from pool that can best service the request.
Storage gets assigned to Nova or App in VM.
Controller works with compute, network to set QoS.
2
1 1 13
3
4
4 4
4
Develop key requirements by working with ecosystem and finalize open source enabling plans in 2H’14
51
Agenda
• The problem
• Software Defined Storage (SDS) vision
• Why SDS matters?
• Customer perspective
• SDS gaps and Intel response
• Intel role in SDS
• Summary
52
How is Intel Helping?
Virtual Storage Manager (VSM)†
Plug-ins (e.g. Intel ISA-L Open
Source Version)COSBench
Object Storage RA Virtual Block RA
Silicon innovations, software optimizations to accelerate Open source Software Defined Storage
Swift Ceph Cinder
Media Transcoding VDI
INTEL®
TRUE SCALEINTEL®
SSDINTEL®
ETHERNET
Intel® Enterprise Edition for Lustre* software
Intel® Data Plane Development Kit
•Intel® Intelligent Storage Acceleration Library (Intel® ISA-L)•Intel® Cache Acceleration Software (Intel® CAS)
STORAGE
Intel® Platform Storage Extensions
Intel Ingredients
(Silicon, Platforms, Software)
NETWORK
Intel® Open Network PlatformIntel® QuickAssist AcceleratorsIntel® Silicon Photonics
COMPUTEIntel® Processor GraphicsIntel® Trusted Execution Technology (Intel® TXT)Intel® AVXIntel® AES New Instructions
Intel® Media SDKfor Servers
Intel® Advanced Vector Extensions (Intel® AVX); †Virtual Storage Manager (VSM) to be open sourced in Q4’14
Open Source(Storage)
Reference Architectures
53
Agenda
• The problem
• Software Defined Storage (SDS) vision
• Why SDS matters?
• Customer perspective
• SDS gaps and Intel response
• Intel role in SDS
• Summary
54
Summary
• Software Defined Storage is needed to address tomorrow’s challenges
- Data needs are growing at a rate unsustainable with today’s infrastructure and labor costs
- Traditional storage silos drives management complexity and inefficient
• SDS is a framework aimed at serving the needs of emerging storage requirements. But there are gaps -
- Industry wide focus is needed to create standards for application and storage system interoperability
- Open, federated “control plane” that provisions, monitors, ensures SLAs are needed
• Intel is working with community and ecosystem to address these gaps
55
Call to Action
• Take advantage of SDS disruptive wave
- Consider SDS POCs in Q1’15 (based on open source controller)
• Engage with the community to address storage challenges
- Provide feedback on storage requirements
- Help us drive open source, standards based Software Defined Storage controller
• Innovate storage offerings with Open source framework
56
Additional Sources of Information
• A PDF of this presentation is available from our Technical Session Catalog: www.intel.com/idfsessionsSF. This URL is also printed on the top of Session Agenda Pages in the Pocket Guide.
• Demos in the showcase –
- Transform storage with Software Defined Storage controller, Swift and Ceph (Booth: 122)
- Fujitsu Hyperscale*: New software-based distributed storage system featuring Intel® Virtual Storage Manager software (Booth: 123)
- Making it Real: Rack Scale Architecture for Software Defined Infrastructure and Orchestration (Booth: 120)
57
Additional Sources of Information
Poster Chats:
• DATC003 - Optimizing Ceph with Intel® Technologies
- Data Center & SDI Community, Station 2, Tuesday, 3:00pm – 5:00pm
• DATC010 - Improving Storage Performance Through I/O Hinting
- Data Center & SDI Community, Station 2, Tuesday, 5:00pm – 7:00pm
58
Legal DisclaimerINFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined". Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information.The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or go to: http://www.intel.com/design/literature.htm
Intel, Look Inside and the Intel logo are trademarks of Intel Corporation in the United States and other countries.
*Other names and brands may be claimed as the property of others.Copyright ©2014 Intel Corporation.
59
Risk FactorsThe above statements and any others in this document that refer to plans and expectations for the second quarter, the year and the future are forward-looking statements that involve a number of risks and uncertainties. Words such as “anticipates,” “expects,” “intends,” “plans,” “believes,” “seeks,” “estimates,” “may,” “will,” “should” and their variations identify forward-looking statements. Statements that refer to or are based on projections, uncertain events or assumptions also identify forward-looking statements. Many factors could affect Intel’s actual results, and variances from Intel’s current expectations regarding such factors could cause actual results to differ materially from those expressed in these forward-looking statements. Intel presently considers the following to be important factors that could cause actual results to differ materially from the company’s expectations. Demand for Intel's products is highly variable and, in recent years, Intel has experienced declining orders in the traditional PC market segment. Demand could be different from Intel's expectations due to factors including changes in business and economic conditions; consumer confidence or income levels; customer acceptance of Intel’s and competitors’ products; competitive and pricing pressures, including actions taken by competitors; supply constraints and other disruptions affecting customers; changes in customer order patterns including order cancellations; and changes in the level of inventory at customers. Intel operates in highly competitive industries and its operations have high costs that are either fixed or difficult to reduce in the short term. Intel's gross margin percentage could vary significantly from expectations based on capacity utilization; variations in inventory valuation, including variations related to the timing of qualifying products for sale; changes in revenue levels; segment product mix; the timing and execution of the manufacturing ramp and associated costs; excess or obsolete inventory; changes in unit costs; defects or disruptions in the supply of materials or resources; and product manufacturing quality/yields. Variations in gross margin may also be caused by the timing of Intel product introductions and related expenses, including marketing expenses, and Intel's ability to respond quickly to technological developments and to introduce new products or incorporate new features into existing products, which may result in restructuring and asset impairment charges. Intel's results could be affected by adverse economic, social, political and physical/infrastructure conditions in countries where Intel, its customers or its suppliers operate, including military conflict and other security risks, natural disasters, infrastructure disruptions, health concerns and fluctuations in currency exchange rates. Intel’s results could be affected by the timing of closing of acquisitions, divestitures and other significant transactions. Intel's results could be affected by adverse effects associated with product defects and errata (deviations from published specifications), and by litigation or regulatory matters involving intellectual property, stockholder, consumer, antitrust, disclosure and other issues, such as the litigation and regulatory matters described in Intel's SEC filings. An unfavorable ruling could include monetary damages or an injunction prohibiting Intel from manufacturing or selling one or more products, precluding particular business practices, impacting Intel’s ability to design its products, or requiring other remedies such as compulsory licensing of intellectual property. A detailed discussion of these and other factors that could affect Intel’s results is included in Intel’s SEC filings, including the company’s most recent reports on Form 10-Q, Form 10-K and earnings release.
Rev. 4/15/14
60
Backup
61
SDS – Industry Definitions
62
SDS – Industry DefinitionsIDC: “Any storage software stack that can be installed on any commodity resources (x86 hardware,
hypervisors, or cloud) and/or off-the-shelf computing hardware and used to offer a full suite of storage services and federation between the underlying persistent data placement resources to enable data mobility of its tenants between these resources.”
63
SDS – Industry DefinitionsIDC: “Any storage software stack that can be installed on any commodity resources (x86 hardware,
hypervisors, or cloud) and/or off-the-shelf computing hardware and used to offer a full suite of storage services and federation between the underlying persistent data placement resources to enable data mobility of its tenants between these resources.”
SNIA: Software-defined storage (SDS) is a term for computer data storage technologies which separate storage hardware from the software that manages the storage infrastructure. The software enabling a
software-defined storage environment provides policy management for feature options such as deduplication, replication, thin provisioning, snapshots and backup.
64
SDS – Industry DefinitionsIDC: “Any storage software stack that can be installed on any commodity resources (x86 hardware,
hypervisors, or cloud) and/or off-the-shelf computing hardware and used to offer a full suite of storage services and federation between the underlying persistent data placement resources to enable data mobility of its tenants between these resources.”
SNIA: Software-defined storage (SDS) is a term for computer data storage technologies which separate storage hardware from the software that manages the storage infrastructure. The software enabling a
software-defined storage environment provides policy management for feature options such as deduplication, replication, thin provisioning, snapshots and backup.
Wikipedia: Software-defined storage (SDS) is a term for computer data storage technologies which
separate storage hardware from the software that manages the storage infrastructure.
Characteristics: 1) Abstraction of logical storage services and capabilities from the underlying physicalstorage systems, and in some cases pooling across multiple different implementations. 2) Automationwith policy-driven storage provisioning with service-level agreements replacing technology details. 3)
Commodity hardware with storage logic abstracted into a software layer. 4) Scale-out storage architecture.
65
SDS – Industry DefinitionsIDC: “Any storage software stack that can be installed on any commodity resources (x86 hardware,
hypervisors, or cloud) and/or off-the-shelf computing hardware and used to offer a full suite of storage services and federation between the underlying persistent data placement resources to enable data mobility of its tenants between these resources.”
SNIA: Software-defined storage (SDS) is a term for computer data storage technologies which separate storage hardware from the software that manages the storage infrastructure. The software enabling a
software-defined storage environment provides policy management for feature options such as deduplication, replication, thin provisioning, snapshots and backup.
Wikipedia: Software-defined storage (SDS) is a term for computer data storage technologies which
separate storage hardware from the software that manages the storage infrastructure.
Characteristics: 1) Abstraction of logical storage services and capabilities from the underlying physicalstorage systems, and in some cases pooling across multiple different implementations. 2) Automationwith policy-driven storage provisioning with service-level agreements replacing technology details. 3)
Commodity hardware with storage logic abstracted into a software layer. 4) Scale-out storage architecture.
Intel’s definition includes “all” above elements but with emphasis on end to end storage management at a data center level