© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved.
Predictive Analytics & Visualization Towards Data Driven Insights for Openstack Clouds
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential Cisco Confidential Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved.
Cloud/Big-data Winnovation Team
Alex Holden Damian Van Vuuren Ailing Zhang
Marc Solanas Matt Weeks Pengfei Zhang
Kai Zhang
Xinyuan Huang
Debo Dutta
Yathiraj Udupi
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential
Visualization in OpenStack today
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential
Tomorrow?
Sources: 2001 A Space Oddesy, MGM. Minority Report, Twentieth Century Fox Film Corporation. Mad Men, AMC
OpenStack, deploy me an
app
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential
Our Approach: Predictive Analytics & Awesome Visualisation Why Predictive? ● OpenStack has lots of moving parts ● Rule/Model/Policy based system hard without insight
Why Predictive? ● Ops folks want to know right now!
Why Visualisation? ● Easy to observe insights by humans ● East way to consume lots of information from loads oflogs/metrics ● Improve user experience
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved.
Digital Exhaust = Metrics/Logs What information can we get from our cloud? Where can we get it?
● Ceilometer ● OpenStack Syslog ● Project APIs (Nova, Neutron, Cinder etc)
What can we do with it? ● Extract Insights ● Figure out what failed and why ● Move workloads around ● Detect attacks on the cloud
● Physical Servers ○ CPU ○ Hardware Faults ○ OpenStack Logs ○ OpenStack Messaging Queue
● Tenant/Project Data ○ Quotas
● Virtual Machines ○ Config ○ CPU ○ Disk ○ Network Flows ○ Performance ○ Latency
● Application Level ○ Application Logs ○ Application Requirements ○ Application Performance
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential Cisco Confidential Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved.
AVOS Dashboard
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential
AVOS: Summary ● Analytics and Visualization on OpenStack ● Stateless: easy deployment ● Client side architecture:
○ Polls some OpenStack APIs ○ Listens to the messaging bus ○ Minimum overhead
● Provides users and service providers full insight into cloud
○ Real time info on all instances/networks etc by project and as admin ○ Historical data on performance ○ Stack-search: Search for all components in the cloud, quickly. ○ Multiple select (See combined data)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential
https://www.youtube.com/watch?v=vyb-Cd4VEDg
AVOS Demo
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential
Bottlenecks
Use Cases: Why use AVOS?
Attacks
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential
Use Case: Bottlenecks
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential
Use Case: Attack Detection ● Extract Metrics ● Analyse Patterns ● We can differentiate
between: ○ Hadoop Traffic ○ Network Faults ○ DDoS Attack
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential
Future Plans • Advanced Search Primitives • Reducing the volume of logs • Real-time error analytics • Add to/edit the cluster • Ability to preview changes
Curvature:
https://github.com/CiscoSystems/curvature
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential
Notes: OpenStack changes • Ceilometer
Better API queries, optimizing return structure Custom meters: VM to VM traffic flows
• Other suggestions/Ideas
Standardization of the Python clients (several inconsistencies between method calls) Integration into horizon
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential
Conclusion
Openstack Predictive Analytics/Visibility = Reduce Ops pains We must 1) gather data 2) extract insights 3) present to ops folks with awesome UX
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco
Confidential
Thank you. Please take our survey: http://goo.gl/CMAZje
Coming Talk: Storage Visibility and Optimization for OpenStack:
Wednesday, May 14 • 3:30pm - 4:10pm
Top Related