© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential Cisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.

Network VisibilityFor Efficient OpenStack Operations

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential Cisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.

The Cloud / Big-Data Winnovation Team

Alex Holdenajonasho@cisco.com

Damian Van Vuurendvanvuur@cisco.com

Ailing Zhangailzhang@cisco.com

Marc Solanasmsolanas@cisco.com

Matt Weeksmaweeks@cisco.com

Pengfei Zhangpenfzhan@cisco.com

Kai Zhangkazhang2@cisco.com

Xinyuan Huangxinyuahu@cisco.com

Debo Duttadedutta@cisco.com

Yathiraj Udupiyudupi@cisco.com

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

Today...

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential Cisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

Outline• Data Gathering Architecture

• Network Visibility

• Network Analysis

• Conclusion

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

Virtual status

Performance

Why is my job slow?

Operators

Physical status

Hotspots

What are they doing in my cloud?

Users

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

Network flowsNew ceilometer agent

Collects and aggregates data from Neutron

VM-VM flow traffic insight

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

Application logs

OpenStack Logs

Server Syslogs

VM Syslogs

Application (Hadoop, etc) Logs

...

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

AVOS: Analytics and Visualization on OpenStack Demo

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved.

Data Insights

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

Data insights - What is going on in a Hadoop cluster?

Net

wor

k flo

ws

(bit/

s)

Outgoing network flows from VMs

Time (s) Time (s)

Incoming network flows to VMs

Net

wor

k flo

ws

(bit/

s)

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

Data insights - What is going on in a Hadoop cluster?

Time (s)

Net

wor

k flo

ws

(bit/

s)

Outgoing network flows

master <-> slave

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

Data insights - What is going on in a Hadoop cluster?

Time (s)

Net

wor

k flo

ws

(bit/

s)

Outgoing network flows

slave <-> slave

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

Data insights - How to troubleshoot?

Normal Server down

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

Data insights - How to troubleshoot?

INFO org.apache.hadoop.hdfs.DFSClient: Exception in createBlockOutputStream … java.io.IOException: Bad connect ack with firstBadLink as 192.168.0.7:…

INFO org.apache.hadoop.hdfs.DFSClient: Abandoning blk_…

INFO org.apache.hadoop.hdfs.DFSClient: Excluding datanode 192.168.0.7:…

WARNING nova.compute.manager … [instance: 6a475088-...] Instance shutdown by itself. Calling the stop API." …

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

● Collect Multiple Metrics● Analyze Patterns● Classify Cloud States

Data insights - How to detect abnormals?

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

Future PlansNetwork Distance

Data-driven Diagnoses

More data sources & integrated analysis

© 2013 Cisco and/or its affiliates. All rights reserved.Cisco

Confidential

Thank you.Please take our survey:

http://goo.gl/CMAZje