Why Go Hybrid Cloud?
9
| Log management as a service Simplify Log Management #LDFE Why Go Hybrid Cloud? Manoj Chaudhary CTO & VP of Engineering at Loggly
-
Upload
loggly -
Category
Technology
-
view
268 -
download
0
Transcript of Why Go Hybrid Cloud?
| Log management as a service Simplify Log Management #LDFE
Why Go Hybrid Cloud?
Manoj Chaudhary
CTO & VP of Engineering at Loggly
| Log management as a serviceReveal What Matters | @loggly - #StackWorld16
Today’s Talk
Hybrid CloudOur Use
Case and Learnings
Go Hybrid
| Log management as a serviceReveal What Matters | @loggly - #StackWorld16
Operated by a 3rd party provider. Utilized for the work loads that are not
core for the private cloud.
Hybrid Cloud
Connect
Private Cloud
Public Cloud
Operated in your own data center. Typically within a firewall for organizations.
Controlled and maintained by Ops to ensure interoperability with private
cloud.- BEST OF BOTH WORLDS -
This model offers versatility and convenience, while preserving management, control and security.
Delivered as a cloud service over the Internet. Sold on-demand, typically by the minute or the hour. Customers only pay for the CPU cycles, storage or bandwidth they
consume.
This cloud computing environment which uses a mix of on-premises, private cloud and third-party,
public cloud services with orchestration between the two platforms.
| Log management as a serviceReveal What Matters | @loggly - #StackWorld16
Loggly’s Use Case
Use Case• Centralized log
management• Allow customers to
analyze large amount of data
• Real time processing of data
Our Challenges• Massive incoming event
stream• Fundamentally multi-
tenant• Near real-time indexing• Near real-time searches• Near real-time alerts
Our Data Processing Stack
• Collector• Kafka• Elasticsearch• Redis/Memcached• Ngnix
| Log management as a serviceReveal What Matters | @loggly - #StackWorld16
Event Processing
Loggly’s Big Data PipelineEvent Processing
Event Processing
Event Processing
Event Processing
Event Processing
KafkaQueue
| Log management as a serviceReveal What Matters | @loggly - #StackWorld16
Loggly’s Big Data Pipeline
From Kafka
Elastic Search Clusters Multi-TieredElastic Cluster
| Log management as a serviceReveal What Matters | @loggly - #StackWorld16
Loggly’s - Learning 1St Deployment
Our First Attempt Was Public Cloud
Why Did We Try Public Cloud?
• Fastest route to go live• Less upfront cost• We could learn about
application behavior and needs
Findings
• Our resources grow consistently
• Our work load requires extensive Compute, NIO and IO
• Reliability and consistent performance is key
• Public Cloud - Not cost efficient for our model
| Log management as a serviceReveal What Matters | @loggly - #StackWorld16
Go Hybrid
Hybrid Cloud
Private Cloud
Public CloudConnect
Why Hybrid?
• Uses the right cloud for right kind of work load• Gives us more control overs hardware components,
compute, failover options• Reliability - use of resources is predictable• Suits our Infrastructure growth• Cost efficient for big data work load
Public Vs. Private Cloud Services
• There isn’t one answer on how to decide. It is entirely based on needs.
RecommendationsPublic Cloud• All internet facing
services • All services which can
burst and need high elasticity
Private Cloud• Big data processing
services • Services that process
sensitive data
| Log management as a serviceReveal What Matters | @loggly - #StackWorld16
Reach Out to Me!
Reach me at [email protected]
Blogs at http://bit.ly/ManojBlogs
About Us:Loggly is the world’s most popular cloud-based log management solution, used by more than 5,000 happy customers to effortlessly spot problems in real-time, easily pinpoint root causes and resolve issues faster to ensure application success. Try Loggly for Free! → https://www.loggly.com/
Visit us at loggly.com or follow @loggly on Twitter.
