Cloudera Data Analyst training for Apache Hadoop - Xebia Training
Xebia Knowledge Exchange (jan 2011) - Trends in Enterprise Applications Architecture
-
Upload
michael-figuiere -
Category
Technology
-
view
2.970 -
download
0
description
Transcript of Xebia Knowledge Exchange (jan 2011) - Trends in Enterprise Applications Architecture
Trends in Enterprise Applications Architecture
An overview of upcoming changes
1/6/2011 Michaël Figuière
Buzzwords are around...
CloudNoSQL
Scalability
Elasticity
Virtualization
Distributed
Enterprise applications, nowadays
Enterprise applications architecture
• Huge, monolithic architectures are no longer the trends
• SOA is now mature, not dead
• Web applications are everywhere, REST is around
Service Oriented Architecture is ubiquitous
After extremism, pragmatism is now the trend
Frontier between Web and Services is now blurred
Service Oriented Architecture
Service Oriented Architecture
How to make it scale exactly to our needs ?
The Cloud
Cloud Computing principles
• Abstraction
• Automation
• Elasticity
SaaS, PaaS, IaaS, ... actually « (.*) as a Service »
Reduces manual tasks as much as possible
Scale up or down depending on the current needs
Cloud’s basic abstractions
SaaS
Provides an empty server
Client / UserApplication
Web
Middleware
PaaS
Application Web IaaS
Provides a middleware
Owned by the Cloud Provider
Owned by the enterprise
Cloud and automation
• Resources provided by the Cloud are exposed as services
• Maintenance and deployment tasks can be easily scripted
• The Cloud provider handles the maintenance tasks
Makes it possible to allocate a new server, create a new database, or change network security with a service call
Reduce manual talks necessary for infrastructure changes
The customer keep focused on the application development
Cloud and elasticity
• Automation ability makes it possible to easily start and stop instances
• Elasticity makes it possible to make allocated resources stick to current load
• Elasticity makes it possible to handle very high peak of load
• Simplifies capacity planning
And subscribe / unsubscribe from load-balancer
Pay only what you use
Elasticity in action
StorageInstance 1
ServiceInstance 1
ServiceInstance 2
StorageInstance 2
StorageInstance 3
ServiceInstance 3
ServiceInstance 4
WebInstance 1
WebInstance 2
Public Cloud players
Public Cloud vs. Private Cloud
Typical server loadduring a whole day
Public Cloud vs. Private Cloud
Private Cloud
Public Cloud
Example at guardian.co.uk
But the Cloud magic comes at a price
Elasticity requires symmetric instances
StorageInstance 1
ServiceInstance 1
ServiceInstance 2
StorageInstance 2
StorageInstance 3
ServiceInstance 3
ServiceInstance 4
WebInstance 1
WebInstance 2
Elasticity requires symmetric instances
StorageInstance 1
ServiceInstance 1
ServiceInstance 2
StorageInstance 2
StorageInstance 3
ServiceInstance 3
ServiceInstance 4
WebInstance 1
WebInstance 2
Some solutions
• Leader election can be used to avoid symmetric issue
• Other tools can be used to make the set of instances more collaborative
• Cloud providers solutions can be used
Zookeeper is a cluster management framework
Zookeeper provides distributed locks, queues, states...
Amazon RDS, SimpleDB, Google BigTable, ...
Configuration and logging
Application’ inputs and outputs
Application / Service
OutputInput
ConfigurationLogs /
Monitoring
Application’ inputs and outputs
Application / Service
OutputInput
ConfigurationLogs /
Monitoring
Application’ inputs and outputs
Application / Service
OutputInput
ConfigurationLogs /
Monitoring?
ok
Typical Configuration
• .properties or .xml files are typically used for configuration
• Puppet, CfEngine and Chef can help
• Finally, most settings aren’t used
Large enterprises applications may have 100+ conf files...
... but isn’t it just a hack ?
... and developers may not know their meaning
Configuration options are a hope that your users, with less knowledge of how the system works, will find the right solution where you didn't.
Jonathan EllisProject Leader Apache Cassandra
Another way to configure
• Uses zookeeper to maintain the current application configuration
• Uses your Cloud provider storage and tools
• Reduce configuration options to minimum
• Uses Convention over Configuration
Highly available and consistent distributed storage
S3 buckets, Elastic Block Store
There is probably too much options...
Typical Logs
• Append-only text files, stored locally or sent to Syslog
• Queried with Grep, Sed, Awk, ...
• A Regex implicitly defines the schema of the log
Can be large for especially with stacktraces, ...
Simple but doesn’t fit for complex queries
The customer keep focused on the application development
Typical Logs
Regex acts as a non negotiated schema
Logs as a dataset
• Logs made of serialized plain entities
• Centralized and combined for easy query through MapReduce
• Some command-line tools can be provided for simple queries
Log Schema updates are easier
Any query can be executed
Logging as a Service
Applications send their logs to Loggly which provides some added value features
Compared to a vendor solution, Loggly benefits
from a shorter and richer feedback loops
Traditional Design vs. Design for failures
Traditional Design
• Technical errors typically redirect the user to an error page
• Some single points of failure, or bottlenecks
• Timeouts and slow backends, are typically not handled
SQLException, IOException, unavailable backend...
Highly available stateful services may be expensive
One backend gets slow and the whole application slows down
Design for failures
• Failures will happen
• Handle slow backends and timeout with circuit breakers
• Cloud and NoSQL brings some highly available storage
Let’s embrace them rather than avoid them
The application may use graceful degradation
The most critical data can then be made highly available
Summary of interests for enterprises
• Elasticity helps to reduce costs and to handle high load
• Cloud is high availability provided by experts
• Cloud middleware can be expected to gain quality quickly
• But .... JEE 7 won’t come before 2013 / 2014
and simplifies capacity planning
Avoid expensive, custom and/or brittle solutions
Thanks to reduced and richer feedback loop
Questions / Answers
?@mfiguiere
blog.xebia.fr