Cloud Development with Camel and Amazon Web Services
-
Upload
robin-howlett -
Category
Technology
-
view
1.175 -
download
2
description
Transcript of Cloud Development with Camel and Amazon Web Services
![Page 1: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/1.jpg)
CLOUD DEVELOPMENT WITH APACHE CAMEL AND
AMAZON WEB SERVICES
@robinhowlett
Friday, May 18, 12
![Page 2: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/2.jpg)
ABOUT ME
• Robin Howlett
• Senior Architect at Silver Chalice (Boulder, CO)
• Lead for Advanced Media Platform
Friday, May 18, 12
![Page 3: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/3.jpg)
ABOUT SILVER CHALICE
• Work with sports and media entities to build digital businesses
• Focus on mobile, digital sales, video, social and digital networks, and new media technologies
• Backed by Chicago White Sox ownership
• Clients are major media and sports companies
Friday, May 18, 12
![Page 4: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/4.jpg)
WHY CAMEL + AWS?
• Startup: small team, big project, quick deadline
• Existing products in market; needed to scale on day 1
• Complex data ecosystem; variety of providers, data formats and distribution methods
• Needed expressive, powerful integration framework
Friday, May 18, 12
![Page 5: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/5.jpg)
STYLE
• Brief overviews for context
• Technical detail over broad descriptions
• Code-first approach; fewer slides
• Increasing complexity
• Personal experience
• Q&A best for business and management questions
Friday, May 18, 12
![Page 6: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/6.jpg)
AGENDA
• Introduction to Amazon Web Services
• Details of selected AWS products and architectures
• Overview of AWS SDK for Java and AWS Toolkit for Eclipse
• Demo: Camel’s AWS components
• Scaling Demo: Camel and AWS Elastic Beanstalk
• Cloud Routing Demo: Camel and AWS Simple Workflow (SWF)
• Q&A
Friday, May 18, 12
![Page 7: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/7.jpg)
AMAZON WEB SERVICES
• Pay-as-you-go
• Scalable-on-demand resources
• HA
• Global
• Computing, Storage, Middleware, Database, Networking, CDN, Security, DevOps
• Open source technologies prominent
• Web service APIs and SDKs
• IaaS, PaaS, ?aaS
Friday, May 18, 12
![Page 8: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/8.jpg)
AWS MAPFriday, May 18, 12
![Page 9: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/9.jpg)
IMPLICATIONS
• Rapid engineering
• Smaller teams with greater scope
• Budgeting
• SLAs
• Security
• Non-traditional engineering
• Complexity
Friday, May 18, 12
![Page 10: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/10.jpg)
AWS PRODUCTS & SERVICES
• EC2 & EBS
• S3 & CloudFront
• RDS, SimpleDB & DynamoDB
• SQS
• SNS
• SWF
• IAM
• EMR
• SES
• Elastic Beanstalk
• ELB & Auto-Scaling
• Route 53
• ElastiCache
• CloudSearch
• CloudFormation
• VPC
• CloudWatch
• AMIs
Friday, May 18, 12
![Page 11: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/11.jpg)
EC2
• Micro-instances to high-computing clusters
• On-demand, spot or reserved pricing options
• EBS vs Instance Stores: implications
• Templating with AMIs and CloudFormation
• Security: IAM users, key pairs and security groups
• Load Balancing and Elastic IPs
Friday, May 18, 12
![Page 12: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/12.jpg)
S3
• What’s cooler than a trillion objects?
• Buckets of objects, uniquely identified
• REST & SOAP interfaces; HTTP and BitTorrent
• SLA: 99.9% “uptime”; 99.999999% durability
• Static content, publicly accessible
• Combine with CloudFront for an instant global CDN
Friday, May 18, 12
![Page 13: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/13.jpg)
RDS, SIMPLEDB & DYNAMODB
• RDS: MySQL or Oracle relational databases
• RDS: Invisible, auto-sharding, multi-AZ deployment, read replicas
• SimpleDB: non-relational (NoSQL) key-value-based data store; basically giant tables
• SimpleDB: automatically replicated and geographically distributed
• DynamoDB: NoSQL database service; columns of key-values like SimpleDB; non-binary data
• DynamoDB: SSDs! Only 64KB limit on item/row however and complex pricing arrangement
Friday, May 18, 12
![Page 14: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/14.jpg)
SQS
• Hosted, distributed queue
• Ordering, idempotency and approximations
• Consumers control message lifecycle with visibility timeouts
• Unlimited number of messages can be processed (64KB of text max)
Friday, May 18, 12
![Page 15: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/15.jpg)
SNS
• Pub-Sub web service
• Notification protocols include HTTP(S), SQS, email, SMS
• Subscribers must opt-in (SQS is automatic)
• Great for monitoring and mobile
Friday, May 18, 12
![Page 16: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/16.jpg)
AWS SDK FOR JAVA
• https://github.com/amazonwebservices/aws-sdk-for-java
• Apache 2.0 License; some Apache Commons dependencies
• Almost all products and services supported
• Samples included
Friday, May 18, 12
![Page 17: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/17.jpg)
AWS TOOLKIT FOR ECLIPSE
• Includes AWS SDK for Java
• AWS Explorer GUI administration tool
• Go through couple of AWS samples
• Demo toolkit
Friday, May 18, 12
![Page 18: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/18.jpg)
CAMEL-AWS
• AWS-SQS (2.6.0)
• AWS-SNS (2.8.0)
• AWS-S3 (2.8.0)
• AWS-SES (2.8.4)
• AWS-SDB (2.8.4)
• AWS-DDB (2.10.0*)
Friday, May 18, 12
![Page 19: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/19.jpg)
DEMO: CAMEL-AWS COMPONENTS
Friday, May 18, 12
![Page 20: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/20.jpg)
AWS ELASTIC BEANSTALK
• Platform-as-a-Service
• Provisioning, deployment, auto-scaling policies, load balancing, monitoring
• Leverages EC2, S3, SNS, Elastic Load Balancing, and Auto-Scaling
• Linux AMI, Apache Tomcat (6/7, 32- or 64-bit), Apache Web Server, Java EE
• Demo with AWS Toolkit
Friday, May 18, 12
![Page 21: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/21.jpg)
SCALING DEMO:CAMEL & ELASTIC BEANSTALK
Friday, May 18, 12
![Page 22: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/22.jpg)
SWF
• Simple Workflow Service
• Distributed coordination hub web service
• Manages state, tasks, execution dependencies, scheduling, monitoring and history
• Console provided
• Tasks, Workers and Deciders
• Camel complements it
Friday, May 18, 12
![Page 23: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/23.jpg)
Workflow Engine
Activity Workers
Workflow Workers Workflow
Scheduler
The workflow engine is the coordination and orchestration system.
It is capable of controlling clusters of
workers, load balancing, handling distributed
exceptions and tracking workflow progress.
Workflow workers decide the order, timing and concurrency
of tasks the activity workers
work on
The workflow scheduler is
responsible for when workflows should be started
Activity workers are independent task processors. They can do a
variety of different tasks e.g. poll, store and converts providers’ data
to our reference data models
Different workers can be responsible for each processing
step, and can be combined in any way, so they are extremely
scaleable
Friday, May 18, 12
![Page 24: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/24.jpg)
AWS FLOW FRAMEWORK
• Java programming framework for SWF
• Translates method calls into schedule decisions; assigns tasks to workers
• Long running workflows, large media processing
• @Workflow and @Activities (show simple example)
Friday, May 18, 12
![Page 25: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/25.jpg)
CLOUD ROUTING DEMO:CAMEL & SWF
Friday, May 18, 12
![Page 26: Cloud Development with Camel and Amazon Web Services](https://reader034.fdocuments.in/reader034/viewer/2022042700/5552bedfb4c905920f8b4716/html5/thumbnails/26.jpg)
Q&A
Friday, May 18, 12