Introducing Amazon Simple Workflow (Amazon SWF)

© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Amazon SWFhttp://aws.amazon.com/swf

Balan SubramanianSr. Product Manager

[email protected]

© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.2

Introducing Amazon SWF

Simple Workflow Service

Enables scalable, resilient, truly distributed applications

Run application workflows and business processes in the cloud

Durable, distributed state management

Accessible anywhere from any programming language

Programming model and framework


When to use Amazon SWF

Dispatch tasks to application components

Coordinate distributed application processing

Implement ordered execution of application steps

Manage application state during distributed execution

Reliable, auditable periodic execution

Asynchronous invocation and event-driven execution


Why we built Amazon SWF

Application control flow logic = workflow

Orchestration: foundational need for business process implementations and distributed applications

Reliable task dispatch & state management are difficult

Lack of task based programming models

Developers shouldn’t have to build plumbing for orchestration


Emerging Needs

Asynchronous interactions and loosely coupled design

Applications that run across datacenters and clouds

Efficient resource usage, selective scalability


Amazon SWF for cloud applications

Build applications with components that run anywhere, anytime and even intermittentlyThink tasks, not messagesFocus on application logic and save development effortTake applications to market fasterScale gradually and pay only for what you useUse resources efficiently by scaling selectivelyReduce infrastructure and administration costs – you no longer need to run orchestration infrastructure


Business processes with Amazon SWF

A reliable and durable foundation for your implementations

Elastic scalability for business process executions

Build tailor made implementations and add your own value

Start with minimal cost, Scale economically

Remove infrastructure and administration costs

Leverage SWF from your existing toolsets, modeling frameworks

Partner with SWF and drive your own innovation!

© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

How Amazon SWF works

Amazon SWF

Execution History- Input data (Video URL)

- Encode task complete

- DRM task complete

- Execution complete

Decisions:

1. Schedule encoding of the video

2. Schedule DRM protection3. Complete execution

DECIDERMakes decisions on what tasks to schedule, when, in what order

Example use case: Video transcoding application

Start > Encode video > Add DRM protection > Finish

Start Workflow Execution

Web Server Front End

Accepts video URL, Starts execution with URL

Worker for adding DRM

Worker for encoding

Long poll

Long Poll

Long Poll

DRM Tasks

Encode Tasks

Decision TasksGet task

Get task

Get task

Return results

Return results

Return decisions


What Amazon SWF provides

Non-duplicated, at most once task delivery

Task queuing and routing

Timeout and execution status tracking

Consistent state management

Individually tracked executions and tasks

Durable long running executions


Using Amazon SWF

Determine application flow and activities

Setup domains

Determine workflow and activity types

Write workers for activities

Write deciders to drive

application flow

Write workflow initiator code

Start executions

View executions in the console

Start workers and deciders


AWS Management Console

Manage domains and types

View and work with executions

Fine grained execution details

Audit retained workflows

Use workflow execution metrics to monitor and react


Features

Timers

Signals

Tags

Markers


More Features

Child workflow executions

User data and execution results

Flexible task distribution with task lists

Retention


AWS Flow FrameworkEntire workflow looks like simple, sequential code

Abstracts polling, scheduling tasks, remote execution

Easy to program complex dependencies

Simple method call to asynchronously execute an application step

Express dependencies by passing output of one task as input of another.

Handle remote failures as standard exceptions

Uses Amazon SWF as the backend

Open-sourced on Github.

Currently in Java. Soon in more languages.


Pricing

Pay for application ‘resources’ when you use them

How many workflows you run, what you do when they run, how long they run, how long they are retained

$1 for 10,000 workflow executions when started

$0.25 for every 10,000 chargeable actions

$0.05 per day for every 10,000 workflows that run past a day or are retained


Pricing Example

25,000 files encoded and protected, each workflow runs for 12 hours and execution history is retained for 2 days

Starting 25,000 workflow executions: 25,000 x $1/10,000 = $2.50

Scheduling 10 tasks per workflow: 10 x 25,000 x $0.25/10,000 = $6.25

Running 25,000 workflow executions (12 hours each) = $0.00

Retaining 25,000 workflows for 2 days: 25,000 x $0.05/10,000 x 2 = $0.25

Total: $9


Pricing in different usage scenarios

Short running application executions• Number of application workflow executions• Application steps executed for each application execution• # of days for which you retain each execution trace for debugging

Long running business processes• Number of workflows started• Days, weeks, or months that the workflow runs• Tasks scheduled, timers set, # of checkpoints (markers), signals• # of days executions are retained for auditing/analytics


Use cases


Amazon SWF @ Sage Bionetworks

Cross-cloud execution

Controlled task routing

Long running executions

Visibility

Support for different programming languages

Case Study

http://aws.amazon.com/swf/testimonials/swfsagebio/


Amazon SWF @ NASA JPL

Process gigapixel Mars rover images in minutes

Quickly move processing to cloud on demand

Complex data processing workflows

Parallel image processing

Infrastructure automation

Case study

http://aws.amazon.com/swf/testimonials/swfnasa/


Getting Started

Console sample walkthrough

Flow framework samples

Free Tier

Forum

http://console.aws.amazon.com/swf/home

http://aws.amazon.com/code/Amazon-SWF?browse=1

http://aws.amazon.com/swf/pricing/

https://forums.aws.amazon.com/forum.jspa?forumID=133


Thank You!

For more information:

aws.amazon.com/swf

[email protected]

Introducing Amazon Simple Workflow (Amazon SWF)

Business

Transcript of Introducing Amazon Simple Workflow (Amazon SWF)