(CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

34
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Guy Davies, Sophos Ltd. October 2015 CMP407 Lambda as Cron Scheduling Invocations in AWS Lambda

Transcript of (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Page 1: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Guy Davies, Sophos Ltd.

October 2015

CMP407

Lambda as CronScheduling Invocations in AWS Lambda

Page 2: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

About me

• Guy Davies, Senior Systems Engineer, Sophos Ltd

Sophos use AWS extensively in our operations to support our anti-

malware, anti-spam, and threat-detection software and hardware

devices.

Also provide security products for AWS customers to complement their

cloud security profiles:

• UTM [ free trial available! ]

• Secure Server for Linux [ AMI available in Marketplace ]

www.sophos.com/aws

Page 3: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

What to expect from the session

• How to schedule tasks in AWS Lambda

• Overview of the various options available

• Building up a pure Lambda scheduling infrastructure

• Resources and templates to implement this yourself

Page 4: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Events? Scheduling?

Page 5: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Why would I want to schedule Lambda?

• Lambda designed as event-driven computing

• Event-driven computing is awesome

• Most of the time you want to trigger because something

happens…

HOWEVER

Page 6: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Why would I want to schedule Lambda?

Sometimes you just plain need to do something on a

schedule.

Examples:

• Log cleanup

• Batching up statistics

• Alarm clock

• Infrastructure automation

Page 7: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Scheduling options on traditional infrastructure

Unix

• cron: recurring tasks

*/2 * * * * do_something

• at: run a task at a specific timeat 1615 oct 7 <<< “mail –s’AWS talk!’ [email protected]

Windows

• Scheduled tasks

• AT

Page 8: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Options for scheduling Lambda functions (1)

1. Spin up an Amazon EC2 instance, and use crontab to

invoke Lambda

• Why bother running Lambda at all?

• Many folks want a pure-Lambda deployment

• Running an instance means more to manage

• Not hugely financially efficient

Page 9: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Options for scheduling Lambda functions (2)

2. Unreliable Town Clock (townclock.io)

• Awesome public Amazon SNS topic

• Chimes every fifteen minutes

• Community supported

• Has a 15 minute granularity

Page 10: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Options for scheduling Lambda functions (3)

3. Others…

• Trigger from Amazon SWF

• Trigger from an instance in AWS Data Pipeline

• Trigger from an AWS CloudTrail upload into an Amazon S3

bucket

All of these could be your solution! But what if we want a

“pure” Lambda implementation that’s managed by AWS?

Page 11: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

A pure Lambda scheduler

Page 12: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

How do we generate a timing signal in AWS?

0

0.2

0.4

0.6

0.8

1

Photo: ‘Signetics NE5555N, …’ by Stefan506 is

licenced by CC BY-SA 3.0. Source:

https://en.wikipedia.org/wiki/555_timer_IC

Amazon

CloudWatch

Page 13: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

CloudWatch as a time signal

1. Set an alarm on a CloudWatch metric

2. Alarm goes into ALARM state:

• Triggers Amazon SNS which triggers Lambda

• Lambda inverts the state of the metric

3. Alarm goes into OK state:

• Triggers SNS which triggers Lambda

• Lambda inverts the state of the metric

Page 14: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Configuring the CloudWatch alarm

Page 15: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Configuring the CloudWatch alarm

Once the metric is

inverted, the alarm will

trigger at the top of the

next minute:

1-minute resolution

Page 16: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Configuring the CloudWatch alarm

All three states trigger the

same SNS notification. The

Lambda function figures

which was the trigger and

sets the CloudWatch

metric to the opposite state

Page 17: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Putting it together

Lambda

cron

function

CloudWatch

metric

CloudWatch alarm

triggers

SNS topic10

Invoke further Lambda

functions (if scheduled)

Page 18: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

The Lambda function

Page 19: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Lambda function

1. The “event” is the SNS notification that is sent by

CloudWatch.

2. Invert the value of the CloudWatch metric.

3. Jobs and schedule are managed as a separate JSON

file in the bundle.

4. Invoke any Lambda functions in the schedule that are

scheduled to be run this minute.

5. Done.

Page 20: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Main Lambda function

exports.handler = function(event, context) {

async.waterfall([

function (callback) { flip_cloudwatch(event,callback); },

read_crontab,

execute_lambdas

], function (err) {

if (err) context.fail(err);

else context.succeed(); });

};

Page 21: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Flipping CloudWatch

The event is an SNS message from CloudWatch:

var snsmessage = JSON.parse(event.Records[0].Sns.Message);

If it’s just gone into alarm, we want to reset to zero. Likewise if it’s just gone into

OK, reset to 1.

if (snsmessage.NewStateValue == 'ALARM') { value = 0.0 }

else if (snsmessage.NewStateValue == 'OK' || snsmessage.NewStateValue ==

'INSUFFICIENT_DATA') { value = 1.0 };

Push the new value to CloudWatch:

var params = { MetricData: [ { MetricName: 'LambdaCron', Timestamp: new Date,

Unit: 'None', Value: value } ], Namespace: 'LambdaCron' };

cloudwatch.putMetricData(params, function(err, data) { . . . });

Page 22: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Crontab-like Lambda configuration

{

"jobs": [ {

"schedule": "*/3 * * * *",

"function": "testfunction",

"args": {

"key1": "test1",

"key3": "test3",

"key2": "test2"

}

} ]

}

Page 23: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Checking the schedule (1)

Use the fantastic cron-parser library (https://github.com/harrisiirak/cron-parser/

MIT licenced)

var parser = require('cron-parser');

Create a Date object which refers to the top of the current minute to compare

the schedule to.

var d = new Date();

d.setSeconds(0);

d.setMilliseconds(0);

Parse the crontab to find the ‘next’ runtime for the job (= now if it needs to run)

var interval =

parser.parseExpression(job["schedule"],{currentDate: d});

var runtime = interval.next();

Page 24: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Run each job that needs running

if (datestring == runtimestring) {

var lambda = new AWS.Lambda();

var params = {

FunctionName: job["function"],

InvocationType: "Event",

Payload: JSON.stringify(job["args"])

};

lambda.invoke(params, function(err,data) {

if (err) iteratorcallback(err);

else iteratorcallback(null);

});

}

}

Page 25: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Demo

Page 26: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Use cases

Trialling this for intelligent scaling on a schedule

• AWS scheduled scaling has limitations:

• Absolute number of desired instances

• Limit of 120 tasks per month

• Lambda function triggered by lambda-cron could do for

example:

• Every morning at 08:00 UTC, add 20% of capacity unless we

have > 30 instances already running

Page 27: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Reliability and monitoring

Empirically pretty reliable.

Running since April without (much!) intervention.

Even comes back after outages!

Page 28: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Reliability and monitoring

Monitoring: CloudWatch!

• Lambda invocation metrics will tell you that it’s running

• Application-level monitoring on the jobs you are

triggering

60 invocations / hr

Page 29: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Summary

1. Use CloudWatch alarms as two states to provide a

timing signal to Lambda.

2. Trigger off all three states to enhance reliability.

3. Allows us to schedule tasks purely within Lambda.

4. The cron function invokes once a minute.

Page 30: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Resources

Github:

github.com/g-a-d/lambda-cron

Email:

[email protected]

AWS

CloudFormation

stack

Lambda

function

Page 31: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Resources

Libraries used:

• async: https://github.com/caolan/async

• avoid nested-callback-hell (great for folks more used to

procedural programming)

• MIT license

• cron-parser: https://github.com/harrisiirak/cron-parser

• parse crontabs

• MIT license

Page 32: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Resources

Remember to check www.sophos.com/aws

• UTM for AWS (free trial available)

• Secure server for AWS

• Free trials, free home use AV

Page 33: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Remember to complete

your evaluations!

Page 34: (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Thank you!