AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

29
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Marc Brooker, Principal Engineer, AWS 30 November 2016 IOT309 NEW LAUNCH! Lambda Everywhere

Transcript of AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Page 1: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Marc Brooker, Principal Engineer, AWS

30 November 2016

IOT309

NEW LAUNCH! Lambda Everywhere

Page 2: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

What to Expect from the Session

• Design a single architecture spanning devices and the cloud

• Build AWS Lambda functions for the cloud, and for AWS Greengrass

• Work through an example architecture

• Lots of Python Code

Page 3: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Part One: What We’re Building

Page 4: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Fleet of Sensors

OperatorAmazon

SNS

AWS

LambdaAWS

IoT

AWS

Greengrass

Core

Amazon

DynamoDB

Page 5: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Example: Combining Readings From Multiple Sensors

Temperature

Humidity

Page 6: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Example: Combining Readings From Multiple Sensors

Temperature

Humidity

Page 7: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Part Two: The Cloud

Page 8: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

AWS Lambda lets you run code without provisioning

or managing servers.

Just upload your code and Lambda takes care of everything required to run and scale your code with high availability.

You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app.

Page 9: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Fleet of Sensors

OperatorAmazon

SNS

AWS

LambdaAWS

IoT

AWS

Greengrass

Core

Amazon

DynamoDB

Page 10: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

IoT Event Loading

# Extract the building ID, temp. and humidity from the IoT event

# Our GGC packs its data up in JSON form

building = event["building_id"]

new_temp = float(event["max_temp"])

new_humidity = float(event["max_humidity"])

Page 11: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Lambda Performance Tip

import boto3

print 'Runs Once On Container Start'

dynamodb = boto3.client('dynamodb')

def lambda_handler(event, context):

print 'Runs on every invoke’

This code runs once per container start.

This code runs every invoke.

Page 12: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Lambda Performance Tip

import boto3

print 'Runs Once On Container Start'

dynamodb = boto3.client('dynamodb')

def lambda_handler(event, context):

print 'Runs on every invoke’

Create Your Clients Here

Not Here

Page 13: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

# Load the building configuration from DynamoDBdef load_building_config_from_dynamodb(building_id):

response = config_table.get_item(Key = { 'building' : building_id } )

if 'Item' in response:# The building configuration exists, use itreturn (float(response['Item']['min_temp']),

float(response['Item']['max_humidity']))

else:# Use default values if no config existsreturn (default_min_temp, default_max_humidity)

Loading Config and History with Amazon DynamoDB

Page 14: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

# Apply the alarm threshold logicdef should_alarm(temp, humidity, config):

return temp < config[0] and humidity > config[1]

# Load configuration and previous values from DynamoDBlast_temp, last_humidity = load_previous_from_dynamodb(building)config = load_building_config_from_dynamodb(building)

# Send an SNS notification if the values move over the thresholdif should_alarm(new_temp, new_humidity, config)

and not should_alarm(last_temp, last_humidity, config):send_sns_notification(building, new_temp, last_temp,

new_humidity, last_humidity)

Apply the Latching Business Logic

Page 15: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

# Send a notification to the SNS topic, formatted as JSONdef send_sns_notification(building, t, last_t, h, last_h):

notification = json.dumps({ "building" : building,"current_values" : { "temp" : t, "humidity" : h },"last_values" : { "temp" : last_t, "humidity" : last_h }

})

sns.publish(TopicArn = topic_arn,MessageStructure = 'json',Message = json.dumps({ 'default' : notification })

)

Sending a Notification with Amazon SNS

Page 16: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Fleet of Sensors

OperatorAmazon

SNS

AWS

LambdaAWS

IoT

AWS

Greengrass

Core

Amazon

DynamoDB

AWS

Lambda

Amazon

Redshift

Page 17: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Part Three: Local Filtering

Page 18: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Fleet of Sensors

OperatorAmazon

SNS

AWS

LambdaAWS

IoT

AWS

Greengrass

Core

Amazon

DynamoDB

Page 19: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Fleet of Sensors

OperatorAmazon

SNS

AWS

LambdaAWS

IoT

AWS

Greengrass

Core

Amazon

DynamoDB

Page 20: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Loading and Parsing A Shadow

def current_record_data(cls):

try:

# The get_thing_shadow API loads a shadow

shadow = iot_client.get_thing_shadow(thingName=cls.name)['payload']

record_shadow = json.loads(shadow)

version = record_shadow['state'].get('version', 0)

return record_value, last_push_time, version

except ShadowError as e:

# The shadow doesn't exist yet, so return some default values

if str(e).startswith('Request for shadow state'):

return None, None, 0

else:

raise e

Page 21: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Lambda

Function

MQTT

TopicIoT

Sensor

Shadow to Record

High Water Mark

Shadow to Record

Last Publish Time

MQTT Topic

Publish to AWS IoT

AWS

IoT

Page 22: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Updating a Shadow

def update_shadow(self):# Loop until shadow state update successfully occurswhile True:current_shadow, last_push, version = self.current_record_data()

if not self.should_update(self, current_shadow):return False

new_shadow = self.new_shadow(version)if self._update_shadow(new_shadow):return True

# Sleep with jittertime.sleep(random.random())

Page 23: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Publish an MQTT Message to the Cloud

def push_record(self):# Loop until shadow state update is successfulwhile True:record, last_push_time, version = self.current_record_data()

if not self._should_push_record(last_push_time):# Another Lambda has already posted. Job done!return False

# Try to update the thing shadowif self._try_update_shadow(version):# If that works, publish the messageiot_client.publish(topic=self.record_reporter_topic,

payload=str(record))return True

# Sleep for a short while, with jittertime.sleep(random.random())

Page 24: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Lambda

Function

MQTT

TopicIoT

Sensor

Shadow to Record

High Water Mark

Shadow to Record

Last Publish Time

MQTT Topic

Publish to AWS IoT

AWS

IoT

Shadow with

Configuration

Page 25: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Greengrass API Highlights

• Shadow API• Communication with devices

• Communication with AWS IoT

• Small local database

• Messaging API• Communication with devices

• Communication with AWS IoT

Page 26: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

In the Cloud and in Greengrass: What’s The Same

• Python 2.7 support

• Event-based programming model

• Container lifecycle

Page 27: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

What’s Different in Greengrass

• Local event sources (MQTT message routing)

• IoT-centric deployment and operational tools

• Authentication and authorization designed for devices

• ARM and x86 support

Page 28: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Thank you!

Page 29: AWS re:Invent 2016: NEW LAUNCH! Lambda Everywhere (IOT309)

Remember to complete your evaluations!