Post on 06-Jan-2017
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
HLC304
Building IoT Applications
with AWS and Amazon Echo
November 28, 2016
Chris McCurdy - AWS Specialist SA in Healthcare and Life Sciences
Nitin Gujral – Directory of Software Engineering and Innovation at Boston Children's
What to Expect from the Session
• What is IoT and what are some patterns that are being
used in industry?
• What are some alternatives if I just want health sensor-
related data?
• How is KidsMD using Alexa to help improve home care?
What is IoT?
What is IoT?
What is AWS IoT?
What is AWS IoT?
What is AWS IoT?
What does AWS IoT consist of?
Security First• All traffic to and from AWS IoT must be encrypted
over Transport Layer Security (TLS)• AWS Managed Certificates
• Certificate Signing Request (CSR) Certificates
• Just-in-time (JIT) Registration
• Protect your certificates• chroot
• Security Enhanced Linux (SELinux)
• One-Time Programmable (OTP) Fuses
• Trusted Platform Modules
• Smartcards
• Locks and Boxes
• FIPS-style hardware
What does AWS IoT consist of?
Thing Gateway
What does AWS IoT consist of?
Thing Gateway
Supported Protocols
HTTPS, Websockets, Secure MQTT
What is MQTT?
A lightweight pub/sub protocol, designed to minimize network bandwidth and
device resource requirements. MQTT supports TLS for encryption.
What does AWS IoT consist of?
Thing Gateway var device = awsIot.device({
keyPath: 'private_key.pem',
certPath: 'certificate.pem',
caPath: 'root.pem',
clientId: clientId,
region: 'us-east-1' });
var button = new groveSensor.GroveButton(6);
function readButtonSensorValue() {
var buttonState = button.value();
if (buttonState == 1) {
device.publish(topic, JSON.stringify({
SubmitTime: (new Date()).toJSON()}));
}
}
setInterval(readButtonSensorValue, 100);
What does AWS IoT consist of?
IoT Rules Engine
Amazon
Kinesis
Amazon S3
bucket
What does AWS IoT consist of?
IoT Rule Engine• Augment or filter data received from a device.
• Write data received from a device to an Amazon DynamoDB
database.
• Save a file to Amazon S3.
• Send a push notification to all users using Amazon SNS.
• Publish data to an Amazon SQS queue.
• Invoke a Lambda function to extract data.
• Process messages from a large number of devices using
Amazon Kinesis.
• Send data to the Amazon Elasticsearch Service.
• Capture an Amazon CloudWatch metric.
• Change an Amazon CloudWatch alarm.
• Send the data to Amazon Machine Learning to make
predictions based on an Amazon ML model.
What does AWS IoT consist of?
IoT Rule Engine
{
"sql": "SELECT * FROM 'Sensor/DispensingSensor'",
"ruleDisabled": false,
"actions": [
{
"kinesis": {
"roleArn": "arn:aws:iam::XX:role/sensor_status_role",
"streamName": "sensor_status_stream",
"partitionKey": "${ClientID}"
}
}]
}
What does AWS IoT consist of?
Thing Registry
• Associate MQTT Client IDs
• Associate Certificates
• Specify up to 3 custom attributes
What does AWS IoT consist of?
Thing Registry
{
"version": 3,
"thingName": "PortableDispenser",
"defaultClientId": "abc123",
"thingTypeName": "Dispenser",
"attributes": {
"model": "123",
"dispensingVolume": "75"
}
}
What does AWS IoT consist of?
Thing Registry
response = client.update_thing(
thingName=PortableDispenser',
attributePayload={
'attributes': {
'dispensingVolume': '150'
},
},
expectedVersion=2,
)
What does AWS IoT consist of?
Thing Shadow
• Store and retrieve current and future state of the
thing
• Simple JSON Document
• Get and set the state of a thing over MQTT or
HTTP
• Access the shadow regardless of whether the
thing is connected to the Internet
What does AWS IoT consist of?
Thing Shadow{
"state" : {
“desired" : {
"sensor" : "ON"
},
"reported" : {
"sensor" : "OFF"
},
"delta" : {
"sensor" : "ON"
} },
"version" : 10
}
What does AWS IoT consist of?
Thing Shadow
data = {
"state" : {
"desired" : {
"switch" : "on"
}
}
}
response = client.update_thing_shadow(
thingName = PortableDispenser',
payload = mypayload
)
Other services in an scalable IoT architecture
Kinesis
• Streams are made of Shards
• Each Shard ingests data up to 1 MB/sec, and up to 1000
TPS
• Each Shard emits up to 2 MB/sec
• All data is stored for 24 hours – 7 days
• Scale Kinesis streams by splitting or merging Shards
• Replay data inside of 24Hr -7days Window
Other services in an scalable IoT architecture
Amazon DynamoDB
• Provisioned at the table level• Write capacity units (WCUs) are measured in 1 KB per
second
• Read capacity units (RCUs) are measured in 4 KB per
second
• RCUs measure strictly consistent reads
• Eventually consistent reads cost ½ of constant
reads
• Read and write throughput limits are independent
• Increase as necessary, decrease at most 4 times per
UTC day
Other services in an scalable IoT architecture
Lambda
• AWS Lambda offers 23 "power levels"
• Higher levels offer more memory and more CPU
power
• 128 MB, lowest CPU power
• 1.5 GB, highest CPU power
• Compute price scales with the power level
• Duration ranging from 100 ms to 5 minutes
Example IoT architecture
IoT MQTT
protocol
IoT
certificateIoT
ruleIoT
topic
Amazon
Kinesis
AWS
Lambda Amazon
DynamoDB
Amazon
SNS
Alexa
Medication Status
monitoring device
Medication Status Backend
Node.js
AWS
LambdaAWS
Lambda
Elephant in the room
http://nos.twnsnd.co/post/104252656546/elephants-tea-party-robur-tea-room-24-march
Amazon
Kinesis
AWS
Lambda
Amazon
DynamoDB
Amazon
SNS
Alexa
AWS IoT
HIPAA Eligible Not HIPAA Eligible
Example IoT (PHI) Architecture
IoT MQTT
protocol
IoT
certificateIoT
ruleIoT
topic
Amazon
Kinesis Amazon
DynamoDB
Alexa
Medication Status
monitoring device
Medication Status Backend
Node.js
AWS
Lambda
Amazon
S3
Amazon EC2
client
Amazon EC2
AWS
Lambda
SQS
Validic Digital Health Platform
Clinical, Fitness, Wellness and
Nutritional Devices, Wearables,
Applications, and Equipment
Healthcare Providers, Clinical
Research, Wellness Apps, Insurance,
Academia
Validic architecture
3rd Party Devices,
Apps, and APIs
Validic Core Platform
VitalSnap™ OCR
Image Algorithm
Calibration
Validic
Standardization
Services
Health Data
Retrieval
Services
Admin
Database
Primary
Database
Replica
Databases
Validic API
Healthcare Systems,
Clinical Trials Platforms,
Wellness Applications
Patient / Individual
Alexa Voice Service in Healthcare
Nitin Gujral
Director of Software Engineering, Innovation and Digital Health
Boston Children’s Hospital
#1 ranked pediatric hospital
nationwide (8 out of 10
specialties)
400 plus bed; children's
hospital affiliated with Harvard
Medical School
Largest recipient of NIH funds
in pediatric-focused research
Innovation and Digital Health AcceleratorShaping the future of health care and extending Boston Children’s pediatric
leadership globally
BCH DataClinical Expertise HCIT Development
Products Platforms Ventures
Boston Children’s Expertise & Capabilities:
+Industry Partners & Startups, like:
=To launch disruptive health care focused:
Learn more:
bostonchildrens.org/accelerator
GE GrandRoundsCerner
IBM Watson Health Rock Health
Ask the Internet? –Parents are actively looking for guidance—traditional medical content sites lack
the trustworthy and readily accessible health guidance.
Although I will ALWAYS go with my gut about my kids
being sick, I typically ask the internet to confirm my
suspicion about what might be happening with them.
– Female, 36, Kids ages 1 & 4
Source: 3rd Party consumer research survey (n=200)
Decision support in consumers’ hands
Boston Children’s
Data & Expertise
Withings
Raiing
Kids.MD™ is delivering consumer facing digital health tools to patients and families across digital mediums.
Kids.MD™
powered by:
Boston Children’s Hospital
web
mobile
voice
device
research
platform
“Alexa, ask Kids.MD”
Kids.MD – Alexa Skill Architecture
AVS +
Kids.MD
“Alexa, ask Kids.MD about fever”
Skill identified in Alexa
voice service
Request sent to Kids.MD
clinical decision support
Sends
‘recommendations’ back
to Alexa voice service
Alexa converts ‘text’ to
spoken words and
streams audio
Or access
via mobile &
web app
Kids.MD™
Electronic Health
Record
Patient
Portal
Kids.MD on AWS
Kids.MD™
Amazon RDS Amazon S3
AWS Lambda API Gateway Amazon EC2
Amazon Dynamo DB
Kids.MD Skill: Development
Node Module - alexa-sdk
Handlers for intents
Node Module - alexa-sdk
Custom events
Node Module - alexa-sdk
State Machine
Context persistence using DynamoDB
Node Module - alexa-sdk
Setting context
Response
Project structure
• Structure backend code; e.g.,
handlers, model and other assets
- including tests
• Organize Alexa Skills Kit speech
assets
Debugging
Locally debug AWS Lambda code
Reference:
Blog by Nathan Grice (https://developer.amazon.com/public/community/post/Tx24Z2QZP5RRTG1/New-Alexa-
Technical-Tutorial-Debugging-AWS-Lambda-Code-Locally
• within any IDE
supporting
Node.JS
• simulate AVS
request /
response
Unit testing
Javascript testing with Jasmine (a behavior driven framework)
https://jasmine.github.io/
NPM module - jasmine or jasmine-node
Ability to mock requests and responses
Reference:
Article by Jeremy Bohrer: https://medium.com/@jjbskir/unit-testing-an-amazon-alexa-skill-with-node-js-and-
jasmine-98982544471f
Integration testing – developer tools
Lambda deployment
• NPM package
• Leverages AWS SDK
• Packaging for Lambda
• Upload to Lambda
• Supports unit testing as well
https://github.com/Tim-B/grunt-aws-lambda
Kids.MD Skill: technical
considerations
States and decision trees
Managing context in an Alexa skill
• short time frame for the duration of skill invocation
• access to user and application info
• persistent context for a user
Intent Schema Slot types
• better voice recognition
• more accuracy with user response
• supports free form responses
Custom Slot Types
Built-in Slot Types
• e.g. AMAZON.NUMBER, AMAZON.DATE,
AMAZON.DURATION
Speech Synthesis Markup Language (SSML)
• for more natural speech
• pauses and predefined formats (characters, digits, date, telephone,
address, etc.)
<s>Let us start again.</s>
<p>Would you like to hear this information again?</p>
<break strength='weak'/>
<say-as interpret-as="digits">101.4</say-as>
Speech Synthesis Markup Language (SSML)
• improved pronunciation using “phoneme”
• specifically for medical terminology
<phoneme alphabet=\"ipa\"
ph=\"sɪmp.təms\">symptoms</phoneme>
<phoneme alphabet=\"ipa\" ph=\"moətrɪŋ\">Motrin</phoneme>
Logging, auditing, and monitoring
• AWS Lambda CloudWatch logs and metrics
• Custom error logs
• Custom auditing of user interactions
• Monitoring of various service components
Account linking
• Link an Alexa user to the user of your system
• Enables contextual interactions based on user’s preferences
• Performed at the time of skill installation
• Using OAuth authorization flow
Reference:
https://developer.amazon.com/public/community/post/Tx3CX1ETRZZ2NPC/Alexa-Account-Linking-5-
Steps-to-Seamlessly-Link-Your-Alexa-Skill-with-Login-wit
Images and audio in Alexa Skills
• Display images in Cards
• JPG or PNG (less than 2 MB) Need CORS support for image URLS
• SSML Audio Tags <audio>
• MP3 support
• 90 sec limit
• Audio Streaming
• Leverage built-in intents for playback
Reference: https://developer.amazon.com/public/community/post/Tx1DSINBM8LUNHY/New-Alexa-Skills-Kit-ASK-
Feature-Audio-Streaming-in-Alexa-Skills
Voice design – lessons learned
• Maintain intuitive conversation
• provide guidance via prompts
• Balance the amount of verbal responses
• Effectively utilize Companion App(s)
• with images, audio and streaming sound
Some enterprise deployment challenges
Security
Connectivity
Voice in healthcare A new way for patients and clinicians to access health information and
personalized care in various settings
@ home
@ inpatient
@ outpatient
by patients & families
by clinicians
Context: Users:
Photo by John Blanding for The Boston Globe
Example usage:
• Home Health Hubs
• At the Bedside for Patient Care
• Virtual Assistant for Clinicians
Voice in healthcare – a new mediumStreamline and ease delivery of health care services
“Alexa, ask Children’s Hospital”
Growth of “Internet of Healthcare”
Promise of connected healthcare
Portable
Proactive
Patient Centric
Thank you!
Remember to complete
your evaluations!