Dr.MarjorieSkubicandMengxuanMareu.cs.missouri.edu/REU16/Projects/Team3/Poster.pdfPowerPoint or...

1
RESEARCH POSTER PRESENTATION DESIGN © 2015 www.PosterPresentations.com In order to convert the data into a more readable form, it is then converted to hex bits, and the hex bits are interpreted according to the documentation given by the Bluetooth Smart website and Angel Sensor website. This data is then store into the Dynamo DB Database, provided by Amazon Web Services. This NoSQL database was chosen due to the simplicity of communication between the database and the Amazon Echo. ANGEL SENSOR Angel Sensor OBJECTIVE 1: Implement a smart application that monitors health status OBJECTIVES An interactive healthcare application that utilizes the Angel Sensor and Amazon Echo INTRODUCTION AMAZON ECHO A speaker device that allows for user interaction The Amazon System Needed a web server to host and run the skill program, for usage by the application EXPERIMENTS Testing the voice recognition accuracy of the Amazon Echo Dr. Marjorie Skubic and Mengxuan Ma Karen Ai and Jordan Hubbard The AngelEcho Project Angel Sensor puts data in database Program accesses data in database Amazon Echo reads data for users • Database with health info ready • Must access info Request & Response A cloud web server provides service Amazon skill Request & Response • Alexa Voice service • Provide info to users Speaks to user The Amazon Echo is a voice-enabled wireless speaker. It responds to “Alexa”, “Echo”, or “Amazon”, and is comparable to the Siri capability of the iPhone. Its many capabilities include making to-do lists, streaming podcasts, telling jokes, and accessing other real-time information. The Echo was chosen for its easy usability as well as personable communication system. Developing The Amazon Skill Amazon Echo skill: A capability that allows users to interact with the device with functionalities of Echo Alexa Skills Kit: A collection of self-service APIs, tools, documentation and code samples The Alexa Skills Kit was used to develop an Amazon Echo skill that would be used to access the correct data from the Angel Sensor and correctly interact with the user “Alexa” is the wake word “Ask…Doctor” is on of the supported phrases for requesting service “Doctor” is the invocation name that identifies the service the customer wants “What is my heart rate?” is the specific question that elicits response. “heart rate” is the keyword invoke heart rate intent. EXAMPLE QUESTION: Alexa, ask Doctor ”What is my heart rate?” Setting Up A Server User asks: What is my heart rate? Amazon Voice Services Skill on Amazon Lambda Services Angel Sensor data in database: At first the program for the skill was run on the AWS Lambda Services. We then tried to transition to an outside server and experienced difficulties. The first major issue was obtaining certification for our outside server, the second major issue was finding the correct remote endpoint to link the server to the Amazon skill. In the end, we decided to transition back to AWS Lambda services, but in the future we plan on successfully implementing the outside server. CONCLUSION Interfacing between the elderly and technology is best when simplicity is involved. Speech recognition makes it easy for an elderly human to communicate task to an electronic device. Our goal is to implement a smart app that monitors health status, combining the Angel Sensor for data collection and the Amazon Echo for voice access to health information and status. Knowing the limitations of the current state-of-the-art, we can develop a way for elderly patients to receive their health state without lifting a finger. OBJECTIVE 3: Conduct experiments to analysis the speech recognition accuracy OBJECTIVE 2: Design a system to receive health status by voice command interface We have successfully created the skeleton for a personal healthcare application utilizing the Amazon Echo and Angel Sensor. From here, there are a vast number of alterations and changes that can take place to improve the reliability and usability of this application. A possible immediate next step is setting up an outside server for running the program for this app, or finding additional health information the app could provide. The results of our experiment were conclusive in the sense that they allowed for us to taper our application to better suit the needs of our target audience. We conducted our experiment by gathering a group of subjects older than 65 and another group of subjects between the ages of 18-30. We then had each of these groups read out a list of phrases to the Amazon Echo, and using the pre-built “Simon says” skill in the Echo, we were able to get the Echo to repeat every sentence. We listened closely to the repeated phrase and marked off every word that had been picked up inaccurately, as well as the sentence structures that had been changed. The Angel Sensor is a health wearable sensor that collects health information. The sensors included are a heart rate sensor, skin temperature sensor, step counter, accelerometer waveform sensor, optical waveform sensor, and much more. The Angel Sensor was created for developers and researchers because of its open source capabilities. It provides unrestricted access to health information it collects over a Bluetooth low energy connection. Data Flow Diagram Client Device Dynamo DB Database GATT Protocol Activating/Handling Data Retrieval Write “01” and “10” to enable notifications and indications, respectively New thread is started to receive data Wearable sensor device that tracks health information Data retrieved as ascii values Ascii values are converted to hex (binascii python module) Hex bits are interpreted With this plan, health information from the Angel Sensor can be collected, processed, and effectively combined to analyze and determine a patient’s health status without interfering with their daily life. Thanks to the National Science Foundation and the University of Missouri for this opportunity! Older Adults (65 and older) Younger Adults (18 30) Male 10.9% 3.9% Female 7.66% 5.58% All 8.75% 4.6% RESULTS Percentages of Words Missed Some common words that were recognized as others are: “pause” for “pulse”, “house” or hills” for “health”. REFERENCES S. S. Ltd, "Angel Sensor – open mobile health Wearable," 2016. [Online]. Available: http://angelsensor.com. Accessed: Jul. 1, 2016. H. Huo, Y. Xu, H. Yan, S. Mubeen, and H. Zhang, “An Elderly Health Care System Using Wireless Sensor Networks at Home,” 2009 Third International Conference on Sensor Technologies and Applications, 2009. Amazon Services, "Amazon DynamoDB," Amazon Web Services, 2016. [Online]. Available: https://aws.amazon.com/dynamodb/. Accessed: Jul. 4, 2016.

Transcript of Dr.MarjorieSkubicandMengxuanMareu.cs.missouri.edu/REU16/Projects/Team3/Poster.pdfPowerPoint or...

Page 1: Dr.MarjorieSkubicandMengxuanMareu.cs.missouri.edu/REU16/Projects/Team3/Poster.pdfPowerPoint or “Print -quality” PDF. Print your poster When you are ready to have your poster printed

QUICK START (cont . )

How to change the template color themeYou can easily change the color theme of your poster by going to the DESIGN menu, click on COLORS, and choose the color theme of your choice. You can also create your own color theme.

You can also manually change the color of your background by going to VIEW > SLIDE MASTER. After you finish working on the master be sure to go to VIEW > NORMAL to continue working on your poster.

How to add TextThe template comes with a number of pre-formatted placeholders for headers and text blocks. You can add more blocks by copying and pasting the existing ones or by adding a text box from the HOME menu.

Text sizeAdjust the size of your text based on how much content you have to present. The default template text offers a good starting point. Follow the conference requirements.

How to add TablesTo add a table from scratch go to the INSERT menu and click on TABLE. A drop-down box will help you select rows and columns. You can also copy and a paste a table from Word or another PowerPoint document. A pasted table may need to be re-formatted by RIGHT-CLICK > FORMAT SHAPE, TEXT BOX, Margins.

Graphs / ChartsYou can simply copy and paste charts and graphs from Excel or Word. Some reformatting may be required depending on how the original document has been created.

How to change the column configurationRIGHT-CLICK on the poster background and select LAYOUT to see the column options available for this template. The poster columns can also be customized on the Master. VIEW > MASTER.

How to remove the info barsIf you are working in PowerPoint for Windows and have finished your poster, save as PDF and the bars will not be included. You can also delete them by going to VIEW > MASTER. On the Mac adjust the Page-Setup to match the Page-Setup in PowerPoint before you create a PDF. You can also delete them from the Slide Master.

Save your workSave your template as a PowerPoint document. For printing, save as PowerPoint or “Print-quality” PDF.

Print your posterWhen you are ready to have your poster printed go online to PosterPresentations.com and click on the “Order Your Poster” button. Choose the poster type the best suits your needs and submit your order. If you submit a PowerPoint document you will be receiving a PDF proof for your approval prior to printing. If your order is placed and paid for before noon, Pacific, Monday through Friday, your order will ship out that same day. Next day, Second day, Third day, and Free Ground services are offered. Go to PosterPresentations.com for more information.

Student discounts are available on our Facebook page.Go to PosterPresentations.com and click on the FB icon.

©  2015  PosterPresentations.com2117  Fourth  Street  ,  Unit  C                Berkeley  CA  [email protected]  POSTER  PRESENTATION  DESIGN  ©  2015

www.PosterPresentations.com

In order to convert the data into a more readable form, it is then converted to hex bits, and the hex bits are interpreted according to the documentation given by the Bluetooth Smart website and Angel Sensor website. This data is then store into the Dynamo DB

Database, provided by Amazon Web Services. This NoSQL database was chosen due to the simplicity of communication between the database and the Amazon Echo.

ANGEL  SENSOR

Angel  Sensor

OBJECTIVE 1:Implement a smart application that monitors health status

OBJECTIVES

An interactive healthcare application that utilizes the Angel Sensor and Amazon Echo

INTRODUCTION AMAZON  ECHO

A speaker device that allows for user interaction

The  Amazon  SystemNeeded a web server to host and run the skill program, for usage by the application

EXPERIMENTSTesting the voice recognition accuracy of the Amazon Echo

Dr.  Marjorie  Skubic  and  Mengxuan  Ma

Karen  Ai    and  Jordan  Hubbard

The  Angel-­‐Echo  Project

Angel Sensor  puts  data  in  database

Program  accesses data  in  database

Amazon Echo  reads  data  for  

users

• Database with  health  info  ready

• Must access  info

Request  &Response

• A  cloud  web  server  provides  service

• Amazon  skill

Request  &  Response • Alexa  Voice  

service• Provide  info  to  users

Speaks  to user

The Amazon Echo is a voice-enabled wireless speaker. It responds to “Alexa”, “Echo”, or “Amazon”, and is comparable to the Siri capability of the iPhone. Its many capabilities include making to-do lists, streaming podcasts, telling jokes, and accessing other real-time information. The Echo was chosen for its easy usability as well as personable communication system.

Developing  The  Amazon  SkillAmazon Echo skill:

A capability that allows users to interact with the device with functionalities of Echo

Alexa Skills Kit:A collection of self-service APIs, tools, documentation and code samples

The Alexa Skills Kit was used to develop an Amazon Echo skill that would be used to access the correct data from the Angel Sensor and correctly interact with the user

“Alexa” is the wake word “Ask…Doctor” is on of the supported phrases for requesting service“Doctor” is the invocation name that identifies the service the customer wants“What is my heart rate?” is the specific question that elicits response.“heart rate” is the keyword invoke heart rate intent.

EXAMPLE QUESTION: Alexa, ask Doctor ”What is my heart rate?”

Setting  Up  A  Server

User asks: What is my heart rate?

Amazon Voice Services

Skill on Amazon Lambda Services

Angel Sensor data in database:

At first the program for the skill was run on the AWS Lambda Services. We then tried to transition to an outside server and experienced difficulties. The first major issue was obtaining certification for our outside server, the second major issue was finding the correct remote endpoint to link the server to the Amazon skill. In the end, we decided to transition back to AWS Lambda services, but in the future we plan on successfully implementing the outside server.

CONCLUSION

Interfacing between the elderly and technology is best when simplicity is involved. Speech recognition makes it easy for an elderly human to communicate task to an

electronic device.Our goal is to implement a smart app that monitors health status, combining the Angel Sensor for data collection and the Amazon Echo for voice access to health information

and status. Knowing the limitations of the current state-of-the-art, we can develop a way for elderly patients to receive their health state without lifting a finger.

OBJECTIVE 3:Conduct experiments to analysis the speech recognition accuracy

OBJECTIVE 2:Design a system to receive health status by voice command interface

We have successfully created the skeleton for a personal healthcare application utilizing the Amazon Echo and Angel Sensor. From here, there are a vast number of alterations and changes that can take place to improve the reliability and usability of this application. A

possible immediate next step is setting up an outside server for running the program for this app, or finding additional health information the app could provide. The results of our

experiment were conclusive in the sense that they allowed for us to taper our application to better suit the needs of our target audience.

We conducted our experiment by gathering a group of subjects older than 65 and another group of subjects between the ages of 18-30. We then had each of these groups read out a

list of phrases to the Amazon Echo, and using the pre-built “Simon says” skill in the Echo, we were able to get the Echo to repeat every sentence. We listened closely to the repeated

phrase and marked off every word that had been picked up inaccurately, as well as the sentence structures that had been changed.

The Angel Sensor is a health wearable sensor that collects health

information. The sensors included are a heart rate sensor, skin temperature sensor, step counter, accelerometer waveform sensor, optical waveform sensor, and much more. The Angel Sensor was created for developers and researchers because of its open

source capabilities. It provides unrestricted access to health information it collects over a

Bluetooth low energy connection.

Data  Flow  Diagram

Client  Device

Dynamo  DB  Database

GATT  Protocol

Activating/Handling  Data  RetrievalWrite “01” and “10” to enable notifications and indications,

respectively

New thread is started to receive data

Wearable sensor device that tracks health information

Data  retrieved  as  ascii  values

Ascii  values  are  converted  to  hex  (binascii  python  

module)

Hex  bits  are  interpreted

With this plan, health information from the Angel Sensor can be collected, processed, and effectively combined to analyze and

determine a patient’s health status without interfering with their daily life.

Thanks to the National Science Foundation and the University of

Missouri for this opportunity!

Older  Adults  (65  and  older) Younger Adults  (18  -­‐ 30)

Male 10.9% 3.9%Female 7.66% 5.58%All 8.75% 4.6%

RESULTSPercentages of Words Missed

Some common words that were recognized as others are: “pause” for “pulse”, “house” or “hills” for “health”.

REFERENCESS. S. Ltd, "Angel Sensor – open mobile health Wearable," 2016. [Online]. Available:

http://angelsensor.com. Accessed: Jul. 1, 2016.

H. Huo, Y. Xu, H. Yan, S. Mubeen, and H. Zhang, “An Elderly Health Care System Using Wireless Sensor Networks at Home,” 2009 Third International Conference on Sensor

Technologies and Applications, 2009.

Amazon Services, "Amazon DynamoDB," Amazon Web Services, 2016. [Online]. Available: https://aws.amazon.com/dynamodb/. Accessed: Jul. 4, 2016.