Semantic-Based Emotional Inference and Agent Interaction Applied in Education

AUTHORS: I-HEN TSAI1, RUI-TING SUN1 , REN-YING FANG1, KOONG H.-C. LIN1, MIN-CHAI SHIEH1, JIUN-SHENG LI2, CHU-CHUAN HUANG2, JHING-FA WANG3

1NATIONAL UNIVERSITY OF TAINAN 2HCI TECHNOLOGY CENTER, ITRI 3NATIONAL CHENG KUNG UNIVERSITYPRESENTER:I-HEN TSAI

Semantic-Based Emotional Inference and Agent Interaction Applied in

Education

Overview

Abstract: In a nutshellIntroductionSystem DesignPutting together the piecesExperimentationResultsConclusionFuture Works

Abstract

A system for student interaction in education environment

Text inputInference emotion from textText to agent for visual interaction

Introduction

System to interact with students Improve concentration Bring student attention back to class

Based on analyzing text Emotion What they think of the class

Semantic Analysis

Semantic information extraction Ontology approach Connection between concepts

OMCSnet Text string input Output term relation values with target concepts Common sense inference

Dogs <bark> not <meow> Mapping rules: attributes + operations Predicates database: 250000 items

Simplified Algorithm: Sentence Parsing

Data structure requirement: 2 dequeues, digestedSymbol & digestedToken2 stacks, symbolDequeue & tokenDequeue

For each token from argv[1] to argv[n]On [* : push * into symbolDequeue, push an empty string into tokenDequeue.

If currentToken is [NP, skip to the corresponding NP]On *] : push symbolDequeue.top() into “digestedSymbol,” push tokenDequeue.top() into “digestedToken.”On * : Append currentString in tokenDequeue.top()

Algorithm: Emotional Inference

Define I[] = set of tokens translated into English.

Define E[]= { concentrate, happy, relax, easy },

emotions[sizeof(I)][4];

pathSum = 0;

for each I[i]

for each E[j]

Let D[i][j] = distance( I[i], E[j] )

pathSum += D[i][j]

end of for

for each E[j]

emotions[i][j] = (pathSum - D[i][j])/pathSum

end of for

pathSum=0

end of for

Process Flow of Semantic Analysis

Agent

Visual avatar 偽春菜 or “ukagaka” ( 伺か ) C-based system Varied interaction capabilities Can be user defined to suit need Can script wanted dialogue Interchangeable skin

System Flow

Text inputTranslationParse text to pick out significant termsMatch sig. terms with target concepts for

term relation valueDetermine concentration valuePass result to agentAgent picks dialogue based on received value

System Process Flow

Experiment Setup

Educational domain Educational institutional background and direction The speech occurrences of student chatter in class

TranslationBalanced sets

S6我吃飽了I am full

S7真想睡覺

Really want to go to bed

S8要聊天的話，請出去

To chat, please go

S9這堂課真是無聊透頂

This class is really boring the extreme

S10好無聊喔

Oh well bored

S1這次考試真容易

The examination was really easy

S2我下課後一定要讀書

I have to study after school

S3這堂課真是有趣

This class is really interesting

S4上課請專心

Please concentrate on school

S5知識就是力量

Knowledge is power

S11 我討厭上數學I hate math

S12同學們考試的時候不要東張西

望the students do not look around

during the examination

S13舉頭望老師，低頭吃午餐

raise my eyes to the teacher, bow their heads to eat lunch

S14學生上課應該要專心聽講class, the students should

concentrate on listening and speaking

S15 學習是無止盡的learning is endless

S16 學物理對我沒有用physics right I did not use

S17老師的課讓人想睡覺

the teacher's class people want to sleep

S18上課做筆記會協助記憶

class and taking notes will assist the memory

S19教室有冷氣好睡覺

Classrooms are air-conditioned sleeping well

S20同學不要趴在桌上睡覺

Students do not sleep lying on the table

Results

Main concept: concentrationIndicators: happy, easy, relaxMain concept triggers interactionIndicators allow the viewer to have some

insight of what students think of the class

Table 2. Inference Results (Concentration)

Sample Sentence

LabelInference Statistics

Concluding State

S1

concentrate:0.000000happy:0.428571relax:0.000000easy:0.571429

Not concentrating

S2


Concentrating

S3


Concentrating

S4


Concentrating

S5

concentrate:1.451128happy:1.575188relax:1.451128easy:1.522556.

Concentrating

S6


Not Concentrating

S7


Not concentrating

S8


Not concentrating

S9


Concentrating

S10


Not concentrating

S11

concentrate:0.714286happy:0.761905relax:0.761905 easy:0.761905

Concentrating

S12


Not Concentrating

S13


Not Concentrating

S14


Concentrating

S15


Not Concentrating

S16


Not Concentrating

S17


Concentrating

S18


Concentrating

S19

concentrate:0. 714286

happy:0. 785714

relax:0. 0.785714

easy:0. 714286

Concentrating

S20

concentrate:0.00000

happy: 1.303030

relax: 1.393939

easy: 1.303030

Not concentrating

Table 3. Inference Results (Mood)

Sample Sentence Label

Inference Ration Comparison

Inferred Mood State

S1 1.0:0.0:1.3 Easy

S2 1.0:1.05:1.07 Easy

S3 1.1:1.0:1.0 Happy

S4 1.0:1.0:1.1 Easy

S5 1.1:1.0:1.1 Happy

S6 1.0:1.0:1.1 Easy

S7 1.0:1.0:1.0 Happy

S8 0.0:0.0:0.0 Null

S9 1.1:1.0:1.0 Happy

S10 0.0:0.0:0.0 Null

S11 1.0:1.0:1.0 AllS12 1.0:0.0:3.7 EasyS13 1.1:1.0:1.2 EasyS14 0.0:0.0:0.0 NullS15 0.0:0.0:0.0 NullS16 0.0:0.0:0.0 NullS17 1.02:1.0:1.003 HappyS18 1.1:1.0:1.0 HappyS19 1.1:1.1:1.0 Happy, RelaxS20 1.0:1.1:1.0 Relax

Conclusions

Main issues: Translation issues OMCSnet

Concept search must be perfect match, eg. <concentrate> does not equal <concentrated>

Lack of desired link between concept Shortest path first

Agent Needs feedback mechanism

Future works

Better translation alternative(?)OMCSnet

Weighting system(?), in attempt Alternate ontology mapping structure Add more items

Agent Feedback mechanism

Interaction affects future inference Other multimedia outputs

Thank You for your attention!

Semantic-Based Emotional Inference and Agent Interaction Applied in Education

Documents

Transcript of Semantic-Based Emotional Inference and Agent Interaction Applied in Education