oad Wrapup, Questions & Exam...
20
Cognitive Modeling: Multitasking & Workload 1/26 Wrapup, Questions & Exam Info Felix Putze, Dominic Heger 14.7.2011 Lecture „Cognitive Modeling“ SS 2011
Transcript of oad Wrapup, Questions & Exam...
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
1/26
Wrapup, Questions & Exam Info
Felix Putze, Dominic Heger
14.7.2011
Lecture „Cognitive Modeling“
SS 2011
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
2/26
What did we cover?
• Introduction • Cognition, Behaviorism, Cognitive Science, cognitive revolution, Turing
test, neurological basics, measuring brain activity
• Cognitive architectures • Definition, requirements, types of architectures, ACT-R (chunks,
production rules, goal, utility, …), evaluation of architectures, PSI (motivation, emotion, …)
• Memory Modeling & Knowledge Representation • Types of memory, three different memory models, knowledge
representation (logic, frames, semantic networks, neural networks, Bayesian networks)
• Empirical Cognitive Models • General architecture (multimodal sensor data, preprocessing, feature
extraction, recognition, fusion), integration with cognitive architectures
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
3/26
What did we cover?
• Multitasking & Workload • Dual tasks, Wickens’ multi-resource model, Threaded cognition,
measuring workload, predicting workload
• Attention & Visual Perception • Bottom up vs. top down processing, Broadband’s filter theory, Anne
Triesman’s feature integration, iCub’s attention model, guided search model
• Game Theory & Reinforcement Learning • Utility functions, game theory, Nash equilibrium, Markov decision
process, Q-Learning, Dopamine
• Affective Models • Affective computing, Emotion models (descriptive models, appraisal
theories), Big Five personality theory, affect recognition, affect expression
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
4/26
What did we cover?
• Human Learning & Machine Learning • Differences between HL and ML, biological vs. artificial neural nets,
Hebbian learning, hopfield nets, conditioning, programming by demonstration, human learning theory, Leo‘s social interactive learninig, learning in ACT-R
• Human Behavior Models • Huitt‘s systems approach to human behavior, Emotion and decision
making (four theories), Behavior in social context, Social network analysis
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
5/26
Exam
• Oral exam, ~15 minutes
• If you haven‘t done already, make an appointment with Frau Scherer ([email protected]) • New appointments will not be available before the start of next
semester
• How to prepare? • Read the slides again, together with your notes from the lecture
• Additional material may help to get a deeper understanding
• Form learning groups, simulate exam situations together
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
6/26
What do I need to learn?
• The exam will be based on the content of the slides as well as the lecture (of course you did take notes!)
• However, it is not sufficient to just memorize every word on the slides • You should be able to explain any concept mentioned on the slides. If
you lack the necessary background, use additional resources to cover this ground
• You will get asked transfer questions which ask for things which are not explicitly on the slides (e.g. think of example for a concept, compare two methods, argue how two approaches can be combined, …)
• Yes, we may ask formalisms (e.g. formulas, algorithms, …) • There are not that many, after all
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
7/26
How to make a good impression
• For important concepts, think of examples and applications that were not given in the lecture • Shows that you have really understood the material
• This is not a history test, but: Know the important names and dates (e.g. when did the cognitive revolution start?)
• Find connections between the lectures on different topics, try to have a big picture of all the presented building blocks
• Think of applications for developers of computer systems or designers of man-machine interfaces
• Know the technical terms (e.g. names of experiments, methods, models) • Helps you to give precise, on-the-spot answers
• Remember that you only have 15 minutes to show what you have learned!
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
8/26
Demo Questions
• On the next slides, we give some questions on important topics of the lecture
• They are not (!!!) representative for the true exam questions • They do not cover everything that will get asked
• The final questions may be easier, harder or in a completely different style
• However, if you are well-prepared, you should be able to answer them without consulting your material • Some questions may take more than a moment to come up with a
good answer
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
9/26
• History: Describe the relation of Behaviorism and Cognitive Science during the cognitive revolution and today!
• Game Theory: Consider the game “Chicken”: • http://en.wikipedia.org/wiki/Chicken_(game)#Popular_versions
(first, only read the introduction)
• How does the utility function of both players look like?
• Is there a dominant strategy? How many deterministic Nash equilibria are there? (Bonus: How does optimal play look like?)
• Reinforcement Learning: Discuss which of the general rules on human learning also hold for Reinforcement Learning as described on the slides!
• Multitasking: Analyze a dual task using Wickens’ multi-resource model! • a) Driving vs. typing in a text message on a mobile phone
• b) Chatting on the phone vs. solving Rubik’s cube
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
10/26
• General: A customer asks you to improve his virtual learning environment (a 3D avatar teaches math by asking questions, evaluating answers and gives hints) • How could you use cognitive models for this task?
• Which models would you propose as most promising?
• Cognitive architectures: Describe how the learning of production rules in ACT-R models automation of behavior!
• Appraisal models: Consider different variants of a exam situation: • a) Taking the exam unprepared, confronted with unexpected questions
• b) Taking an exam well prepared with a supportive examiner
• Model the experienced emotions using both the OCC model and Scherer’s model. Is one model better suited for the task then the other? In what way?
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
11/26
• Human/Machine Learning: You have a Hopfield net with 3 neurons and you train it with the examples (1 1 1), and (-1 -1 -1). Describe what recall happens in the 3 conditions when you activate one neuron and deactivate the others!
• Attention/Learning: Describe how attention modeling can help a machine in learning?
• Human/Machine Learning: What changes occur in Leo’s Task model when you teach him “turn one button on, off, and on again”?
• Attention: Compare what you’ve learned about the human visual system to David Marr’s perception model!
• Attention: Explain how Wolfe’s Guided Visual Search can be implemented in a humanoid robot. Describe a situation where the robot’s behavior would be different than the one of an iCub. When can top-down knowledge improve performance?
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
12/26
Student Research at CSL: Workload Prediction
• Want to estimate mental workload of the user
• Model the human mind as queuing network
• Calculate throughput and determine bottlenecks
• Available as study-, diploma-, bachelor- or master thesis
• Necessary: Good programming skills (pref. Java) and interest in researching cognitive models
• Contact: [email protected]
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
13/26
Student Research at CSL: Gaze+EEG
• From recorded signals of a person, we can draw conclusion on its visual attention: • From gaze, we can estimate the spatial focus of attention
• From gaze patterns, we can estimate the current visual task (e.g. tracking, searching, …)
• From EEG, we can estimate attention level and sudden stimuli
• Combine those signals to create a “neuro-visual mouse” • From a video stream, a person has to identify and mark certain items
• Mouse interaction has proven inefficient, gaze + gutton pressing better
• Your task: Combine EEG and natural gaze to identify the objects which draw the person’s attention no manual interaction required anymore!
• Available as diploma or master thesis
• In cooperation with the Fraunhofer IOSB
• Contact: [email protected]
+
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
14/26
Student Research at CSL: CM+EEG
• For HCI, it is relevant to detect whether a decision situation for the user is routine or novel/difficult • We can estimate this by comparing EEG patterns of trained and
untrained situations
• We can estimate this using a cognitive model based on Reinforcement Learning (entropy of score distribution, frequency of state visits, …)
• The goal of this work is to bring together both approaches: • In a cognitive multi-step decision task (maze solving)
• Model the task using a RL approach
• Record EEG of people actually solving this task
• Combine the prediction of the model and the estimation from the EEG
• Available as study-, diploma-, bachelor- or master thesis
• In cooperation with the institute of psychology in Heidelberg
• Contact: [email protected]
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
15/26
Student Research at CSL: Feature Selection
• Existing biosignal processing framework provides a large set of different features (esp. for EEG) • Each feature captures a specific property of the signal
• Most features calculated for single electrode multiply number by 16
• For each application, we need to identify a reliable and robust feature set for classification • Detection of motor activity requires other features and locations than
estimation of cognitive workload
• Goal of this work: Improve feature selection • A number of established methods are available
• Task is: systematic evaluation, implementation of additional methods and their extension to increase stability e.g. penalize features which only work well for few participants/situations
• Available as study-, diploma-, bachelor- or master thesis
• Contact: [email protected]
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
16/26
EEG based Emotion Recognition
• Entwicklung eines multimodalen (EEG, PPG, EDA) Erkennungssystems für Emotionen
• Emotionsmodell von Russell (V, A)
• Basierend auf Erkennerframework (Matlab)
• Ziel: Extraktion von geeigneten Merkmalen aus dem EEG zur zuverlässigen Klassifikation von Emotionen
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
17/26
Robot Programming by Brain Activity
• Normalerweise werden BCIs für einfache vorgestellte Bewegungen verwendet (z.B. Cursorsteuerung)
• Ziel: Analyse von komplexen Bewegungen mittels Brain Computer Inferfacing Techniken
• Können komplexe, real ausgeführte Bewegungen erkannt und modelliert werden?
• Wie kann man solche BCI Informationen für das Programmieren durch Vormachen nutzen?
• SFB588 – Humanoide Roboter
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
18/26
ECoG based Speech Recognition
• Spracherkennung aus invasiv gemessener Gehirnaktivität
• Datenanalyse von ECoG Daten
• Bleeding edge research topic
• Kenntnisse in Mustererkennung, Datenanalyse, und Matlab sind von Vorteil
• Projekt mit a*star Singapur und Wadsworth Center Albany
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
19/26
Diplom- oder Masterarbeit
Aktuell: Diplom- oder Masterarbeit in der EMG-basierten Spracherkennung!
Ihr wisst alles über Akustik, GMMs, HMMs, …?
Wie wär‘s mal mit etwas Neuem? Am CSL erforschen wir seit Jahren sehr erfolgreich die Erkennung von Sprache mit elektromyographischen Signalen, die Muskelaktivität abbilden.
• Unser aktuelles Ziel: Bessere Erkennung von paralleler Aktivität
• Gut geeignet für DA oder MA
• Vorkenntnisse: Grundlagen der Programmierung, Verständnis für Algorithmik
• Selbständiges Arbeiten im Team mit guter Betreuung, eigene Aufnahmen erwünscht
• Bitte melden bei: Michael Wand ([email protected])
Co
gnit
ive
Mo
del
ing:
Mu
ltit
aski
ng
& W
ork
load
20/26
Studien/Diplom/Bachelor/Masterarbeiten
Ein weiteres heißes Thema in der EMG-basierten Sprachverarbeitung!
• Experimente mit Elektrodenarrays • Elektrodenarrays/Multielektroden geben eine sehr schöne
Repräsentation des Muskelsignals!
• Viele interessante Experimente möglich!
• Spracherkennung + Sprachsynthese, neue Themen ab ca. September
• Bei Interesse bitte melden bei [email protected] oder [email protected]
