North Carolina S&H PDF · 2016-08-15 · Sample ALP Graphics - Animated Grids and Scenes Protest &...
Transcript of North Carolina S&H PDF · 2016-08-15 · Sample ALP Graphics - Animated Grids and Scenes Protest &...
4/7/16
1
Visual Immersion System™: Communica5on Enhancement for
Au5sm Spectrum Disorders Howard C. Shane
North Carolina Speech & Hearing Associa8on April 27, 2016
Harvard Medical School Boston Children’s Hospital
Visual Immersion System™: Communica5on Enhancement for Au5sm Spectrum Disorders
• This short course will overview a communica?on strategy for children with a moderate to severe au?sm spectrum disorder known as the Visual Immersion System™. This uses visual supports and state of the art technology and takes advantage of the strong visual processing skills and interest in electronic media of children on the au?sm spectrum.
Released 2008 Available on Amazon as of October, 2014
PREFACE
Purposes
1. Overview the principles and Framework of the Visual Immersion System
2. Differen8ate between grids, visual scene and scene cue displays
3. Describe the ra8onale and methodology for combining visual graphics into the building blocks of language.”
4/7/16
2
Purposes
Overview the principles and framework of the visual immersion system
‘Visual Immersion System’ (VIS) • A visual instruction system for teaching language
concepts that tend to be difficult for people with moderate to severe autism (e.g., verbs, prepositions and attributes)
• VIS is a closed visual language, limited to the essential
vocabulary and syntax needed to support the comprehension and expression of practical, everyday communication exchanges
• Training includes instruction in the virtual environment (video clips) and tabletop environment (photographs, toy figurines and miniature objects), then extends to the natural environment to enable functional communication at home, school and community
Key Ingredients of VIS • Visual language for both language comprehension and
expression
• Both mentors and learners use the same visual symbols to communicate with one another
• Targets language comprehension: viewed as foundation for expression
• Advanced computer and video technologies attract and maintain children’s attention, provide compelling multimedia language instruction, and enable the use of dynamic visual symbols
• Immersion is Immersion – Sign Language Analogy
• Immerse for VIS or other options
Three Modes of Visual Support Addressed in VIS
I. Visual Instructional Mode (VIM): Visual cues used for the purpose of comprehension, which are imposed as an alternative to, or in conjunction with, speech.
II. Visual Expressive Mode (VEM): Visual cues used for the purpose of expressive communication.
III. Visual Organizational Mode (VOM): Visual cues used to represent the organization of an activity, routine, script, or schedule.
Overview of Visual Immersion Program (VIS)
Seven Communicative Operations
4/7/16
3
Communicative Functions Communication instruction for moderate to
severe ASD:
1. Protesting and Refusal 2. Organization and Transitions (1 & 2 language that supports organization and control) 3. Requests 4. Directives 5. Comments 6. Questions 7. Social Pragmatics 3 - 7 language for everyday functions
Communicative Functions
Instruction of the seven functions should not be viewed as hierarchical – not a serial process
Today will cherry pick* from each of the functions
* Cherry Pick
Selec8vely choose (the most beneficial items) from what is available.
cherry-‐pick something Fig. to choose something very carefully. (As if one were closely examining cherries on the tree, looking for the best.)
Cherry Pick Selec8vely choose (the most beneficial items) from what is available.
Communicative Functions Communicative Functions do not include abstract language
(…with liberty and justice for all), passive voice (The book was
read by the boy), complex syntactic structures (If he hadn’t
checked the weather in the morning, then he would have
forgotten to bring his umbrella), figurative language (She flew
to the bookstore) or humor (Why did the chicken cross the
road?).
Purpose
Differen8ate between grids, visual scene and scene cue displays
So we’re all on the same page
4/7/16
4
Display Formats
Grid Display – A grate-‐like screen arrangement containing a target areas into which symbols or text are placed. Targets serve as either a vocabulary end point or lead to rela8onal targets (symbols) on other screens.
Grid based Display
Grid based Topic Display – Direc8ve or Comment -‐ singing
Display Formats Visual Scene Display -‐ Visual scene displays (VSDs) portray events,
people, ac8ons, objects and ac8vi8es against the backgrounds within
which they occur or exist. These scenes are used as an interface to
language and communica8on. A VSD may represent:
• a generic context (e.g., a drawing of a house with a yard, an
office with workers or a school room with a teacher and students.)
• a personalized context (e.g., a digital photo of a child playing in
his bedroom or a picture of the family on a beach while on
vaca8on.)
ACN NEWS, August 2004
Recogni8on of de-‐constructed Scene
Visual Scene Deconstructed Elements
Research / Literature Background
Summary of Drager et al. (2003) • Purpose: To compare children’s ability to locate vocabulary on grid vs. visual scene display
• Par8cipants: 2 ½ year old children, n = 30 • Results: Children did significantly befer with the VSD than the grid display
4/7/16
5
Scene Based Display
Companion (Circa 1995) – Commercialized as screen format op8on on the Freestyle
Companion Home Screen
Companion Sub-‐Level Screen Companion Sub-‐Level Screen
Display Formats
Visual Scene Cue -‐ symbols that represent an en5re event/ac5vity. They contain the relevant background context, agents, objects, ac8ons and the interac8ons of the agents and objects. Visual scenes are either: • Dynamic scenes –full-‐mo8on video clips that illustrate how the ac8vity/event unfolds over 8me, or
• Sta8c scenes –digital photographs that capture a single prototypical moment in an ac8vity/event.
Visual Immersion System: A Guide to Communica5on Enhancement for Au5sm Spectrum Disorders)
Display Formats
1. Rela8vely recent display format 2. Expanded due to technological advancements 3. Are recognized earlier than grid displays at earlier
developmental age 4. Seemingly more intui8ve -‐ -‐ when compared to de-‐
constructed scene 5. Scenes phrase or concept based ; grids element based 6. Scenes – circumvent Language
Visual Scene Displays and Scene Cues
4/7/16
6
Research / Literature Background
Summary of Schlosser, R. W., Laubscher, E., Sorce, J., Koul, R., Flynn, S., Hotz, L., Abramson, J., Fadie, H., & Shane, H. (2013) • Purpose: Assess effect of visual scenes and scene cues on comprehension
• Par8cipants: 10 subjects with ASD • Results: Significant improvement in comprehension when scene cue provided
Research / Literature Background
Summary of Remner, R, Baker, M., Karter, K., Kearns, K., & Shane, H. (In Press) Use Of Augmented Input To Improve Understanding Of Spoken Direc5ves By Children With Moderate To Severe Au5sm Spectrum Disorder • Purpose: Assess effect of visual scenes and scene cues on
comprehension & replicate Schlosser et al, 2013 • Par8cipants: 10 subjects with ASD • Results: Significant improvement in comprehension
when scene cue provided
Research / Literature Background
• Limited research founda8on for use of visual scene, scene cue or grid displays despite considerable popularity as founda8on for AAC.
• The limited research seems to favor VSDs with respect to vocabulary iden8fica8on.
• When compared to speech direc8ves, VSCs improve understanding the intent of preposi8onal direc8ves
Developmental Trajectory
Clinical observa8on (and limited research) suggests a developmental con8nuum that extends from Noun Elements, Scene Cues and Visual Scene Display comprehension during early developmental periods to Grid Display made up of elements (and Text) at later stages
Clinical / Developmental Trajectory (Profile of Poten8al Consumers of Visual Graphic Language)
_____________________________________________________
Six Consumer Groups
Pre-‐sceneà Visual Scene àTransi8on (to Grid) àGrid àTransi8on (to Text) àText
1 2 3 4 5 6
Display Display Display
4/7/16
7
Developmental Trajectory
Vast majority of individuals with moderate to severe ASD primarily employ the communica8ve func8on of reques8ng, likely because: 1. Not mo8vated to share 2. Difficulty with joint afen8on 3. Inability to combine the visual elements of language –when do
combine, based on scripted sentences
Clinical Decision Making to Enhance Visual Communication
An Evolving Clinical Exercise
Grid and Scene Displays
Seven Operations* • Protest & Refusal • Organization & Transition • Requesting • Commenting • Directives • Questions • Social Pragmatics
Based on: Enhancing Communication in Individuals with Autism: A Guide to the Visual Immersion System
Grid and Scene Displays
Function 1: Protest & Refusal – Definition -‐Protest: Behavior that expresses objec8on or disapproval of an ac8vity, event, or person. -‐ Refusal: Behavior that expresses rejec8on of an object, ac8vity, or event suggested or ini8ated by another person (Shane, O’Brien & Sorce, 2009).
Means:
• Behavioral / physical • Conventional gesture • Symbol
Grid and Scene Displays
Protest & Refusal – Behavioral / physical: can be both dangerous and destructive Goal to move from behavioral to symbolic expression
Likely Causes of Protestation
• Transitions • Demands or directives to perform a task or activity • Removal or cessation of an enjoyable task or activity • Expecting participation in a distasteful task or activity when introduced • Lack of ability to request goods and services • Desire to escape • Desire for attention
4/7/16
8
Nature of Visual Supports
• Exert control over Environment • Language of Control
• Stop • No • More • Finished • Help
Sample ALP Graphics - Animated
Grids and Scenes
Protest & Refusal
⏎ Visual Scene Display
Grid Display
à
à
Protest & Refusal – Protest Bar Imbedded into Scene
One possible difference and likely advantage of Scene Display is placement of a “protest strip” within the seqng in which a symbolic communica8ve replacement might be needed
Function 2: Organization & Transitions
Definitions: Organization - arranging elements (e.g., tasks, events) into an orderly, functional, structured whole. Transition - the process of changing from one state, stage, or activity/environment to another.
• Traditionally considered a VOM
• Not a speech act per se
Assessment of Organiza8on and Transi8ons
Part 1: Transi8ons Ques8on 1: Does the learner have difficulty transi8oning between ac8vi8es? Ques8on 2: Does the learner have difficulty with novel environments? Ques8on 3: Does the learner have difficulty with changes in rou8ne? Ques8on 4: Does the individual exhibit aggression, non-‐compliance, or passivity during /in Transi8ons?
Assessment of Organiza8on and Transi8ons
Part 2: Organiza8on Ques8on 1: Does the learner have difficulty sustaining par8cipa8on in learning tasks? Ques8on 2: Does the learner have difficulty understanding sequences? Ques8on 3: Does the learner have difficulty with appropriate expecta8ons? Ques8on 4: Does the individual exhibit aggression, non-‐compliance, or passivity during learning situa8ons?
4/7/16
9
Aligning With Characteris5cs of Au5sm
Behavior Control & Aggression Problems
Method: First – Then Displays: Countdown Boards: Social
Stories: Timer
Function 2: Organization & Transitions
• First/Then Displays
• Countdown Boards
• Timers
• Visual Schedules (daily, calendar, and activity schedules)
School Work
Then
Playground
First
10 minutes
First-Then Low Tech - Expanded • Built in duration & trial information – static scene cues
10 Trials
Math
Next Then
Basketball Videos
10 Trials
First
5 Baskets 15 minutes
First-Then – Low Tech - Expanded • Built in multiple steps
Traditional Daily Schedule – Low Tech
Traditional LayoutØ Typically refers to Where or Location
Ø Can refer to “who” as marker
Function 2: Organization & Transitions
Daily Schedule - Expanded
Event: The theme of the event Who: The participants in the task
Where: The location where the task will be performed What: Activities that will be performed
How Long/Many: Number of repetitions required or duration of task
Reward: Reinforcement for participating in the activity, if applicable
4/7/16
10
Additional Innovations in Visual Organization
• If I -Then Display
• Preparation for Uncomfortable Circumstances
Organiza8on & Transi8on Final Thoughts
• Be certain the level of representa8on is understood -‐-‐ Content of grid can be a who, what or where
• Grids oten use scenes as content – scene enhances comprehension
• Most schedules, calendars, first then displays, etc. are grids, yet content is oten scene based
Grid and Scene Displays
Requesting
Expressing a desire for objects (e.g., toys, food), ac5vi5es, people, affec5on, aSen5on, recurrence, assistance, and informa5on/clarifica5on
Requesting: Goal • An important goal of AAC / reques8ng is to provide a symbolic rather than physical means of expression
Physical Requests vs Symbolic Requests
-‐ Limited -‐ Easily understood
-‐ Needs instruc8on
Cookie
Func8on 3: Thoughts about Reques8ng • Communication should be easy • Don’t reduce spontaneity with artificial syntax • Motivation • Level of Representation • Mands work • Practice across environments and partners • Stilted requesting speech is hard to break
• What is the real benefit of: Name + Want + Item + Please
Func8on 3: Thoughts about Reques8ng Overwhelming Clinical Focus on the Requesting Communicative Function Reasons? Thoughts?
4/7/16
11
Display Formats that Enable Reques8ng
• Noun element reques8ng displays • Visual scene displays • Linked visual scene displays • Hybrid visual scene displays • Grids • Text
Picture Exchange Communica8on System (PECS)
Classic Example of Noun Element Reques8ng
PECS (Likely Most Popular Grid Display)
Behaviorists approach to communica8on Enhancement Ubiquitous
– Why à it works – Why à easy to understand steps
But
– is it a Glass Ceiling? – It is the an8thesis of teaching genera8ve language – Really only about reques8ng ( I + want + __ + please)
Strengths of PECS • Demonstrable success - - program comprehensible •
• Mand is teachable
• Early focus on requesting -- an achievable skill • ‘Tear and put’ selection
– Primary difference from past AAC requesting approaches (although not necessary for all children).
– Establishes relationship between learner and mentor and provides a visual model of syntactic construction.
– Electronic version of ‘tear and put’ is symbol selection to message window.
• Children learn to use symbols to communicate – learn that others can understand them – learn that they can specify their reinforcement
• Reduces early behavioral problems caused by decreased ability to request goods and services
Visual Scene Display Reques8ng
Use familiar, natural background setting
Goldfish in a lunchbox Goldfish on a shelf
Linked Visual Scene Displays
• Launches learner from one scene to another • Supports independence, naviga8on skills
4/7/16
12
Grid Displays
• For combining symbols to create sentences • Message window • Expanded naviga8onal demands
Reques8ng Summary
• Many display types • Reques8ng should be highly mo8va8ng • Reques8ng is a good founda8on for acquisi8on of other func8onal opera8ons
• Move the learner along the con8nuum of visual supports when possible
Four Types of Directives
1. Control – regulates behavior “stop” “stand up”
2. Rou8ne – par8cipa8on in daily rou8nes “open” “put” “get”
3. Instruc8onal – needed for educa8onal ac8vi8es “circle” “point” “show”
4. Play-‐based – enable par8cipa8on in games, play “roll” “throw” “pop”
Function 4: Directives Direc8ves: Ra8onale for Instruc8on
Recep*ve: • Direc?ves make up a large por?on of the spoken language received by learners
• Difficult due to distrac?bility, mo?va?on, non compliance, poor comprehension
• Breakdowns result in frustra?on and weariness for caregivers
Direc8ves: Ra8onale for Instruc8on Expressive: • Learners with ASD usually give few direc8ves
– lack variety of verbs – have reduced syntax – limited vocabulary
• Poor expression of direc8ves can lead to communica8on breakdowns and helplessness
• * Language Comprehension is Essen8al
Teaching Directives: Understand and use informa8on from Dynamic Scene Cues (videos), Sta8c Scene Cues (photographs), and Isolated Elements (Noun, verb, etc) to:
– Improve understanding of noun labels, verbs – Teach single-‐step direc8ons
4/7/16
13
Teaching Directives (Noun & Verb Elements)
Early stage show noun and verb elements. Model expected reaction. e.g., KICK
Verb (Animated) element: anima8on showing a familiar Ac8on (verb)
Direc8ves: Verb Elements
Teaching Direc8ves: Verb Element Purposes
Describe the ra8onale and methodology for combining visual graphics into the building blocks of language
Syntax
Rule based order / organiza8on of words in a spoken sentence Symbol Syntax refers to construc8on of a sentences using the elements of language in a visual form
Why is Syntax Important?
1. For persons with moderate to severe au8sm the use of genera8ve language, where symbols are combined to form novel sentences, is rare
2. I + “desired item” is a scripted sentence and not novel, genera8ve language
4/7/16
14
Prizant (1983)
“The reproduction of memorized multiword units would seem to be devoid of the creative and generative linguistic processes typically associated with the spontaneous production of multiword utterances.” (p. 303)
79
Reality of Expressive Language in Moderate to Severe Au8sm
• Spoken Output Predominantly Scripted Phrases • Reques8ng Predominant Communica8ve Func8on • Social Interac8ons Frequently Require Promp8ng • Instruc8on oten Focused on Expression • PECS Predominant Instruc8onal Strategy
Clinical Reality of Language Comprehension in Moderate to Severe Autism
• Greater Noun Comprehension (Spoken And Visual)
• Limited Understanding (Verbs, Prepositions, Adjectives, Question Forms)
• Spoken Language Within Routines Often Understood
• Context Facilitates Understanding
AUGMENTED INPUT
AUGMENTED INPUT DEFINED
The Introduc5on of a Visual En5ty (Graphic or Object), Typically in Conjunc5on with Speech, that is Intended to Improve Understanding. (Allen & Shane, 2014) The Synchronized Presenta5on of Graphic Symbols or Objects Cues while Speaking with the inten5on of improving understanding and / or expression
Improving Comprehension
APPROACHES DESCRIBED IN THE LITERATURE
Augmented Communica8on Input • First use of term Augmented Communica5on Input • Purpose to improve the comprehension of spoken language • Augmented communica?on input “….communica?on partners]
wri?ng or typing key elements of their messages onto some sort of system that provides a visual display to the individual” (Beukelman & Garrea, 1988, p. 119)
• Used with aphasia adults
4/7/16
15
Improving Comprehension APPROACHES DESCRIBED IN THE LITERATURE Aided Language S8mula8on (ALgS) • Aided language s*mula*on (ALgS) (Goossens, 1989) • Simple, short phrases, a slowed rate of speech, expand on language produced by the learner, and emphasize keywords.
• “When providing aided language s?mula?on, the facilitator points out picture symbols on the child’s communica?on display in conjunc?on with all ongoing language s?mula?on” (Goossens’, 1989, p. 16)
• Used with children with Cerebral Palsy
“Aided Language S8mula8on”
Does simply poin8ng to symbols while speaking improve understanding?
“Aided Language S8mula8on”
Would A Message / Accumulator Window Improve Understanding?
Message window Message Window
Improving Comprehension
APPROACHES DESCRIBED IN THE LITERATURE Augmented Input (with visual scene cues) Scene cues Defined
Visual Language Instructional Phases:
• Learner progresses through all/some of three phases of visual language symbols, from concrete to abstract representations:
Dynamic Scene Cues: full-motion video clips of action
scenes
Animated Verb Graphics - Scene
4/7/16
16
Visual Language Instructional Phases
• Learner progresses through all/some of three phases of visual language symbols, from concrete to abstract representations:
1. Dynamic Scene Cues: full-motion video clips of action scenes
2. Static Scene Cues: photographs that capture a prototypical moment in the action scene
Func8onal Scene Cue -‐ Play
Toy cars
Put man in car Drive car up/down ramp Make the car go fast/slow Build a track (add a piece)
Ac8vity Possible Scene Cue
Func8onal Scene Cue -‐ ADL
Bed8me
Socks in the hamper Put on pajamas Toothpaste on toothbrush Brush teeth Climb in bed
Ac8vity Possible Scene Cue
Role of Scene Cues
Enables Func8onal / Prac8cal Opportuni8es Serves as Founda8on for Symbol Syntax Learning
Implications of Scene Cue Mastery Use of dynamic and static cues bypasses language
processor – Due to load on language processor when using elements, some
children may use static scenes as their communication system
Mastery of static scene cues is a significant accomplishment – Can be used to promote general understanding and communication
related to: • Daily Living Skills • Play • Transitions • Requesting • Commenting • Clarifying • Directives
Visual Scene Cues: Limitations
Difficult to flexibly apply visual scene cues to spontaneous situations and messages
– Highly context-dependent – Not a cohesive, rule-bound language system
4/7/16
17
Learners Needs
• Be aware that linguistic elements can be combined to make unique meanings
• Understand rules for combining these disparate linguistic elements into meaningful phrases
Visual Language Instructional Phases
• Learner progresses through all/some of three phases of visual language symbols, from concrete to abstract representations:
1. Dynamic Scene Cues: full-motion video clips of action scenes
2. Static Scene Cues: photographs that capture a prototypical moment in the action scene
3. Language Element Cues: graphic icons representing each of the individual linguistic components that comprise an action scene (e.g., subject, object, verb, preposition, adjective, etc.)
In Search of a Visual Language
Symbolate: Language Comprehension?
• Stringing symbols together does not automatically
result in comprehension • Learners must first have a knowledge of language
elements and semantic relationships • Paradoxically, stringing symbols together may
actually interfere with comprehension
What’s Known About AUGMENTED INPUT
• Literature offers some evidence that comprehension and expression can be improved through AUGMENTED INPUT
• Clinical observa8on suggests that exposure to symbol sequencing during an immersive experience can lead to improved comprehension and expression (Op8mal approach/es and candidacy remains unstudied)
What’s Known About AUGMENTED INPUT (Based on experiences in Au8sm Language Program at
Boston Children’s Hospital and Monarch School for Children with Au8sm )
• Considerable clinical outcomes from exposure to Mixed Displays leads to improved understanding of symbol syntax
• Pilot research and clinical outcomes from exposure to Teaching Language Concepts leads to improved understanding of symbol syntax
• Considerable clinical outcomes from exposure to Topic Displays leads to improved understanding of symbol syntax
4/7/16
18
Empirical Ques8on
Can technology facilitate the acquisi8on of symbol syntax (the linking of element cues /sequencing of graphic symbols)?
Teaching Language Concepts
Teaching Language Concepts – Sotware Applica8on Licensed
to SpecialNeedsWare
Overview and Use of Topic Displays
Tool: Topic Display
• Language elements organized for one topic • Contains various parts of speech • Color coded
Topic Display
4/7/16
19
Tool: Topic Displays
Use a mo8va8ng theme Including mul8ple pronouns, verbs, adjec8ves and/or objects Program words in order of sentence structure Begin with two contras8ng concepts
Overview and Use Mixed Displays
Apps To Teach Symbol Syntax Sentence Key
Sentence Match Puddingstone Adventure: Introduction
4/7/16
20
Other Language Approaches
Suskind – Language Acquisi8on through Disney Video viewing Anima5on offers a medium of story telling and visual entertainment which can
bring pleasure and informa5on to people of all ages everywhere in the world
Walt Disney
“She’d do it if Winnie the Pooh told her to do it”
Quote from Mother from ALP Clinic of child with Au5sm
Suskind’s Impact
• Raised Awareness of Disney Anima8ons • Anima8on has a place in language instruc8on • Outcome based on anecdote not Behavioral Science • Poten8al for Nega8ve Consequences – Obviously an Unintended Consequence
Final Thoughts • Direc8ves can be given and received with Visual Scene Displays at earlier age
• Unscripted direc8ves (using genera8ve language) requires knowledge of symbol syntax
• Technology assists in the acquisi8on of symbol syntax – Developmental Trajectory (scene cues to element cues)
4/7/16
21
Function 5: Commenting Commen8ng Defini8on:
A behavior, gesture, vocaliza8on, or other modality that conveys informa8on about observa8ons, experiences, knowledge, opinions, emo8ons, and reac8ons
• Connect shared experiences • Give informa8on, thoughts, feelings • External and observed vs. internal • Essen8al to conversa8on and social exchanges • External and observed vs. internal • Essen8al to conversa8on and social exchanges
Commen8ng: Ra8onale
Commen8ng is challenging because: • Learners may have low mo8va8on for social exchange • Limited vocabulary and concept knowledge
– Hot, cold, shiny, loud, big, preSy, ugly • Restricted interests • Difficulty with reciprocity
Commen8ng: General Goals of Instruc8on
• Build social mo8va8on, connectedness • Expand language
– Vocabulary – Syntax – Pragma8cs – Conversa8on
• Promote genera8ve language
Emerging Commenters
Different Levels of Commen8ng Competence
Emergent Commenters: Goals
• Label -‐ items in familiar environments • Build apprecia8on for joint afen8on and informa8on sharing
• Ini8ate symbolic commen8ng
4/7/16
22
Labeling
The Training Wheels of Comments
A Label Is a Generally a Noun
Power of Learning a Label
Ability to Label Enables “spoon”
Requesting --> ‘Want spoon’ Commenting --> ‘Silver spoon’ Directives --> ‘Get the spoon’ Wh questions --> ‘Where is the spoon?’
Tools for Emergent Commenters
• Visual Scene Displays with voice hotspot
• Animated graphics – Especially Descriptors & Verbs
Commen8ng Goals: Intermediate Communicator
Build concept knowledge Expand vocabulary, syntax, linguis8c knowledge
Emergent Communicators: Tools
Mixed display
Emergent Commenters: Tools
Teaching combina8on of symbols that describe – “Red truck”, “funny boy”
4/7/16
23
Advanced Communicators: Goals
Syntax, Syntax, Syntax Increase genera8ve language and decrease scripted phrases
Pain & Illness
Other Application Suggestions
Preferred video routine
Window routine
Items in unexpected locations
I hear, see, feel (PECS)
Function 6: Questions Ques8ons Defined: Sentences or phrases that are spoken, wrifen, gestured or otherwise expressed as an interroga8ve and directed to someone in order to receive informa8on in reply. Ques8ons have both a recep8ve and an expressive component (Shane, O’Brien, & Sorce, 2009).
Thoughts about Ques8ons
Valuable means of gaining informa8on and socially engaging with others Lay founda8on for conversa8onal exchange Paradox -‐ those with reduced language capabili8es are frequently bombarded with ques8ons – perhaps to compensate for reduced capacity with other forms of expression Context and key words can occasionally serve as a means to answer some ques8ons – not always prac8cal Yes – No – 50% right = chance
Thoughts about Ques8ons Usually begin with specific ques8on markers “who,” “what,” “where,” “how” and so on Elicit informa8on about
• Time (“when” ques8ons) • Place (“where” ques8ons) • people / par8cipants (“who” ques8ons) • Choice (“which” ques8ons) • Uncertainty (“how” and “why” ques8ons)
Can serve as way to compensate for inability to ini8ate requests. “What do you want?” or “Do you want this?” or “Where should we go?”
4/7/16
24
Ques8on Types Addressed in VIS
• Yes/no ques8ons pertaining to labels (e.g., “Is this a train?”) • Yes/no ques8ons pertaining to preferences (e.g., “Do you want some raisins?”)
• Which ques8ons pertaining to choice (e.g., “Which one do you
want?”)
• Who ques8ons
• What + do ques8ons (e.g., “What is he doing?”)
• Where ques8ons
• Certain aspects of when, how and why ques8ons
Ques8ons: Developmental Perspec8ve Typical age of acquisition (comprehension) for question forms
targeted in the VIP (after Chapman, 1981).
Question forms addressed in
the VIS
Typical age of acquisition (year;
month)
1. Yes/no 2:0
2. What + be (What is that?) 2:0
3. What + do (What is he
doing?)
2:6
4. Where (place) 2:6
5. Who 2:6
6. Whose 2:6
7. Why 3:0
8. How 3:6
9. Which 5:6
10. When (temporal) 5:6
Ques8on Graphics
Graphic symbols representing the question words who, what and where as portrayed by Mayer-Johnson Company, PixWriter, (SymbolStix) and the Autism Language Program (ALP) graphics
Text Mayer-Johnson PixWriter SymbolStix ALP Graphics
Who?
What?
What-be
What-do
Where?
Ques8on Graphics
Examples graphic images for Yes and No from three commercial developers of graphic symbols.
ALP Animated Graphics PCS: Mayer-Johnson SymbolStix (xxx)
No
Yes
Ques8on Graphics
Paucity of research to suggest if one graphic or an en8re graphic
set is superior to another or if any one set can actually improve
or facilitate the understanding of ques8ons
Learning Approaches to Ques8ons
Buildable Scenes
4/7/16
25
Instruction for Who, What , Where (non-literate learners)
Instruction for Who, What , Where (literate learners)
Example: Color-coding WH Questions
Which (Question response or behavioral selection from choice set ) Calendar for Teaching ‘When’
Calendar for Teaching ‘When’ Anima8on to Teach Ques8ons
Elusive Language Concepts : (Wh Questions)
Method: Puddingstone Ac5on Factory App
4/7/16
26
Apps to Teach Ques8ons Aligning With Characteris*cs of Au*sm
Elusive Language Concepts : (Wh Questions)
Languagelearning Apps, LLC $24.99
Aligning With Characteris*cs of Au*sm Elusive Language Concepts : (Wh Questions)
SNAKEY
Function 7: Social Pragmatics
Social Pragma8cs: a qualita8ve func8on of communica8on that
reflects an individual’s ability to ini8ate, sustain, terminate,
par8cipate and direct interac8ons with others via verbal or non-‐
verbal communica8ve means and in socially appropriate ways.
Shane, et al Visual Immersion System: A guide to Communica*on
Enhancement in Au*sm
Defini8on
Social Pragmatics
Social pragma8cs allows for the appropriate expression of the six communica8ve func8ons previously reviewed Address social pragma8cs within key contexts that are oten problema8c for severe end of ASD Address those pragma8c skills that most likely respond to systema8c use of visual supports
Social Pragmatics
Social pragma8cs in normal development are both learned and taught Observa8onal deficits and interac8ons by those with ASD frequently lead to un-‐or under-‐developed social pragma8c skill development Language & Cogni8ve limita8ons reduce pragma8c ability
4/7/16
27
Social Pragmatics
Social pragma8cs can also be affected by: • Mo8va8on • Joint afen8on • Cogni8ve ability • Language ability • Afen8onal deficits • Limited TOM (perspec8ve of others)
Newman – Social Pragma8c Language Acquisi8on through Interac8on with Siri
hap://mobile.ny?mes.com/2014/10/19/fashion/how-‐apples-‐siri-‐became-‐one-‐au?s?c-‐boys-‐bff.html?_r=1&referrer=
Personal Assistants
“……they will also be able to carry on more complex conversa5ons about a person’s area of interest. “Your son will be able to proac5vely get informa5on about whatever he’s interested in without asking for it, because the assistant will an5cipate what he likes,” said William Mark, vice president for informa5on and compu5ng sciences at SRI.”
Personal Assistants
Personalize as therapy assistants? Speech improvement Social Pragma8cs: • Store informa?on on friends, family and acquaintances to be recalled prior
to a visit or a phone call to aid in conversa?on development (Alexa, tell me what Aunt Joann’s hobbies are).
• Program to prac?ce basic social nice?es with Avitar Timer Func8on Other
4/7/16
28
Function 7: Social Pragmatics Common Social Exchanges affected by SP deficits:
• Engaging in simple gree8ngs, par8ngs and small talk (a new voca8onal site,
mee8ng new people)
• Sharing and taking turns (wai8ng in line, opening presents, wai8ng for food
to be served)
• Traveling to and tolera8ng new places (church, a mall, a party)
• Mee8ng new people (at a store, family gathering)
• Modifying and managing atypical behaviors according to the environment
(knowing what is appropriate, where)
Function 7: Social Pragmatics Three areas of Social Pragma8cs targeted in VIS: 1. Social language and conversa8on
2. Play skills 3. Challenging Behaviors
Assessment of SP
Part 1. Assessment of Conversa?on Skills 2. Does the learner ini?ate or respond to small talk (e.g., “How was
your weekend?” “What have you been doing?” “What did you do at school?”)
3. Does the learner engage in conversa?on around a shared topic (including responding to and asking ques?ons across mul?ple turns)?
4. Does the learner ini?ate conversa?on with a familiar partner regarding a topic relevant to the partner (e.g., recent past events, interests, family members)?
Assessment of SP
Part 2. Assessment of Play Skills 1. Does the learner engage in sensorimotor games with others (e.g., chasing, 8ckles)? Yes – no interven8on required No – refer to Interven8on sec8on 2A: Sensorimotor Play 2. Does the learner engage in pretend play (e.g., feeding a baby doll)? Yes – no interven8on required No – refer to Interven8on sec8on 2B: Pretend Play 3. Does the learner engage in rule-‐based games with others (e.g., board games)? Yes – no interven8on required No – refer to Interven8on sec8on 2C: Rule-‐based game play
Assessment of SP
Part 3. Assessment of Challenging Social Behaviors 1. Does the learner exhibit persevera8ve behaviors? 2. Does the learner engage in conversa8onal turn-‐taking? 3. Does the learner exhibit overuse of scripted phrases? 4. Does the learner exhibit inappropriate, obtrusive, or emo8onal responses (outbursts)? 5. Does the learner have difficulty understanding and honoring personal space and boundaries? 6. Does the learner exhibit an aberrant or distrac8ng voice or speech?
1. Social language and conversa8on
• Social gree8ngs
• Extended gree8ngs
• Engaging in Small Talk
• Conversa8on Around a Shared Topic
• Ini8a8ng Conversa8on Based upon Prior Knowledge
4/7/16
29
Social language and conversa8on
Tools
Social gree8ngs
• Low Tech -‐
• Hi Tech – Video Modeling
Social Pragmatics – Tools (Greetings)
Low Tech
(Echoic Greeter)
Social Pragmatics – Tools (Greetings)
Hi Tech
Video Modeling (can be self)
Social Pragmatics – Tools (Expanded Greetings)
Social Story
“When I see someone I know I say “hi,” look at them, and wave/shake hands. If they say “How are You?” I can say “Good/fine.” I can say “how are you?” or I can give them a compliment.”
Social Pragmatics – Tools (Expanded Greetings)
Turn taking visual
Social Pragmatics – Tools (Expanded Greetings)
Ini8a8ng and responding to compliments
4/7/16
30
Social Pragmatics – Tools (Small Talk)
Scripted
Last week I….. Ac5vity With (who picture) At (where picture)
Social Pragmatics – Tools (Small Talk)
Ini8a8ng
Function 7: Social Pragmatics Three areas of Social Pragma8cs targeted in VIS: 2. Play skills
• Exploratory
• Rou8ne based
• Rule Governed
• Pretend
Function 7: Social Pragmatics Three areas of Social Pragma8cs targeted in VIS: 3. Challenging Behaviors
Topic Shifting - Preservative Topics
Waiting: Low Tech
4/7/16
31
Waiting: Hi-Tech
Observational Learning - Virtual Character/s 1. Demonstrate Appropriate Behavior 2. Virtually Countdown
Next Genera8on Apps & Tools for Learning & Communica8on
Trends in Applied Technology
Wearable Technology
Scene Cues & Wearable Technology
Primary Aim To determine the effect of “Just-‐In-‐Time” (JIT) visual supports delivered through wearable technology (e.g., the Apple Watch) on language and behavior of individuals with Au8sm Spectrum Disorder (ASD)
Principal Investigator : Amanda O’Brien, MS
Just-‐in-‐Time Supports in AAC
Based on: – Schlosser, R. W., Shane, H. C., Allen, A., Abramson, J., Laubscher, E., & Dimery, K. (2015). Just-‐in-‐?me supports in augmenta?ve and alterna?ve communica?on. Journal of Physical and Developmental Disabili*es. Advance online publica?on.
– DOI. 10.1007/s10882-‐015-‐9452-‐2
4/7/16
32
Just-‐in-‐Time
• Origin: – Japanese business strategy to streamline inventory – Signals, or “Kanban,” inform produc?on line exactly when next part is needed: i.e., "Just-‐in-‐Time”
– Informa?ve Video
• Adapta?on to other fields: – Medicine – JIT text messages – Just-‐in-‐Time Teaching (JITT)
Applica8on to AAC
• Currently: – Research with SLI and word produc?on shows promise of JIT for treatment purposes
– Technological advances support current use of JIT in AAC • Examples:
– Situa?on-‐specific vocabulary by GPS – Shane’s applica?ons:
• symbolTalk • M-‐STARR
– Scene-‐Cues – Light et al. (2012) ASHA paper – Drager et al. (2014) ISAAC paper
Proposal
• Proposal: – Possibility of providing supports that tradi?onally are not provided on an “as needed” basis
• JIT can: – Deliver prompts, reminders, rewards, encouragement, memory aids and error correc?on from a distance
– Improve independence across serngs – Provide more efficient storage, retrieval, and delivery compared to low tech materials
– Allow for mul?ple formats of delivery that can be tailored to an individual’s processing strengths
Organiza?onal Framework
Classifications
Intended Purpose Modalities Source Delivery
Methods
Framework -‐ Intended Purposes • Prompts:
– Used to bring about an ac?on by the recipient • Reminders:
– Given at exact moment when ac?ons should occur – Repeated behaviors will establish a new rou?ne
• Rewards: – Allows for reinforcement is associated with desired behavior
• Encouragement: – Show instantaneous support of successful behavior
• Conversa?on: – Aid in star?ng, maintaining, or ending a conversa?on – May also assist in repairing communica?on breakdowns
• Vocabulary: – Selec?on, instruc?on, organiza?on, and generaliza?on.
• Generaliza?on: – Assist in transfer of skills to natural environment
• Organiza?on: – Clarify organiza?on for confused learners
Framework -‐ Modali8es
• Auditory Modality: – “Earcons” – Spoken language or non-‐linguis?c environmental sounds – Pr-‐recorded or created at ?me of delivery
• Visual Modality: – “Eyecons” – Sta?c/animated graphic symbols, tradi?onal orthography, photos, sta?c/dynamic scene cues, video, and flashing screens/indicators
• Vibrotac?le Modality: – “Vibrons” – Wearable, pulsa?ng mobile technology
4/7/16
33
Framework -‐ Sources
• Self-‐Ini?ated: – The learner becomes aware of a need for support
• Automated: – Supplied when needed without human assistance – Guided by preprogrammed ?mes, environment detec?on, or partner speech analysis
• Mentor-‐Generated: – Supplied by a family member, peer, or support personnel
Framework -‐ Methods of Delivery
• Face-‐to-‐Face: – Mentor-‐Generated JITs
• Wireless Transmission: – Mentor-‐Generated JITs
• Preprogrammed: – Automated JITs
• Teleprac?ce: – Mentor-‐Generated JITs
• Direct: – Self-‐ini?ated JITs
Benefits of JIT
• Decrease demands on Working Memory: – Dynamic displays place addi?onal demands on working memory.
– JIT as a memory aide or to introduce vocabulary/symbols in context
– Less burden on naviga?on = more focus on understanding symbols or ongoing conversa?on
Benefits of JIT
• Situated Cogni?on & Ac?on – Both JIT and Language in general are situa?on specific
– Face-‐to-‐face conversa?ons depends on context – JIT can further clarify the rela?onships between symbols and objects in a par?cular situa?on
– Thus JIT cues of situa?onal features work in conjunc?on with speech to aid in language comprehension
Benefits of JIT
• Teachable Moments: – Opportuni?es for a child to learn when they are excited, engaged, and primed for learning
– Requires ac?ve role of teacher – JIT supports may enhance likelihood of learning due to implica?ons for communica?on partners
Scene Cues & Wearable Technology
Primary Aim To determine the effect of “Just-‐In-‐Time” (JIT) visual supports delivered through wearable technology (e.g., the Apple Watch) on language and behavior of individuals with Au8sm Spectrum Disorder (ASD)
Principal Investigator : Amanda O’Brien, MS
4/7/16
34
Scene Cues & Wearable Technology
Secondary Aims 1. Assess opera5onal competencies
• Percep8on of images on screen. • Tolerance for wearing • Raise arm to view • Tap screen view dynamic cue
2. To explore mul5modality of JIT delivery and comprehension.
• Visual only (e.g., sta8c scene cue, dynamic scene cue). • Auditory cue presented with dynamic scene cues. • Afen8on to, and comprehension of, hap8c prompts.
Scene Cues & Amazon Echo Technology
Primary Aim To determine the effect of “Just-‐In-‐Time” (JIT) visual supports delivered by the Amazon Echo Technology (e.g., on language and behavior of Individuals with an Au8sm Spectrum Disorder (ASD)
Trends in Applied Technology
Mobile Device Interac8vity
Symboltalk
202
SymbolTalk
Unique features – “next genera8on” App 1. Voice driven Speech to Graphic Transla8on 2. Two-‐way transmission between devices 3. Intelligent Q & A tool -‐ AKA Smart Ask
203
SymbolTalk – iPad App
Finn pushes the shopping cart.
Mentor’s device
I understand
!
Learner’s device
Transmit
Next Genera8on Communica8on Apps
DEMO
4/7/16
35
Evolu8on of the Synthesized Computer Voice
Robo8c à Intelligible à Natural à Personalized