Voice Activated Presentation Assistant

V O I C E A C T I V A T E D P R E S E N T A T I O N

A S S I S T A N T

Brian Chen Eunsang Cho

Vahe Jabagchourian Vatche Jabagchourian

EECS 129B Senior Design Project

Professor Raymond Klefstad Winter 2007

.

U N I V E R S I T Y O F C A L I F O R N I A I R V I N E

EECS 129B Voice Activated Presentation Assistant

Voice Activated Presentation Assistant (VAPA)

Executive Summary

The Voice Activated Presentation Assistant (or VAPA) allows the presenter to use his/her voice commands to control PowerPoint slides and activate a guiding countdown timer.

Key Benefits

Plug and Play Hands Free Easy to Use Wireless Modular


Imagination is everything. Imagination is everything. It is the preview of life’s coming attractions.It is the preview of life’s coming attractions.

-- Albert EinsteinAlbert Einstein


Table of Contents Page 1. OBJECTIVE .................................................................................................................................................6 2. OVERVIEW ..................................................................................................................................................7 3. SYSTEM ASSEMBLY..................................................................................................................................8 4. OPERATION INSTRUCTIONS....................................................................................................................9 5. SETUP........................................................................................................................................................10 6. TRAINING ..................................................................................................................................................11 7. SYSTEM LEVEL BLOCK DIAGRAM AND COMPONENT DESCRIPTIONS ..........................................12 8. CIRCUIT DIAGRAM...................................................................................................................................13 9. SR-06 (SPEECH RECOGNITION) DIAGRAM ..........................................................................................14 10. COUNT DOWN TIMER DIAGRAM..........................................................................................................15 11. ARDUINO/DECODER CIRCUITRY.........................................................................................................16 12. ARDUINO/KEYPAD INTERFACE...........................................................................................................17 13. VOLTAGE THRESHOLD LEVELS FOR KEYPAD.................................................................................18 14. USER LEVEL FLOW CHART..................................................................................................................19 15. SYSTEM LEVEL FLOW CHART.............................................................................................................20 16. SOFTWARE SYSTEM OVERVIEW ........................................................................................................21 17. MASTER CONTROLLER SOFTWARE CODE.......................................................................................25 18. SLAVE CONTROLLER SOFTWARE CODE ..........................................................................................37 19. COMPONENT DESCRIPTIONS..............................................................................................................45 20. ENCLOSED ASSEMBLY ........................................................................................................................47 21. COST SHEET ..........................................................................................................................................48 22. TEST PLAN..............................................................................................................................................49 23. CONCLUSION .........................................................................................................................................52 APPENDIX .....................................................................................................................................................53


Table of Figures Page FIGURE 1 – AVAILABLE VOICE COMMANDS .............................................................................................6 FIGURE 2 – FUNCTIONAL OVERVIEW.........................................................................................................7 FIGURE 3 – PROJECT IMAGE (TOP VIEW)..................................................................................................8 FIGURE 4 – HIGH LEVEL BLOCK DIAGRAM...............................................................................................9 FIGURE 5 – TRAINED WORD INDICES.........................................................................................................9 FIGURE 6 – SETUP OPTIONS......................................................................................................................10 FIGURE 7 – KEYPAD DIAGRAM..................................................................................................................10 FIGURE 8 – HOW TO TRAIN WORDS .........................................................................................................11 FIGURE 9 – KEYPAD DIAGRAM..................................................................................................................11 FIGURE 10 – SYSTEM LEVEL BLOCK DIAGRAM .....................................................................................12 FIGURE 11 – CIRCUIT DIAGRAM................................................................................................................13 FIGURE 12 – SPEECH RECOGNITION CIRCUIT DIAGRAM .....................................................................14 FIGURE 13 – COUNTDOWN TIMER DIAGRAM..........................................................................................15 FIGURE 14 – ARDUINO/DECODER INTERFACE .......................................................................................16 FIGURE 15 – ARDUINO/KEYPAD INTERFACE ..........................................................................................17 FIGURE 16 – VOLTAGE THRESHOLD LEVELS.........................................................................................18 FIGURE 17 – USER LEVEL FLOW CHART.................................................................................................19 FIGURE 18 – SYSTEM LEVEL FLOW CHART............................................................................................20 FIGURE 19 – TOP LEVEL COMPONENT DESCRIPTIONS ........................................................................45 FIGURE 20 – ENCLOSED ASSEMBLY........................................................................................................47 FIGURE 21 – LED TRANSISTOR CIRCUIT .................................................................................................49 FIGURE 22 – DECODER PIN LAYOUT DIAGRAM......................................................................................50 FIGURE 23 – SPEECH RECOGNITION PIN LAYOUT.................................................................................53 FIGURE 24 – SPEECH RECOGNITION PIN DESCRIPTIONS ....................................................................54 FIGURE 25 – KEYPAD PIN DESCRIPTIONS...............................................................................................55 FIGURE 26 – ARDUINO / SPEECH RECOGNITION INTERFACE..............................................................56 FIGURE 27 -- TRANSMITTER CIRCUIT BOARD.........................................................................................57 FIGURE 28 – MICROPROCESSOR PIN LAYOUT (ON TRANSMITTER)...................................................58 FIGURE 29 – FUNCTIONAL BLOCK DIAGRAM .........................................................................................59 FIGURE 30 – TRANSMITTER PIN LAYOUT................................................................................................60 FIGURE 31 – FUNCTIONAL BLOCK DIAGRAM .........................................................................................61


1. Objective Purpose of Project To design a voice activated presentation assistant with a count down timer. This will allow the user to control PowerPoint without using his/her hands and also be able to note the amount of time left on his/her presentation by using the count down timer. Main Features

• Voice Activated PowerPoint Commands • Keypad Interface • Large Countdown Timer • Keypad Input • Five Unique PowerPoint Commands

Available Voice Commands

Figure 1 – Available Voice Commands

Command ActionNEXT Advances slidePREVIOUS Recalls previous slideEXIT Exits presentation modeBEGIN Begins presentationCLEAR Blanks screenTIME Activates Time


2. Overview Summary of Functionality

• Speaker says command into microphone • Timer begins countdown sequence if “Begin” command received • Binary code is relayed from voice recognition board to remote control • Remote control sends signal to laptop with receiver • PowerPoint responds by performing action

Figure 2 – Functional Overview

Speech Recognition Board

Microprocessor

Speaker

Laptop with PowerPoint

TransmitterCountdown Timer


3. System Assembly

Figure 3 – Project Image (Top View)


4. Operation Instructions

1. Power-Up Unit 2. Press Reset button on Master and Slave Arduino 3. Verify that LED’s on Timer blink Once and then Twice 4. Set mode of operation using Keypad 5. Train Commands or Set Time 6. Begin Presentation

Figure 4 – High Level Block Diagram

Figure 5 – Trained Word Indices

SpeechRecognition

Board

ArduinoMicro-

Processor

Timer &7-SegmentDisplays

VersaPointTranmitter

SpeechRecognition

Board

ArduinoMicro-

Processor

Timer &7-SegmentDisplays

VersaPointTranmitter

Word IndexWord Index CommandCommand0101 BeginBegin0202 NextNext0303 PreviousPrevious0404 BlankBlank0808 ExitExit0909 TimeTime


5. Setup

Figure 6 – Setup Options

Figure 7 – Keypad Diagram

KeyKey OptionOption1 1 Speech and TimerSpeech and Timer2 2 Speech OnlySpeech Only33 Timer Only (Speech Activated)Timer Only (Speech Activated)4 4 Timer Only (Keypad Activated)Timer Only (Keypad Activated)55 Speech and Timer (Skip Training)Speech and Timer (Skip Training)

KeyKey OptionOption1 1 Speech and TimerSpeech and Timer2 2 Speech OnlySpeech Only33 Timer Only (Speech Activated)Timer Only (Speech Activated)4 4 Timer Only (Keypad Activated)Timer Only (Keypad Activated)55 Speech and Timer (Skip Training)Speech and Timer (Skip Training)

CLR 0 TRN

7 8 9

654

1 2 3

CLR 0 TRN

7 8 9

654

1 2 3


6. Training

Figure 8 – How to Train Words

Figure 9 – Keypad Diagram

KeypadKeypadCommandCommand ActionAction0 1 TRN0 1 TRN Trains First PatternTrains First Pattern2 1 TRN2 1 TRN Trains 21st PatternTrains 21st Pattern9 9 CLR9 9 CLR Clears memoryClears memory0 1 CLR0 1 CLR Clears 1st patternClears 1st pattern

KeypadKeypadCommandCommand ActionAction0 1 TRN0 1 TRN Trains First PatternTrains First Pattern2 1 TRN2 1 TRN Trains 21st PatternTrains 21st Pattern9 9 CLR9 9 CLR Clears memoryClears memory0 1 CLR0 1 CLR Clears 1st patternClears 1st pattern

CLR 0 TRN

7 8 9

654

1 2 3

CLR 0 TRN

7 8 9

654

1 2 3


7. System Level Block Diagram and Component Descriptions

Figure 10 – System Level Block Diagram Component Functional DescriptionKeypad Allows user to train words into memory and set timerMain Arduino Microcontroller Controls timer and accepts input from keypadSecondary Arduino Controller Inputs commands to Speech Recognition circuit and transmitterDigital Displays Displays remaining time to presenterDecoder and PNP Transistors Deocdes binary code from Arduino to PNP transistors which control transmitterPNP Switch Array Activates buttons on speech recognition ports from decoder outputSpeech Recognition Board Processes speech recognition functions and outputs status codes to Second ArduinoTransmitter Tranmits RF signlal to USB reciever module to control PowerPoint

KeypadArduinoµControllerCountdown Timer

ArduinoµController

Decoder andPNP Transistors

PNPSwitchArray

SpeechRecognition

Board

Transmitter


ArduinoµController


PNPSwitchArray

SpeechRecognition

Board

Transmitter


ArduinoµController


PNPSwitchArray

SpeechRecognition

Board

Transmitter


8. Circuit Diagram

Figure 11 – Circuit Diagram


9. SR-06 (Speech Recognition) Diagram

Figure 12 – Speech Recognition Circuit Diagram

Quantity Item Description 1 HM2007 Speech Recognition IC 1 CY6264 8K X 8 SRAM 1 74LS373 8 Bit Register 2 4511 BCD to 7-Segment Decoder/Driver

16 220 Ohm Resistor Resistor (200 Ohm 1/8 Watt) 1 XTAL 3.57 MHz Crystal Oscillator 1 Main Circuit Board Printed Circuit Board 1 Keypad 12 Switches 1 Power Switch On / Off 2 Seven Segment Displays 1 Headset and Microphone 1 9V Battery Snap 1 Coin Battery Holder 1 100K ohm Resistor Resistor (100K Ohm 1/4 Watt) 1 6.8K ohm Resistor Resistor (6.8K 1.4 Watt) 1 22K ohm resistor Resistor (22K 1/4 Watt) 1 .1µFCapacitor 1 0.0047µF Capacitor 1 100µF Capacitor 1 10-22 pF Capacitor 1 7805 Linear Voltage Regulator


10. Count Down Timer Diagram

Figure 13 – Countdown Timer Diagram

Quantity Item Description 2 LED (Green) Bright Green LED 4 74HCT4543E (Texas Instruments) 7-Segment Decoder Driver with Latch 1 74LS373 8 Bit Register

28 110 Ohm Resistor 110 Ohm Resistor 1 520 Ohm Resistor 520 Ohm Resistor


11. Arduino/Decoder Circuitry

Figure 14 – Arduino/Decoder Interface

Digital Pin 8, 9, 10, 11 on Arduino connect to Pins on 1, 2, 3, 4 on Decoder (74HC154)

Quantity Item Description 5 1K Resistor Resistor 1 Arduino Microcontroller 1 74HC154 4-16 Decoder 5 2N3906 PNP Transistor 1 VersaPoint Communicator (Transmitter) RF Transmitter for PowerPoint


12. Arduino/Keypad Interface Most keypads use row plus column number of pins connected to a keypad decoder which connects to a microcontroller. If the microcontroller has a binary keypad decoder then there is no need for a keypad decoder. In our project we have used the principle of voltage dividers by using a resistor matrix. Each key produces an analog voltage ranging from 0 to 5 volts. The analog voltage is decoupled with a tiny capacitor to filter out high frequencies. The press of a key produces a voltage on the input of the ADC pin of the Arduino. The Arduino consists of a 10 bit ADC (0 to 1023 quantized values can be represented by the ADC). Using a driver code we are able to represent the button that has been pressed by its corresponding binary value. The benefits of this resistor matrix are massive as this architecture uses only one pin when interfaced to the host controller. We can use the unoccupied pins for handshaking between the two microcontrollers and controlling the timer circuitry.

Figure 15 – Arduino/Keypad Interface

1nF

1KΩ

820Ω

1KΩ

ADC0

Arduino

180Ω

680Ω

3.3KΩ

15KΩ

Keypad

+5 VDC

1 2 347 8 9* 0 #

5 6

1nF

1KΩ

820Ω

1KΩ

ADC0

Arduino

180Ω

680Ω

3.3KΩ

15KΩ

Keypad

+5 VDC

1 2 347 8 9* 0 #

5 61 2 347 8 9* 0 #

5 6


13. Voltage Threshold Levels for Keypad

Figure 16 – Voltage Threshold Levels Read By ADC on Arduino

4.7264.7004.671#4.5884.5504.50704.2984.2374.170*3.9173.8323.74093.5003.3963.28582.8092.6882.56372.1392.0201.95961.6271.5221.46851.0480.9690.93040.6980.6410.58830.4740.4340.39620.2720.2480.2251MaxAvgMin

Voltage at ADC0Key

4.7264.7004.671#4.5884.5504.50704.2984.2374.170*3.9173.8323.74093.5003.3963.28582.8092.6882.56372.1392.0201.95961.6271.5221.46851.0480.9690.93040.6980.6410.58830.4740.4340.39620.2720.2480.2251MaxAvgMin

Voltage at ADC0Key


14. User Level Flow Chart

Figure 17 – User Level Flow Chart

Start

NoTrain Words

Set Time

TrainMode?

Yes Is Presentation

Over?

BeginPresentation

ContinuePresentation

End

Yes

No

Start

NoTrain Words

Set Time

TrainMode?

Yes

Start

NoTrain Words

Set Time

TrainMode?

Yes Is Presentation

Over?

BeginPresentation

ContinuePresentation

End

Yes

No


15. System Level Flow Chart

Figure 18 – System Level Flow Chart

Start Start

Master Slave

Select Mode

Train Words

Set Time

Count Down

Mode 1, 2, 3, 5

Mode 2, 3, 4, 5

Press CLR

Train Words

Press 5,6

YES

NO

Keypad buttons To SR

Press 5,6

Handshake

Recognize Words

Paused

TRN/”Time”

Valid word?

TRN/“Time”

From SR

NO

Transmit

To Transmitte

r

Time/Begin

To Timer

YES

NO

YES

CLR/”Begin”YES

Start Start

Master Slave

Select Mode

Train Words

Set Time

Count Down

Mode 1, 2, 3, 5

Mode 2, 3, 4, 5

Press CLR

Train Words

Press 5,6

YES

NO


Press 5,6

Handshake

Recognize Words

Paused

TRN/”Time”

Valid word?

TRN/“Time”

From SR

NO

Transmit

To Transmitte

r

Time/Begin

To Timer

YES

NO

YES

CLR/”Begin”YES

ToTransmitter

Start Start

Master Slave

Select Mode

Train Words

Set Time

Count Down

Mode 1, 2, 3, 5

Mode 2, 3, 4, 5

Press CLR

Train Words

Press 5,6

YES

NO


Press 5,6

Handshake

Recognize Words

Paused

TRN/”Time”

Valid word?

TRN/“Time”

From SR

NO

Transmit

To Transmitte

r

Time/Begin

To Timer

YES

NO

YES

CLR/”Begin”YES

Start Start

Master Slave

Select Mode

Train Words

Set Time

Count Down

Mode 1, 2, 3, 5

Mode 2, 3, 4, 5

Press CLR

Train Words

Press 5,6

YES

NO


Press 5,6

Handshake

Recognize Words

Paused

TRN/”Time”

Valid word?

TRN/“Time”

From SR

NO

Transmit

To Transmitte

r

Time/Begin

To Timer

YES

NO

YES

CLR/”Begin”YES

ToTransmitter


16. Software System Overview The Voice Activated Presentation Assistant (VAPA) uses two Arduino microcontrollers: a master and a slave. The master Arduino takes in keypad inputs and controls the timer while the slave Arduino interfaces with the speech recognition circuit as well as the transmitter. Both Arduinos communicate together through their serial transmit and receive lines. Operation of the VAPA is conducted as shown below:

1. Plug power into an Arduino. 2. Wait for the ‘:’ LEDs on the timer to flash once, then twice 3. Turn on the speech recognition circuit 4. Select an operation mode by pressing a key between 1 and 5 on the keypad 5. Train the commands by pressing the appropriate number and then ‘TRN’ 6. Press ‘5’ and then ‘6’ to finish training 7. Enter time on timer 8. Begin speaking commands

Main Components: Keypad The keypad, which is connected to the master Arduino through one analog pin, transmits the different button values using a resistor network. The Arduino will read the analog pin and depending on the voltage value it gets, will assign a value that corresponds to the pressed button to a global variable and will return ‘0’ if there was a valid key pressed, and ‘-1’ otherwise. To distinguish between a key being pressed and one that is held down, the analog values is compared to the previous value and only if the two are not equal will it be possible for a valid press to register. This is because when no buttons are pressed, the analog pin would read a different value from the keypad. Therefore, buttons must be released in between presses. Speech Recognition Circuit The speech recognition (SR) circuit is connected to the slave Arduino for both input as well as output. Originally, the keypad was connected directly to the speech recognition circuit. A button press would short two of the pins on the input bus of the SR, resulting in an input. Since the keypad is used in all aspects of VAPA, it cannot be connected directly to the SR. Instead, transistors that connect the pins of the SR input bus allow the Arduino to simulate the button presses by sending a signal, which activates the transistor and shorts the two pins together.


The output of the SR is also connected to the slave Arduino. It reads the data through its connection to six of the eight data pins of the output bus, which are in binary coded decimal. This allows the Arduino to read numbers up to 39. Timer The timer consists of four seven-segment displays, each with a display driver, which also serves as a latch. The master Arduino interfaces with the timer using 10 digital pins. Four pins are used as a data line to be able to send the numbers 0 through 9 in BCD to the timer’s universal data bus that connects to all four display drivers. Another four pins are used to connect to a latch disable pin on each of the 4 display drivers. Sending a ‘HIGH’ signal to the latch disable pin will all the input of the driver to pass to the output. A ‘LOW’ on the latch disable will capture what is on the input during the high to low transition, hold it on the output, and not let any inputs pass through. One pin is used to connect to the blank pin on all of the latches, which turns off the displays. Finally, one pin is used to control the ‘:’ between the minutes and seconds. Displaying a number on one of the displays of the timer is done as followed: Put the number of the data bus Set the latch disable pin of the appropriate driver HIGH, and then LOW This is repeated for the remaining three displays. Decoder To save pins, a decoder is used to connect between the slave Arduino and the transistors for simulating the button presses on the SR and the transmitter. The decoder is a 4-to-16 decoder and has 13 of its outputs used by the VAPA. Five of the outputs are used for the five buttons on the transmitter. Eight outputs are used for the input bus of the SR. These are 1, 2, 3, 4, 8, 9, TRN, and CLR. 5, 6, and 7 are unnecessary and therefore not used. VAPA Software System Operation: Startup When the system is powered up, the master Arduino will flash the timer’s ‘:’ indicating that it is ready and wait for the slave to start up. When it does, it will send the character ‘R’ to the master through the serial lines and the master will flash the timer’s ‘:’ twice.


Mode Selection Once the system is ready the user will first press a key on the keypad to select a mode of operation. The entered key will set a global variable on the master to determine which of the SR and timer will be used. In each function to train words and to set the time on the timer, this global variable will be checked before proceeding. Train Words In this mode, keypad buttons are checked to ensure they are valid values for the SR, and will then be sent to the slave Arduino. The slave Arduino will then activate the corresponding buttons on the SR. When the user presses the keys ‘5’ and ‘6’ in succession, training will be complete and the master will no longer send button values to the slave. Set Time After commands are trained, the user will enter the time of the presentation length on the timer. Times are entered in a similar way as on a microwave. Numbers are shifted in from right to left with each entry, until four numbers have been entered. Each time a number is entered, variables for minutes and seconds are updated. When the ‘CLR’ buttons is pressed during set mode, numbers are all shifted right by one, discarding the rightmost number. Command Recognition Mode Once a time has been entered, the user can immediately begin speaking voice commands. The slave Arduino handles the outputs of the SR. If the slave reads a valid command it will take one of the following actions:

The first 5 commands will all cause their corresponding commands to be sent by the transmitter. If begin is spoken during set time mode, the timer will also start counting down. If the “Time” command is spoken while the timer is counting down, it will pause the timer, while speaking “Time” when it is paused will start the timer counting down again. The SR works by placing the value it gets on the output bus and keeping it there until a different word is recognized or an error code is output. This presents the problem of not being able tell when a certain word is spoken twice in a row because there would be no

Command Transmit TimerBegin YES Start timeNext YES

Previous YESBlank YESExit YES

Time NO Pause/Restart


change in the output when the word is recognized the second time. To alleviate this problem, upon recognizing a valid word, the slave Arduino will send the pattern 1, 8, CLR to the SR, which serves to separate back-to-back recognitions. Timer Countdown Mode Once a time has been entered, countdown can begin by either speaking the word “Begin”, or by pressing the ‘TRN’ button on the keypad. In either case, the time will be checked to ensure it is valid by using the variables for minutes and seconds. Minutes must be between 0 and 99 while seconds must be between 0 and 59, providing a maximum time duration of 99:59. A time of 00:00 will also not be a valid time. Once the time runs out, the timer will blink 00:00. The software for the timer works by using a pair of nested while loops. The outer while loop is for minutes which checks minutes > 0 while the inner loop checks seconds > 0. For both loops, their respective time variable, minutes or seconds, is decremented after one iteration. Each time the while loop for seconds exits (seconds = 0), seconds is set to 59. When the time variables are updated, they are printed to the displays and master delays for 1 second. During a standard delay, the Arduino cannot do anything. However, the timer needs to be able to accept input from the keypad and slave to start, pause, or clear the time. Therefore, VAPA uses a special function, which breaks up the 1-second delay into 40 equal time chunks. During each time chunk, the Arduino will delay for 25ms and then perform a non-blocking read of the keypad and the serial port. If it gets input, it will take the appropriate action. Otherwise, it will continue delaying and checking for inputs until the second is over. By checking for inputs 40 times per second, the timer is very responsive to inputs from either the keypad or the serial port. The timer can be cleared during countdown or while it is paused by pressing ‘CLR’ on the keypad. This puts the timer back into set time mode. When the time finishes and the displays are blinking “00:00”, pressing ‘CLR’ will put the timer into set time mode as well.


17. Master Controller Software Code //-------CONSTANTS---------// //Keypad Buttons const int KEY_0 = 0; const int KEY_1 = 1; const int KEY_2 = 2; const int KEY_3 = 3; const int KEY_4 = 4; const int KEY_5 = 5; const int KEY_6 = 6; const int KEY_7 = 7; const int KEY_8 = 8; const int KEY_9 = 9; const int KEY_CLR = 10; const int KEY_TRN = 11; const char keyChar[12] = '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'C', 'T'; //Operation modes const int MODE_SPEECH_TIMER = 1; const int MODE_SPEECH = 2; const int MODE_TIMER_SC = 3; const int MODE_TIMER_KC = 4; const int MODE_SPEECH_TIMER_DT = 5; //System states const int STATE_MODE = 0; const int STATE_TRAIN = 1; const int STATE_SETTIME = 2; const int STATE_COUNTDOWN = 3; const int STATE_PAUSED = 4; //LED const int ledPin = 2; //Latch disable pins const int LD3 = 11; const int LD2 = 10; const int LD1 = 9; const int LD0 = 8; //BCD decoder inputs const int D3 = 7; const int D2 = 6; const int D1 = 5;


Master Controller Software Code (Continued) const int D0 = 4; //Blank const int BL = 3; const int keypadPin = 5; /////////////////////////////// //------VARIABLES------------// int opMode = -1; int button = -1; int lastButton = -2; char srCommand = -1; int mins = 0; int secs = 0; int inByte = -1; int keysToGet = 4; int digit3 = 0; int digit2 = 0; int digit1 = 0; int digit0 = 0; int inChar = -1; int sysState = STATE_MODE; boolean hasRun = false; /////////////////////////////// //--------PROTOTYPES----------// void modeSelect(); void trainWords(); void setTime(); void getKey(); int getKeypadButton(); boolean voiceButton(char); //Timer Functions void ready();


Master Controller Software Code (Continued) void latch(int); void dBus(int); void flashLED(int); void countDown(); void getKeyDW(); void kpNumber(); void kpClear(); void kpEnter(); void polledDelay(int); void printTime(int, int); //////////////////////////////// void setup() pinMode(ledPin, OUTPUT); pinMode(LD3, OUTPUT); pinMode(LD2, OUTPUT); pinMode(LD1, OUTPUT); pinMode(LD0, OUTPUT); pinMode(D3, OUTPUT); pinMode(D2, OUTPUT); pinMode(D1, OUTPUT); pinMode(D0, OUTPUT); pinMode(BL, OUTPUT); Serial.begin(9600); flashLED(500); void loop() if( hasRun == false ) hasRun = true; ready(); modeSelect(); trainWords(); setTime();


Master Controller Software Code (Continued) void ready() while( true ) if( Serial.available() > 0 ) inChar = Serial.read(); if( inChar == 'R' ) flashLED(500); delay(250); flashLED(500); break; void modeSelect() sysState = STATE_MODE; getKey(); opMode = button; void trainWords() if( opMode == MODE_SPEECH_TIMER || opMode == MODE_SPEECH || opMode == MODE_TIMER_SC ) sysState = STATE_TRAIN; int leftDigit = 0; int rightDigit = 0; while( !(leftDigit == KEY_5 && rightDigit == KEY_6) ) getKey(); if( voiceButton(keyChar[button]) ) Serial.print(keyChar[button]);


Master Controller Software Code (Continued) leftDigit = rightDigit; rightDigit = button; Serial.print('R'); //Tell slave Arduino to Run void setTime() if( opMode == MODE_SPEECH_TIMER || opMode == MODE_TIMER_SC || opMode == MODE_TIMER_KC || opMode == MODE_SPEECH_TIMER_DT ) sysState = STATE_SETTIME; while( true ) getKeyDW(); void kpNumber() if( (keysToGet > 0) && !((keysToGet == 4) && (inByte == 0)) && (sysState == STATE_SETTIME) ) mins = mins*10 + (secs/10); secs = (secs%10)*10 + inByte; keysToGet--; printTime(mins, secs); void kpClear() if( sysState == STATE_SETTIME ) secs = (mins%10)*10 + secs/10; mins = mins/10; if( keysToGet < 4 ) keysToGet++;


Master Controller Software Code (Continued) else if( sysState == STATE_PAUSED || sysState == STATE_COUNTDOWN ) sysState = STATE_SETTIME; mins = 0; secs = 0; keysToGet = 4; printTime(mins, secs); void kpEnter() if( (validTime(mins, secs) && sysState == STATE_SETTIME) || (sysState == STATE_PAUSED) && (srCommand != 'B') ) sysState = STATE_COUNTDOWN; countDown(); //"Begin" cannot pause or restart time else if( (sysState == STATE_COUNTDOWN) && (srCommand != 'B') ) sysState = STATE_PAUSED; void getKey() boolean hasKey = false; while( !hasKey ) if( getKeypadButton() == 0 ) srCommand = -1; hasKey = true;


Master Controller Software Code (Continued) void getKeyDW() srCommand = -1; //Speech input if( Serial.available() > 0 ) srCommand = Serial.read(); if( srCommand == 'B' || srCommand == 'P') //Begin or Pause kpEnter(); //Keypad input else if( getKeypadButton() == 0 ) inByte = button; if( inByte != -1 ) if( (inByte >= KEY_0) && (inByte <= KEY_9) ) kpNumber(); else if( inByte == KEY_CLR ) kpClear(); else if( inByte == KEY_TRN ) kpEnter(); boolean validTime(int minutes, int seconds) if( (minutes >= 0 && minutes <= 99) && (seconds >= 0 && seconds <= 59) ) if( !(minutes == 0 && seconds == 0) ) return true; return false;


Master Controller Software Code (Continued) boolean voiceButton(char keyPressed) if( keyPressed == '1' || keyPressed == '2' || keyPressed == '3' || keyPressed == '4' || keyPressed == '8' || keyPressed == '9' || keyPressed == 'T' || keyPressed == 'C') return true; return false; //Returns 0 if valid key pressed, else -1 int getKeypadButton() int keyVal = analogRead(keypadPin); int keyButton = -1; lastButton = button; if( keyVal >= 40 && keyVal <= 55 ) keyButton = KEY_1; else if( keyVal >= 80 && keyVal <= 90 ) keyButton = KEY_2; else if( keyVal >= 120 && keyVal <= 135 ) keyButton = KEY_3; else if( keyVal >= 190 && keyVal <= 200 ) keyButton = KEY_4; else if( keyVal >= 300 && keyVal <= 315 ) keyButton = KEY_5; else if( keyVal >= 400 && keyVal <= 415 ) keyButton = KEY_6; else if( keyVal >= 540 && keyVal <= 550 ) keyButton = KEY_7; else if( keyVal >= 685 && keyVal <= 700 ) keyButton = KEY_8; else if( keyVal >= 775 && keyVal <= 790 ) keyButton = KEY_9; else if( keyVal >= 860 && keyVal <= 870 ) keyButton = KEY_CLR; else if( keyVal >= 920 && keyVal <= 935 ) keyButton = KEY_0; else if( keyVal >= 955 && keyVal <= 970 ) keyButton = KEY_TRN;


Master Controller Software Code (Continued) else keyButton = -1; button = keyButton; if( button != lastButton && button != -1 ) return 0; else return -1; void latch(int displayNum) digitalWrite(displayNum, HIGH); digitalWrite(displayNum, LOW); void dBus(int number) int d3 = LOW; int d2 = LOW; int d1 = LOW; int d0 = LOW; if (number % 2 == 1) d0 = HIGH; number = number >> 1; if ( number % 2 == 1) d1 = HIGH; number = number >> 1; if (number % 2 == 1) d2 = HIGH; number = number >> 1; if (number % 2 == 1) d3 = HIGH; digitalWrite(D3, d3);


Master Controller Software Code (Continued) digitalWrite(D2, d2); digitalWrite(D1, d1); digitalWrite(D0, d0); void flashLED(int time) digitalWrite(ledPin, HIGH); delay(time); digitalWrite(ledPin, LOW); void countDown() boolean stop = false; digitalWrite(ledPin, HIGH); while( (mins>=0) && (sysState == STATE_COUNTDOWN) ) while( (secs >= 0) && (sysState == STATE_COUNTDOWN) ) printTime(mins, secs); if( sysState == STATE_COUNTDOWN ) flashLED(250); else if ( sysState == STATE_PAUSED ) digitalWrite(ledPin, HIGH); if( sysState != STATE_PAUSED ) secs--; polledDelay(1000); if( sysState == STATE_COUNTDOWN ) mins--; secs = 59; while (true)


Master Controller Software Code (Continued) if( sysState == STATE_SETTIME || sysState == STATE_PAUSED ) break; printTime(0, 0); flashLED(500); polledDelay(500); digitalWrite(BL, HIGH); polledDelay(500); digitalWrite(BL, LOW); void printTime(int minutes, int seconds) int M1 = 0; int M0 = 0; int S1 = 0; int S0 = 0; M1 = minutes/10; M0 = minutes%10; S1 = seconds/10; S0 = seconds%10; dBus(M1); latch(LD3); dBus(M0); latch(LD2); dBus(S1); latch(LD1); dBus(S0); latch(LD0); void polledDelay(int length) int period = 25; int times = length/period; while( times > 0 )


Master Controller Software Code (Continued) getKeyDW(); delay(period); times--;


18. Slave Controller Software Code //-------CONSTANTS----------// //Keypad Buttons const char KEY_1 = '1'; const char KEY_2 = '2'; const char KEY_3 = '3'; const char KEY_4 = '4'; const char KEY_8 = '8'; const char KEY_9 = '9'; const char KEY_TRN = 'T'; const char KEY_CLR = 'C'; //SR values const int TR_BEGIN = 1; const int TR_NEXT = 2; const int TR_PREVIOUS = 3; const int TR_BLANK = 4; const int TR_EXIT = 8; const int TR_TIME = 9; //SR output pins const int SR_OUT_0 = 2; const int SR_OUT_1 = 3; const int SR_OUT_2 = 4; const int SR_OUT_3 = 5; const int SR_OUT_4 = 6; const int SR_OUT_5 = 7; //Transmitter decoder inputs const int DEC_BEGIN = 3; const int DEC_NEXT = 4; const int DEC_PREVIOUS = 2; const int DEC_BLANK = 1; const int DEC_EXIT = 5; //Decoder Inputs const int SR_1 = 6; const int SR_2 = 7; const int SR_3 = 8; const int SR_4 = 9; const int SR_8 = 10; const int SR_9 = 11; const int SR_TRN = 12; const int SR_CLR = 13;


Slave Controller Software Code (Continued) const int DEC_0 = 8; const int DEC_1 = 9; const int DEC_2 = 10; const int DEC_3 = 11; const int STATE_INIT = 0; const int STATE_TRAIN = 1; const int STATE_RUN = 2; ////////////////////////////// //--------VARIABLES---------// char inChar = -1; boolean hasRun = false; int sysState = STATE_INIT; int srVal = 0; int srPrevVal = -1; ////////////////////////////// //---------PROTOTYPES-------// void trainWords(); void slideNav(); void srPress(char); void getInput(); void readSR(); void interruptSRCom(); int binToInt(int, int, int, int); ///////////////////////////// void setup() pinMode(DEC_0, OUTPUT); pinMode(DEC_1, OUTPUT); pinMode(DEC_2, OUTPUT); pinMode(DEC_3, OUTPUT); pinMode(SR_OUT_0, INPUT); pinMode(SR_OUT_1, INPUT); pinMode(SR_OUT_2, INPUT); pinMode(SR_OUT_3, INPUT); pinMode(SR_OUT_4, INPUT); pinMode(SR_OUT_5, INPUT); digitalWrite(DEC_0, LOW); digitalWrite(DEC_1, LOW); digitalWrite(DEC_2, LOW);


Slave Controller Software Code (Continued) digitalWrite(DEC_3, LOW); Serial.begin(9600); void loop() if( !hasRun ) hasRun = true; delay(5000); Serial.print('R'); //Tell master I am ready trainWords(); slideNav(); void trainWords() sysState = STATE_TRAIN; getInput(); while( inChar != 'R' ) srPress(inChar); getInput(); void slideNav() sysState = STATE_RUN; interruptSRCom(); while( true ) readSR();


Slave Controller Software Code (Continued) void srPress(char srButton) int decIn = -1; if( srButton == KEY_1 ) decIn = SR_1; else if( srButton == KEY_2 ) decIn = SR_2; else if( srButton == KEY_3 ) decIn = SR_3; else if( srButton == KEY_4 ) decIn = SR_4; else if( srButton == KEY_8 ) decIn = SR_8; else if( srButton == KEY_9 ) decIn = SR_9; else if( srButton == KEY_TRN ) decIn = SR_TRN; else if( srButton == KEY_CLR ) decIn = SR_CLR; if( decIn != -1 ) decoder(decIn, false); void decoder(int number, boolean transmit) int i = 0; int origNumber = number; boolean digit[4] = false, false, false, false; int decPin[4] = DEC_0, DEC_1, DEC_2, DEC_3; for(i=0; i <= 3; i++) if( number%2 == 1 ) digit[i] = true; number = number/2; for(i=0; i <= 3; i++)


Slave Controller Software Code (Continued) digitalWrite(decPin[i], digit[i]); if( transmit ) if( (origNumber == DEC_EXIT) && (sysState == STATE_RUN) ) delay(3000); else delay(500); else delay(100); for(i=0; i<= 3; i++) digitalWrite(decPin[i], LOW); void getInput() boolean haveInput = false; while( !haveInput ) if( Serial.available() > 0 ) inChar = Serial.read(); haveInput = true; void readSR() int sr0 = 0; int sr1 = 0; int sr2 = 0; int sr3 = 0; int sr4 = 0; int sr5 = 0;


Slave Controller Software Code (Continued) int command[2][5] = TR_BEGIN, TR_NEXT, TR_PREVIOUS, TR_BLANK, TR_EXIT, DEC_BEGIN, DEC_NEXT, DEC_PREVIOUS, DEC_BLANK, DEC_EXIT; int val0 = 0; int val1 = 0; int val = 0; sr0 = digitalRead(SR_OUT_0); sr1 = digitalRead(SR_OUT_1); sr2 = digitalRead(SR_OUT_2); sr3 = digitalRead(SR_OUT_3); sr4 = digitalRead(SR_OUT_4); sr5 = digitalRead(SR_OUT_5); val0 = binToInt(sr3, sr2, sr1, sr0); val1 = binToInt(sr5, sr4); val = (val1 * 10) + val0; srPrevVal = srVal; srVal = val; int i = 0; for(i = 0; i <= 5; i++) if( val == command[0][i] ) delay(75); sr0 = digitalRead(SR_OUT_0); sr1 = digitalRead(SR_OUT_1); sr2 = digitalRead(SR_OUT_2); sr3 = digitalRead(SR_OUT_3); sr4 = digitalRead(SR_OUT_4); sr5 = digitalRead(SR_OUT_5); val0 = binToInt(sr3, sr2, sr1, sr0); val1 = binToInt(sr5, sr4); val = (val1 * 10) + val0;


Slave Controller Software Code (Continued) break; for(i = 0; i <= 5; i++) if( val == command[0][i] ) decoder(command[1][i], true); interruptSRCom(); if( val == TR_BEGIN ) Serial.print('B'); break; if( val == TR_TIME ) Serial.print('P'); //Pause for time interruptSRCom(); int binToInt(int d1, int d0) int val = 0; if( d0 == 1 ) val += 1; if( d1 == 1 ) val += 2; return val; void interruptSRCom() delay(50); srPress(KEY_1); delay(50);


Slave Controller Software Code (Continued) srPress(KEY_8); delay(50); srPress(KEY_CLR); delay(75); int binToInt(int d3, int d2, int d1, int d0) int val = 0; if( d0 == 1 ) val += 1; if( d1 == 1 ) val += 2; if( d2 == 1 ) val += 4; if( d3 == 1 ) val += 8; return val;


19. Component Descriptions

Figure 19 – Top Level Component Descriptions

33 2211

44

8877

55 66


Component Descriptions (Continued)

1. Timer

2. Master Controller

3. Keypad Interface

4. Transmitter Switches

5. Transmitter

6. Speech Recognition Switches

7. Speech Recognition Board

8. Slave Controller


20. Enclosed Assembly

Figure 20 – Enclosed Assembly


21. Cost Sheet

Arduino MicrocontrollersArduino Microcontrollers $60.00$60.00VersaPoint TransmitterVersaPoint Transmitter $40.00$40.00Speech Recognition BoardSpeech Recognition Board $100.00$100.0074HC154 Decoder74HC154 Decoder $2.00$2.00PNP TransistorsPNP Transistors $5.00$5.0074HC4543 Drivers74HC4543 Drivers $8.00$8.0077--Segment DisplaysSegment Displays $10.00$10.00BreadboardsBreadboards $30.00$30.00Power SupplyPower Supply $20.00$20.00Resistors and WiresResistors and Wires $15.00$15.00Cover and Wooden SupportCover and Wooden Support $60.00$60.00

Total CostTotal Cost $350.00$350.00


22. Test Plan • Check for Damaged Transistors

Verify that both PNP and MOSFET devices are working. Construct simple LED Transistor circuit as shown below to determine if LED turns on. Replace defective transistors.

Figure 21 – LED Transistor Circuit


• Check Decoder Voltages on Input / Output Lines

Apply Voltmeter on pins 1 to 13 of Decoder and verify that the voltages are 5V for an input of 0000 on the input pins. Note that the decoder outputs active low output. Verify that E1 and E0 are both grounded.

Figure 22 – Decoder Pin Layout Diagram

• Test Speech Recognition Circuit

Connect keypad and status display board to SR-06 and train words. Power on circuit using external power source or 9V battery. Train words by pressing pattern number followed by TRN button. Speak command into microphone. Make sure command is no more than 1.92 seconds. If word is successfully processed the red LED will blink once. Test that the words will be recognized by saying the word clearly. The pattern number of the word should appear on the status display board. If the word is not recognized, too long, or too short the following error codes should display on the board.

SR-06 Error Codes:

55 = word too long 66 = word too short 77 = word no match


IMPORTANT

Never short power and ground on SR-06 as this will rapidly heat up voltage regulator and may cause permanent damage to the circuit.

• Verify that VersaPoint Communicator can transmit

Turn on PowerPoint and open a presentation. Connect 3V source to Transmitter and short each pair of terminals (red and black) Check that LED on the other side will blink to indicate successful transmission of signal.

• Test Countdown Timer

Select Timer Only Mode and Set time with Keypad. Press TRN to start countdown.

• Verify that Keypad inputs work

Use a test program that will read inputs from keypad and output both to status board and timer displays

• Check power and ground connections

Verify that power source is being properly distributed to all major components. Measure input voltage on transmitter from 3V regulator. Measure input voltage on SR-06 from Arduino. (Verify that it is 9V) Measure output voltage across 5V regulator on SR-06. Measure input voltages on Timer and Keypad.


23. Conclusion Overall we have concluded that our project works successfully. Before reaching this milestone we encountered several learning experiences that will be discussed. We realized that the Microcontroller Interface between the Arduino and Speech Recognition circuit could have been easily done after trying to directly interface the Speech Kit to the Microcontroller in CPU mode. We debugged the system for about 3 weeks after we abandoned the decision to use CPU mode. We used a much easier interface which connects the Speech Kit to the Microcontroller using switches. The mode of operation for the Speech Kit was Manual mode which was much easier to work with. In addition to this problem we had to debug the noise on the board. We determined that there was noise causing the timer to flicker values. However, once the Arduino’s reset we noticed that the timers blanked out which was good. We learned about Common Annode and Common Cathode configurations for Display Components and learned how to use them with our LCD drivers. Eventually with some ingenious programming we successfully programmed a countdown timer with reset capabilities. We learned that we could use a keypad configured with a resistor matrix to reduce the number of ports used for the keypad and programmed a driver program to accept the commands from the keypad. After noticing that the Speech Kit’s onboard regulator heated up considerably we purchased a clip on heat sink and mounted it on the regulator after relocating the regulator to the breadboard.


Appendix HM2007 Pin Descriptions

Figure 23 – Speech Recognition Pin Layout

Pin Names Pin Number(s) Description

SA0-12 17-24, 27-31 Memory address bus – connections are between the HM2007 and the static RAM IC

D0-7 36-43 Data input bus – three connections for each line: the HM2007, the static RAM, and the microcontrollers inputs

MR/MW and ME 35, 34 Additional control lines for writing to and reading from external RAM

– connection is between HM2007 and RAM

X1 and X2 2, 3 Connection pins for 3.58 MHz crystal

S1-3 4-6 Read/write control pins – connection to microcontroller I/O

K1-4 8-11 Bi-directional data bus (K-bus) – connection to microcontroller I/O

TEST 12 Not used – pull low

LINE 45 Not used – leave disconnected

MICin 46 Microphone connection

WLEN 13 Word-length select pin. Pull high to use 1.92-second words, pull low to use 0.9-second words

CPUM 14 CPU/manual-mode select pin. Pull low for manual mode, pull high for CPU mode. Pulled high to select CPU mode in this report


Speech Recognition Circuit

Figure 24 – Speech Recognition Pin Descriptions

Status LED

GN

D

POW

ER

D7

D6

D5

D4

D3

D2

D1

D0

Voltage Regulator

S1

S2

S3

K1

K2

K3

K4


Keypad Diagram Pin # Corresponding Pin Name 4 S1 5 S2 6 S3 8 K1 9 K2 10 K3 11 K4

Figure 25 – Keypad Pin Descriptions


SR-06/Arduino Interface

Figure 26 – Arduino / Speech Recognition Interface

Pin Mapping Data Bus Arduino On SR-06 Pins D0 2 D1 3 D2 4 D3 5 D4 6 D5 7


The VersaPoint Commuinicator

Figure 27 -- Transmitter Circuit Board


Main Components On The VersaPoint Communicator The Microprocessor The EM78P156EL is an 8-bit microprocessor designed by Elan Microelectronics Corporation that is designed with CMOS technology. A unique feature of the chip is its OTP-ROM. OTP-ROM stands for One Time Programmable Read Only Memory. Major features include.

• 1K X 13 bits on chip ROM • 48 X 8 bits on chip registers (SRAM, and General Purpose Registers) • 2 bi-directional I/O ports • Low power consumption • Power down mode

Figure 28 – Microprocessor Pin Layout (On Transmitter)


Figure 29 – Functional Block Diagram The instruction set in the microprocessor consists of 13-bit words that are composed of an OP code and several operands. Usually, the instructions are executed with one instruction cycle. However, if the PC becomes changed by a MOV, ADD, or by instructions of arithmetic or logic operation then the execution takes two cycles.


The Transmitter The TDA5102 is designed by Infineon Technologies and is an ASK, FSK transmitter whose transmission frequency band is between 905 and 925 MHz. Major features include

• High level of integration • Need for only a few external components • Integration of a PLL (frequency) synthesizer • High efficiency power amplifier to drive a loop antenna • Power down mode • Selectable XTAL oscillator at 7.15MHz/14.3MHz

Figure 30 – Transmitter Pin Layout


Figure 31 – Functional Block Diagram

Voice Activated Presentation Assistant

Documents

Transcript of Voice Activated Presentation Assistant