2. Introduction

2.1 Definition As the name suggests our application aims to integrate voice

recognition abilities in operating systems. The application let the user control the laptop / PC using the voice commands. If the device is connected to a network, the commands can be passed to the device using an android device.

2.2 Objective

The objective of the application is to give user full access without touching the device itself. The application takes the voice input from the user. It uses open source libraries or services to get the most accurate string that the user said. The application compares the string with the commands set and does the functionalities that are meant to be done following the voice command.

2.3 Scope

1) Access to OS for especially abled person:

Providing Operating System access by voice recognition can really help for blind and people with defected limb (handicapped person). The system needs only voice inputs and these people can use system as well as a normal person do.

1 | P a g e

2) Remote access to OS without any physical touch:

System control with voice provide the luxury to access it without any physical touch. It makes the work easy, interesting and more interactive.

3) Access to system for a lay person:

You don’t need to learn any technical aspect to access the system. You are interacting with the system in simple strings which is a part of our day to day life. It is as simple as a conversation with another human being.

4) More interesting interaction with the system.

Doing work remotely without any physical touch and commanding your system via voice is more interactive than pressing keys and moving mouse.

5) Faster access

You are not reaching to the system. You are bypassing all the steps to reach the final output. It saves a lot of time.

E.g. compare between these scenarios:

- Reach to the system,

Open Microsoft office PowerPoint,

File->Open,

Brows to your presentation,

Open it.

- Say “Open PowerPoint”,

Say “Open minor Project Presentation”.

2 | P a g e

2.4 Project Profile

Project Title Voice control in OS

Objective

This application allows the user to control the OS using voice commands. Voice commands can be given to the PC or an android application.

Organization Bugs Computer & Systems

Operating System Windows

User InterfaceNo UI for PC application.Android application uses UI.

Internal Guide

Prof. Jigna Patel

Dr. Kamal Mehta

Prof. Prajakta Rathod

External Guide Mr. Pradeep Wadekar

Submitted By

Saurabh Gaur (10BIT033)

Sarang Singhal (10BIT022)

Fahim Sheikh (09BIT155)

Submitted To

Department Of Computer Engineering, Institute Of Technology, Nirma University, Ahmedabad.

2.5 Project Team

3 | P a g e

Saurabh Gaur (10BIT033)

Sarang Singhal (10BIT022)

Fahim Sheikh (09BIT155)

2.6 Project Scheduling

4 | P a g e

3. System Analysis

3.1 Feasibility Study

A feasibility study is a short, focused study which aims to answer a number of questions:-

1) How efficient is the voice recognition? Does it actually gives faster remote access to the user?

2) Is the voice recognition facility is a burden to process? 3) Is the process really user friendly? Or user ends up cramming the

commands? Can we add user personalization?

3.1.1 Operational feasibility

Operational feasibility measures how well the solution will work in the market and how will end-user feels about system?

Proposed application helps the user to control the PC without touching it. Using PC’s recorder as a command receiver doesn’t help that much in many cases. PC’s recorder generally catches correct and accurate strings within 4 meters of range and thus help in close PC control, but doesn’t helps the idea of controlling the device with flexibility. User might end up saying the command 4-5 times to get the application catch the correct command.

To overcome this range issue, one Android application is made which commutes with the device using network. User have to enter only the IP address of the machine and application transfer the string from phone to PC being control.

5 | P a g e

This extension brings in these advantages,

1) Recorder in the phone is of much better quality than the PC/Laptop. So, the captured strings are more accurate.

2) PC/Laptop recorder has 4-5 meter range. Connecting android device increase the range. Command can be commuted in a wifi network range.

3) Adding a new device in the control transit to a new idea, an Android application which controls the PC and device itself.

3.1.2 Technical feasibility

To implement the application, the main functionality is of the voice recognition module. We have tested open source libraries and services to get the accurate string results in the output.

Making a whole new voice recognition can be a separate project. It’s better to use available open source libraries.

3.1.3 Financial and Economic feasibility

Economic feasibility looks at the financial aspects of the project. Economic feasibility concerns with the returns from the investments in a project. It determines whether it is worthwhile to invest the money in the proposed system. It is not worthwhile spending a lot of money on a project for no returns. To carry out an economic feasibility for a system, it is necessary to place actual money value against any activities needed to implement the project.

6 | P a g e

3.2 Requirement Specification Java Application in PC/Laptop:

For some task administrative privileges are needed.

Android application:

It uses WIFI services of the phone and google voice recognition services. Android versions 4+ have the voice recognition engines installed by default.

7 | P a g e

4. System Design

4.1 Component of the application

4.1.1 PC application

This application uses no UI. Logical components are given below.

The application works in the background. It gets the user voice commands and the string output is generated. The string is then compared with available commands and after the correct match, the command lines for the work to be done are executed.

8 | P a g e

4.1.2 Android Application

9 | P a g e

Application Logo

Application trigger: By pressing this button, user can initiate the command recognition

Choice: Radio choices provided to select the device being controlled.

IP Address of the server machine: The text box becomes visible when PC is being controlled by the mobile over network.User has to give the IP address of the device.

4.2 Sequence Diagram10 | P a g e

Google Voice services: In the PC version of the application, we are using Sphinx open source libraries. In mobile we found the google voice recognition services more efficient.

Benefits: - Better dictation.- Noise avoidance.- Multiple suggestions.- Flexible data structure.

4.3 Use case Diagrams

11 | P a g e

4.3.1 Android Application

4.3.2 PC Application

4.4 Project Flow diagrams

12 | P a g e

4.4.1 Android Application

4.4.2 PC Application

13 | P a g e

4.5 Data Flow Diagram

14 | P a g e

Java Server Application for your Windows PC/Laptop:

15 | P a g e

Second file is for PC which works as server in the Mantra System. “Mantra.bat” is the file name.

Step 1:

It is a bat file. You have to double click on the file and after that a windows will pop-up at the top-right corner of your desktop.

(Maybe you will see some images of command prompt flashing. You don’t have to worry about that as this is all because of the “bat” file.)

16 | P a g e

Figure 1

Here is the sample of what your desktop will look like:

Now as you can see after running “Mantra.bat” file u have this interface at your PC screen:

At this point of time the server is not running so u r getting server’s IP as 0.0.0.0 and no client is connected so blank space.

17 | P a g e

Step 2:

There is a button named “Start Server” on the Mantra’s window.

Click on that to start the server.

After Clicking on start server u will have the following interface:

When you get this screen that’s mean your server is running.

You will get your Server’s IP which u can use in the android application.

18 | P a g e

Figure 2

Step 3:

As you send commands from your android device to the PC, a client-server connection will be established and after that u will get the Client’s IP which is connected with the server.

After all this finally while running your desktop Java Mantra Server application will look like this:

Extras:

You can minimize the window. On click red-cross the server will stop working. If you only want to stop the server not the application just click on

stop server.

19 | P a g e

Implementation:

Following are the methods and logic behind all this functioning including client-server connection, receiving commands from android devices, execution of these commands on the operating system.

Connection between Android Device and Windows System:

Socket programming is the technology used to connect the android and the java device.

The endpoint in an inter-process communication is called a socket, or a network socket for disambiguation. Since most communication between computers is based on the Internet Protocol, an almost equivalent term is Internet socket. The data transmission between two sockets is organized by communications protocols, usually implemented in the operating system of the participating computers. Application programs write to and read from these sockets. Therefore, network programming is essentially socket programming. (Source: Wikipedia)

Here android device is working as client and the desktop java application is working as the server.

20 | P a g e

Server Side Socket programming:

This is the code which is working behind the server side socket programming.

Following are the important points about the code:

Pass only port address to get Server Socket. Using server socket, if the IP and Port address of a request is valid,

accept Client’s socket.

21 | P a g e

Client Side Socket Programming:

This is the code which is working behind the socket programming for client side.

Following are the points regarding the code:

InetAddress’ instance define the Server’s IP. Port address is fixed right now (its 8080). Both addresses are passed in Socket constructor to get

connectivity.

22 | P a g e

Sending and Receiving commands:

For the communication between the android and windows devices we used the above sockets to send and receive messages.

Messaging at Client Side:

This is the code which is working behind the sending messages for client to server.

Following are the points regarding the code:

To send message at server side using server socket get the output stream.

Using buffer writer and print writer, for easy writing, we send the message to the server side.

23 | P a g e

Messaging at Server Side:

Here is the code working behind the receiving messages from the client.

Following are the some points regarding the code:

Using client socket first get the input stream (Standard IO). Using input stream to read data as per our requirement we used

buffer reader.

24 | P a g e

TestingIntroduction:

Software testing has a dual function, it is used to establish the presence of defects in program and it is used to help judge whether or not the program is usable in practice. Thus software testing is used for validation and verification, which ensure that software confirms to its specification and meets the need of the software customer.

Developer resorted Alpha testing, which usually comes in after the basic design of the program has been completed. The project scientist will look over the program and give suggestions and ideas to improve or to correct the design. They also report and give ideas to get rid off around any major problems. There is bound to be a number of bugs after a program have been created.

Analyze and check system representation such as the requirements document, design diagram and the program source code. They may be applied at all stages of the process.

25 | P a g e

Testing Plan:

Analysis and design department makes all analysis for the system and forward the test cases flow of the system and scope of the system to development department.

Development department implements all the forms and sends to QA department for testing.

QA department checks the form for test cases and also performs integrated testing. If any error or bug found it returns to development department otherwise sends to analysis department.

Development department receives bug reports and after completing currently running modules solves those bug reports.

Analysis department receives error free forms and stores it permanent.

After completing form level testing system integrity testing starts. Thus the system is tested per cycle and then it is developed

further.

26 | P a g e

Testing Strategy:

The development process repeats this testing sub-process a number of times for the following phases.

Unit Testing. Integration Testing. System Testing. Acceptance Testing.

27 | P a g e

Test Cases:

Following are the simple test cases from the bunch of the test cases prepared for the testing of the system. Using these cases we can ensure that the system is bug and error free. These test were carried out with the sample data.

Case No. 1Name: Voice to text in Desktop Java Application-PassDescription: The test will check the code converting the voice

commands into text.Test Data Used: Laptop’s Microphone and voice commandsExpected Output: String of the voice command.Actual Output: String of the voice command.Pass/Fail PASS

Case No. 2Name: Voice to text in Desktop Java Application-FailDescription: The test will check the code converting the voice

commands into text.Test Data Used: Laptop’s Microphone and voice commandsExpected Output: Give the best match case and continue listening.Actual Output: Text somewhat similar to the voice command.Pass/Fail PASS

28 | P a g e

Case No. 3Name: Voice to text in Android Application-PassDescription: The test will check the code converting the voice

commands into text.Test Data Used: Mobile’s Microphone and voice commandsExpected Output: String of the voice command.Actual Output: String of the voice command.Pass/Fail PASS

Case No. 4Name: Voice to text in Android Application-FailDescription: The test will check the code converting the voice

commands into text.Test Data Used: Mobile’s Microphone and Voice commandsExpected Output: Give the best match case and continue listening.Actual Output: Text somewhat similar to the voice command.Pass/Fail PASS

Case No. 5Name: Connection between Mobile and PCDescription: The test will check whether PC and mobile are

getting connected with each other or not.Test Data Used: IP address of the server.Expected Output: “Hi” text at server side.Actual Output: “Hi” received at server.Pass/Fail PASS

29 | P a g e

Case No. 6Name: Final Testing, the complete working.Description: The test will check the complete working of the

project i.e. Receiving voice commands and converting it into text at Client (Android Mobile), Sending the text to the PC and execution of the command.

Test Data Used: Voice commands.Expected Output: Execution of the Command at Server Side.Actual Output: Command Executed at Server Side.Pass/Fail PASS

30 | P a g e

APPENDIX

Tools Used:

Eclipse. NetBeans. Sphinix Voice to Text Libraries. Google Voice to Text Libraries.

31 | P a g e