Testing Document - cs.bgu.ac.ilseproj/2006-2007/shaevich/Testing... · Web viewIn case of...

SmartSQLTesting Document

Testing Document

Faculty of Engineering ScienceSoftware Engineering Program

Fourth Year Engineering Project

SmartSQL

SmartSQL1


Contents

1. Testing Functional Requirements..........................................................4 1.1.....................................................................Generating SQL query from NL query.

4 1.2......In case of misunderstanding SmartSQL will ask the user for clarification

with alternative word suggestions.........................................................................6 1.3. Generation of glossaries for NL analyzing process:.....................................6 1.3.1. DB schema............................................................................................6 1.3.2. DB connection......................................................................................7 1.3.3. Table conversion....................................................................................7 1.3.4. Field conversion.....................................................................................8 1.3.5. DB fields values......................................................................................8 1.3.6. Keywords db...........................................................................................9 1.3.7. Query type db.......................................................................................10 1.4. Integration of glossaries for the NL analyzing process................................10 1.5............................................... “SQLBuilder” - Building SQL queries using GUI.

11 1.6............ Extracting and Presentation of Information in case of “Select” query.

11 1.7............................................................................................ Login and access control.

12 1.8.................................................... Adding/Removing database to/from the system.

12 1.9...................................................................................... Setting of working database.

12 1.10. Add/Remove users to the system...................................................................13 1.11...................................................................................Saving NL (executed) queries.

13 1.12..............................................................................................Open saved NL queries.

132. Testing Non-functional Requirements...................................................................14 2.1. System speed......................................................................................................14

2.2. Safety & Security..............................................................................................142.3. Usability.............................................................................................................142.4. Availability.........................................................................................................142.5. Portability..........................................................................................................14

3. Test-Driven Development..........................................................................144. Random & Automatically-Generated Tests.....................................14

SmartSQL2


5. Testing The User Interface.....................................................................................156. Testing Build, Integration & Deployment.............................................................15

SmartSQL3


1.Testing Functional Requirements.

1.1.Generating SQL query from NL query.No

Test case Test Description

Expected Result Test Status

ARD Ref.

1 Testing of “simple Select” Queries.

Input a NL query that doesn’t require “join” and “where” operations and doesn’t specify any explicit fields to view.

A “Select * From (Table-name)” query is generated.

Passed 1,3,4,6,7,15b

2 Input a NL query that doesn’t require “join” and “where” operations but there are explicit fields to be shown.

Select (field-name1 , field-name2,…) FROM (Table-name)

Passed 1,3,4,6,7,15b

3 Input a NL query that requires a simple join between two tables.

Select * FROM (Table-name1) INNER JOIN (Table-name2) ON ( Table-name1.field-name = Table-name2.field-name )

Passed 1,3,4,6,7,15b

4 Input a NL query that requires join between two tables through a transaction table.

Select * FROM ( Table-name1 INNER JOIN Transaction-table ON Table-name1.field-name = Transaction-table.field-name ) INNER JOIN Table-name2 ON ( Transaction-table.field-name = Table-name2.field-name )

Passed 1,3,4,6,7,15b

5 Testing of “where” Queries.

Input a simple query that requires the “where”

Select * FROM Table-name1 WHERE Table-

Passed 1,3,4,6,7,15b

SmartSQL4


operation. name1.field-name < num

6 Input a NL query that requires AND in the “where” operation. The “where” operation consists of the same fields.

Select * FROM Table-nameWHERE Table-name.field-name > num1 AND Table-name.field-name < num2

Passed 1,3,4,6,7,15b

7 Input a NL query that requires AND in the “where” operation. The “where” operation consists of different fields.

Select * FROM Table-nameWHERE Table-name.field-name1 < num1 ANDTable-name.field-name2 > num2

Passed 1,3,4,6,7,15b

8 Input a NL query with “where” operation that includes a value

Select * FROM Table-name WHERE Table-name.field-name = 'field-name-value'

Passed 1,3,4,6,7,15b

9 Testing of “order by” Queries.

Input a simple NL query with sort operation.

Select * FROM Table-nameORDER BY Table-name.field-name

Passed 1,3,4,6,7,15b

10 Input a NL query with sort operation and Desc request.

Select * FROM Table-name ORDER BY Table-name.field-name DESC

Passed 1,3,4,6,7,15b

11 Input NL query with sort and “where” operations that refer to the same field.

Select * FROM Table-nameWHERE Table-name.field-name > num ORDER BY Table-name.field-name

Passed 1,3,4,6,7,15b

12 Input NL query with sort and “where” operations that refer to different fields.

Select * FROM Table-nameWHERE Table-name.field-name1 > numORDER BY Table-

Passed 1,3,4,6,7,15b

SmartSQL5


name.field-name2 DESC

13 Testing of “count” Queries.

Input a “count” NL query.

Select Table-name1.field-name1,COUNT(Table-name1.field-name1) FROM Table-name2 INNER JOIN Table-name1 ON (Table-name2.field-name2 = Table-name1.field-name2 ) GROUP BY Table-name1.field-name1

Passed 1,3,4,6,7,15b

14 Input a “count” all NL query.

Select Count(*) FROM Table-name

Passed 1,3,4,6,7,15b

15 Input a “count” NL query with “group by” operation.

Select Table-name.field-name, COUNT(Table-name.field-name) FROM Table-name GROUP BY Table-name.field-name

Passed 1,3,4,6,7,15b

16 Testing of “average” Queries

Input a simple “average” NL query.

Select AVG(Table-name.field-name) FROM Table-name

Passed 1,3,4,6,7,15b

17 Input an “average” NL query which includes “join” between two tables, and “goup by” operations.

Select Table-name1.field-name1,AVG(Table-name2.field-name2) FROM Table-name2 INNER JOIN Table-name1 ON ( Table-name2. field-name2= Table-name1. field-name1) GROUP BY Table-name1. field-name1

Passed 1,3,4,6,7,15b

18 Testing of “max/min” Queries

Input a simple NL query with “max” operation.

Select MAX(Table-name.field-name) FROM Table-name

Passed 1,3,4,6,7,15b

19 Input NL query Select Table- Passed 1,3,4,6,7,

SmartSQL6


with “min”, “join” between two tables and “group by” operations.

name1.field-name1,MIN(Table-name2.field-name2) FROM Table-name2INNER JOIN Table-name1 ON (Table-name2.field-name3 = Table-name1.field-name4 ) GROUP BY Table-name1.field-name1

15b

20 Testing of “sum”

Queries

Input a simple NL query with “sum” operation.

Select SUM(Table-name.field-name) FROM Table-name

Passed 1,3,4,6,7,15b

21 Input NL query with “sum”, “join” between two tables and “group by” operations.

Select Table-name1.field-name1,SUM(Table-name2.field-name2) FROM Table-name2 INNER JOIN Table-name1ON (Table-name2.field-name3 = Table-name1.field-name4 ) GROUP BY Table-name1.field-name1

Passed 1,3,4,6,7,15b

22 Testing of “Insert” Queries

Input NL query of “Insert” type. Should be of the following format:“Insert into table-name field-name1,field-name2 values val1,val2”.

Insert INTO Table-name(field-name1, field-name2) VALUES (val1,’val2’)

Passed 1,3,4,6,7,15b

23 Testing of “Update” Queries

Input NL query of “Update” type. Should be of the following format:Update Table-name set field-name1 = val1,field-name2 = val2 when field-name1 equal val3.

Update Table-name1 Set field-name1=val1, field-name2=val2 WHERE field-name1 =val3

Passed 1,3,4,6,7,15b

SmartSQL7


24 Testing of “Delete” Queries

Input NL query of “Delete” type. Should be of the following format:Erase from Table-name when field-name equal val.

Delete FROM Table-nameWHERE field-name = val

Passed 1,3,4,6,7,15b

1.2.In case of misunderstanding SmartSQL will ask the user for clarification with alternative word suggestions.

No Test case Test Description

Expected Result

Test Status

ARD Ref.

1 Input query with multiple meanings of a certain word.

Input query:“Give me names.”

The system will return list of fields which corresponds to the word “names” and the user can choose one of them.

Passed 2,15b

2 Input “bad” NL query

Input query:“Give me mames.”

The system will return message, that given NL query is not valid.

Passed 2,15b

1.3.Generation of glossaries for NL analyzing process:1.3.1. DB schema- fields and tables of the

database.No Test case Test

DescriptionExpected

ResultTest

StatusARD Ref.

1 Testing of creation DB scheme

Choose DB, update XML of database structure.

The system will create (or update) XML document of table and field names.

Passed 16a

2 Testing of Disconnect DB from The system will Passed 16a

SmartSQL8


creation DB scheme: connection failed

the server, choose DB and update XML of database structure.

return message, that it couldn’t connect to the DB.

SmartSQL9


1.3.2. DB connection-connections between tables.


Expected Result

Test Status

ARD Ref.

1 Testing of creation DB connection

Choose DB, update XML of database connection.

The system will create (or update) XML document of table connection.

Passed 16b

2 Testing of creation DB connection: connection failed.

Disconnect DB from the server, choose DB and update XML of database connection.

The system will return message, that can’t connect to the DB.

Passed 16b

1.3.3. Table conversion- list of words that user will use for the given table of the database.


Expected Result

Test Status

ARD Ref.

1 Testing addition of alternative to table name

Add alternative to the specified table, save changes, reread alternatives for this table.

The added alternative will be in the returned list

Passed 16c

2 Testing removal of alternative from table name.

Remove existing alternative from the table’s alternatives list, save changes, reread table’s alternatives.

Deleted alternative shouldn’t be in the returned list of alternatives.

Passed 16c

SmartSQL10


1.3.4. Field conversion- list of words that user will use for the given field of the database.


Expected Result

Test Status

ARD Ref.

1 Testing addition of alternative to field name

Add alternative to the specified field, save changes, reread alternatives for this field.

The added alternative will be in the returned list.

Passed 16d

2 Testing removal of alternative from field name.

Remove existing alternative from the field’s alternatives list, save changes, reread field’s alternatives.

Deleted alternative shouldn’t be in the returned list of alternatives.

Passed 16d

1.3.5. DB fields values - will hold possible values for the given field in order to recognize field by the given value during NL analyzing process, only relevant for fields with close range of values.


Expected Result

Test Status

ARD Ref.

1 Testing addition of new value to the field.

Add new value to the field’s values, save changes, reread field’s values.

The returned list should contain added value.

Passed 16e

2 Testing removal of existing field’s value.

Remove existing value from the field’s values list, save changes, reread field’s values

The returned list shouldn’t contain removed value.

Passed 16e

SmartSQL11


3 Testing getting list of unique values for specified field.

For specific field with known list of unique values, get the list from DB.

The returned list should be equal to the known list.

Passed 16e

SmartSQL12


1.3.6. Keywords db - holds SQL query constants that common to all databases, for example: or, and, count, sum. Will be build manually by administrator of the system using GUI.


Expected Result

Test Status

ARD Ref.

1 Testing addition of new keyword.

Add new keyword to the list of keywords, reread keyword’s list


Passed 16f

2 Testing removal of existing keyword

Remove existing keyword from keyword’s list, reread the list.


Passed 16f

3 Testing addition of new value to the keyword.

Add new value to the given keyword; reread the list of values for this keyword.


Passed 16f

4 Testing removal of existing value from keyword.

Remove existing value from given keyword’s values, reread the list of values for this keyword.


Passed 16f

SmartSQL13


1.3.7. Query type db- holds list of words which represents different query types, for example: Select, Insert, etc. Will be build manually by administrator of the system using GUI.


Expected Result

Test Status

ARD Ref.

1 Testing addition of new value to specified query type

Add new value to the given query type; reread the list of values for this query type.


Passed 16g

2 Testing removal of existing value from given query type’s values.

Remove existing value from the list of query type’s values; reread the list of values for this query type.


Passed 16g

1.4.Integration of glossaries for the NL analyzing process.


Expected Result Test Status

ARD Ref.

1 Test of “Table conversion” glossary.

Input a sentence with word which associated with a certain table.

The system will return a table or a list of tables which corresponds to the word which is a part of the “Table conversion” glossary and the user can choose one of them.

Passed 3,15a

2 Test of “Field conversion” glossary.

Input a sentence with word which associated with a certain field.

The system will return a field or a list of fields which corresponds to the word which is a part of the “Field conversion” glossary and the user can choose one of them.

Passed 3,15a

3 Test of “DB Values” glossary.

Input a sentence with word which is a value of a certain field.

The system will find and assign the value to its corresponding field.

Passed 3,15a

Note: Rest of the glossaries is checked during “section 1.1” tests.

SmartSQL14


1.5.“SQLBuilder” - Building SQL queries using GUI.


Expected Result

Test Status

ARD Ref.

1 Testing “Select” queries.

Testing all GUI and query variations

Correct SQL query

Passed 12,15c

2 Testing “Insert” queries.


Correct SQL query

Passed 12,15c

3 Testing “Update” queries.


Correct SQL query

Passed 12,15c

4 Testing “Delete” queries.


Correct SQL query

Passed 12,15c

5 Testing “Create” queries.


Correct SQL query

Passed 12,15c

6 Testing “Alter” queries.


Correct SQL query

Passed 12,15c

7 Testing “Drop” queries.


Correct SQL query

Passed 12,15c

1.6.Extracting and Presentation of Information in case of “Select” query.

No Test case Test Descriptio

n

Expected Result

Test Statu

s

ARD Ref.

1 Test in NL mode.

Input correct SQL query.

Correct SQL query generated and corresponding data is presented to the user.

Passed 12,15c

2 Test in “SQLBuilder”

Input correct SQL query.

Correct SQL query generated and

Passed 12,15c

SmartSQL15


mode. corresponding data is presented to the user.

SmartSQL16


1.7.Login and access control.


Expected Result

Test Status

ARD Ref.

1 Testing login with correct data.

Insert correct Login and Password.

The system will permit to user work with the system and show main window.

Passed 15d

2 Testing login with incorrect data.

Insert incorrect Login and Password.

The system will return message, that user can’t login and that the login data was incorrect.

Passed 15d

1.8.Adding/Removing database to/from the system.

No Test case Test Description Expected Result

Test Status

ARD Ref.

1 Testing of Adding Database: “Access”.

Choose Database to be added (“Access” type). Add this DB to the system.

Database will be added to the current DB’s list.

Passed 9,15a

2 Testing of Adding Database: “SQL Server”.

Choose Database to be added (“SQL Server” type). Add this DB to the system.

Database will be added to the current DB’s list.

Passed 9,15a

1.9.Setting of working database.No Test case Test

DescriptionExpected

ResultTest

StatusARD Ref.

1 Testing of changing working DB.

Choose database from current DB’s list (combo box).

The system will change working database.

Passed 10,15a

SmartSQL17


1.10. Add/Remove users to the system.No Test case Test

DescriptionExpected

ResultTest Statu

s

ARD Ref.

1 Testing of Adding new user to the system.

Insert login, password, choose user role and confirm the password.

The system will add a new user to user list and add his data.

Passed 11,15a

2 Testing of Adding new administrator to the system.

Insert login, password, choose administrator role and confirm the password.

The system will add a new administrator to user list and add his data.

Passed 11,15a

3 Testing of Removing user from the system.

Choose a user to removing and remove this user.

The system will remove the user from user list and delete his data.

Passed 11,15a

1.11. Saving NL (executed) queries.


Expected Result

Test Status

ARD Ref.

1 Test after queryExecution.

Saving NL query after its execution.

success Passed 13,15b

2 Test before queryExecution.

Saving NL before execution.

failed Passed 13,15b

1.12. Open saved NL queries.


Expected Result

Test Status

ARD Ref.

1 Open query Open saved NL query.

Corresponding SQL query is opened.

Passed 14,15b

SmartSQL18


2.Testing Non-functional Requirements

2.1. System speed

Checking manually that the system responds within 2 seconds to NL query input.

2.2. Safety & Security

The system does not allow to access to unauthorized users. The test is performed manually and checks that the unauthorized user was unable to login. Only the “administrator” defined users can access the “Administrator” module.

2.3. Usability

Will be checked by default and naïve users that the system is simple to operate, and that can be learned within 30 minutes.

2.4. Availability

The system will run for one week without closing it and users during this time will use our system and we will see if the system is not crashing.

2.5. Portability

The system must work on Windows, thus the application was tested on Windows XP. The system is written in C# on .NET framework 2.0 platform therefore .NET framework 2.0 must be installed on the computer before running the application.

3. Test-Driven Development

We did not make use of the TDD development strategy because the requirements changed all the time.

SmartSQL19


4. Random & Automatically-Generated Tests

We did not perform random & automatically generated tests because in the NL module the system is interactive and users intervene needed during the analyzing process.

5. Testing The User Interface

User interface will be manually tested. The main goals of such tests are:

a. Check that all the GUI components are visible and exist.b. Check that all the GUI components are functioning well.c. Check that GUI is stable.

* In the GUI the input fields (textboxes) will be checked for valid input which is: all the digits and all letters small and capital.

There is no online help to the system but user manual will be provided.

6. Testing Build, Integration & Deployment

1. Installation process is made automatically and consists of the following steps:a. If .Net Framework 2.0 is not installed-installing .Net Framework 2.0b. If WordNet is not installed-installing WordNet.c. If Microsoft Speech SDK is not installed-installing Microsoft Speech SDK.d. Opening folder for the application and copying there all the needed files and

folders including the executable of the application.

Verification of the installation is made automatically which checks that all relevant software is installed and manually by trying to run the executable.

2. Uninstall process is made automatically and consist from:a. Uninstalling all the support software, this is done after user authorization

before each uninstall.b. Deletion of the application folder.

Verification of the uninstallation is made automatically which checks that all relevant software is uninstalled all the relevant folders deleted.

SmartSQL20


SmartSQL21

Testing Document - cs.bgu.ac.ilseproj/2006-2007/shaevich/Testing... · Web viewIn case of...

Documents

Transcript of Testing Document - cs.bgu.ac.ilseproj/2006-2007/shaevich/Testing... · Web viewIn case of...