A Framework for A Framework for Testing Database Testing Database

ApplicationApplication

Author: Author: David Chays, Saikat Dan, David Chays, Saikat Dan, Filippos I. Vokolos, Elaine J. WeyukerFilippos I. Vokolos, Elaine J. Weyuker

Presenter: Liping LiuPresenter: Liping Liu

OutlineOutline• Motivation• Background• Issues in testing DB application• Testing tool• Conclusion

MotivationMotivation• Important role of DB system• DB application program in semi-

declarative language; many testing techniques for programs in imperative language

• Different input and output space of DB application

BackgroundBackground• Focus on Relational DB and SQL• Data is viewed as a collection of relations

relation schema relation (relation state): element of Cartesian product

• Tables, tuples, attributes, constrains• Constrain:

Domain Constrain Uniqueness Constrain not-NULL Constrain Referential Integrity (foreign key) Semantic Integrity (general)

Correctness (def)Correctness (def)• Does the DBMS perform all

operations correctly?• Protect security and privacy?• Is the system fault-tolerant?• ...

Does the application program behave as intended?

Issues in DB applicationIssues in DB application

• traditional• imperative nature

• DB• declarative nature

input

output

input DB state

output DB state

EExample:xample:• Customer-feature table• Customer ID, feature name• Customer ID invalid-> return 0;• Feature invalid-> return 0;• ID and feature valid, feature available-

>return 1;• Feature not available->return 2;• New feature incompatible with old feature-

>return 3;

exampleexample• Naïve approach

I = {customer-IDs} X {feature-names}0 = {0,1,2,3}

• More suitable approach:I = {customer-IDs} X {feature-names}

X {database-states}0 = {0,1,2,3} X {database-states}

DB stateDB state• Ways to deal with DB state:

ignore as environment as part of input and output space

• Problems: Controllability: at desired state

before execution->populate DB with appropriate data

Observability: observe the state after execution->check the state of DB after execution

DB stateDB state• How to obtain DB state• Live data

not reflect sufficiently wide variety of situations

difficult to find the situations of interest violate privacy or security constraints

• Synthetic dataGenerating domain elements and gluing them

togethergenerate interesting data that obey integrity

constraintsUse schema and user supplied info

Suggestionsfrom tester

DB schema

App source

App exec

User input

Output

DB state

Results

Input Generator

State Generator

State Validator

Output Verifier

Log file

ToolTool• State Generator

Input: DB schema, user info (suggested value)Populate DB state

• Input Generator Input: info from State Generator, info of queryGenerate input data for test cases

• Output Verifier Input: output of test caseCheck with expected value

• State ValidatorCheck state after execution

State GeneratorState Generator• Inputs DB schema (in SQL)• Parses schema to derive info about

AttributesTablesconstraints : uniqueness, not-NULL,

referential integrity inputs additional info from user suggested attribute values, divided into

groups, similar to Category-Partition Testing

P | 5 |pname | F| F| F| F| F| F| F| pno | F| F| F| F| F| F| F|

weight| F| F| F| F| F| F| F| color | F| F| F| F| F| F| F|

0

1

2

3

4 city | P | char | ~pr | ~un | ~nn

pname | P | char | ~pr | ~un | ~nn pno | P | char | pr | un | ~nn

weight | P | dec | ~pr | ~un | ~nn color | P | char | ~pr | ~un | ~nn

0

1

2

3

cp

cpcp

cpcp

S | 4 |

globalTablePointer sname | F| F| F| F| F| F| F|

sno | F| F| F| F| F| F| F|

City | F| F| F| F| F| F| F| status | F| F| F| F| F| F| F|

0

1

2

3

sname | S | char | ~pr | ~un | ~nn sno | F| F| F| F| F| F| F|

City | F| F| F| F| F| F| F| status | F| F| F| F| F| F| F|

0

1

2

3

sno | S | char | pr | un | ~nn

city | S | char | ~pr | ~un | ~nn status | S | dec | ~pr | ~un | ~nn

0

1

2

3

cp

cp

cpcp

SP | 3 |

Null pno |SP | char | pr | un | ~nn | foreign

sno |SP | char | pr | un | ~nn | foreign

qty |SP | dec | ~pr | ~un | ~nn

0

1

2

cpcpcp

Data structureData structure

S: supplier; P: partS: supplier; P: part

• Category Partition

Each category (column) can have a list of choices pointed to by cp.

cp low highmedium

10

20

30

300

400

5000

6000

Populated DB tablePopulated DB table• Tester specifies table sizes• Tool generates tuples for insertion

– select data group or NULL, guided by annotations

– select value from data group, obeying constraints

– keep track of values used• Outputs sequence of SQL insert

statements

EExample xample

sno pno qtyS1 P1 5000S1 P2 300S1 P3 10S2 P1 6000S2 P2 400S2 P3 5000S3 P1 20S3 P2 300S3 P3 30S4 P1 6000

pno pname color weight cityP1 NULL blue 100 BrooklynP2 Seats green 300 Florham-ParkP3 airbags yellow 500 Middletown

sno sname status cityS1 NULL 0 BrooklynS2 Smith 1 Florham-ParkS3 Jones NULL LondonS4 Blake NULL Middletown

Table s Table sp

Table p

ConclusionConclusion• Issues in testing DB application• Framework: DB state• Testing tool: focus on the state

generator

Future workFuture work• Future work: An GENDA for testing

relational database applications• 5 components: AGENDA Parser• Data structure ->AGENDA DB; avoids

memory issue; more accessible• Interested?

http://cis.poly.edu/~ytdeng/paper/jstvr_2004.pdf

Thanks!