Deep Typechecking and Refactoring Zachary Tatlock, Chris Tucker, David Shuffleton, Ranjit Jhala,...

1

Deep Typecheckingand Refactoring

Zachary Tatlock, Chris Tucker, David Shuffleton, Ranjit Jhala, Sorin Lerner

University of California, San Diego

3

Communicating with Databases

String based queries are prevalent:JPA, Hibernate, TopLink

JAVA DBStrings

4

Example: Using JPA to query DB

“SELECT w FROM Weblog w WHERE w.id = ?1 AND w.link.id = ?2”

Query in JPA Query Language:

Mapping Java Classes to DB Tables: Expressed in Object Relational Mapping (ORM)

Java syntax in query String

5

Example: Using JPA to query DBString getText(String id, Link link) { String qStr; Query q;

qStr = “SELECT w FROM Weblog w ”; qStr += “WHERE w.id = ?1 ”; qStr += “AND w.link.id = ?2”;

q = createQuery(qStr);

q.setParam(1, id); q.setParam(2, link.id);

Weblog w = (Weblog) q.execQuery();

return w.text;}

6






return w.text;}

1. Build query string

7






return w.text;}


2. Create query

8






return w.text;}


2. Create query

3. Set parameters

9






return w.text;}


2. Create query

3. Set parameters

10






return w.text;}


2. Create query

3. Set parameters

4. Execute query

11






return w.text;}


2. Create query

3. Set parameters

4. Execute query

• Efficient• Flexible

Unsafe

12

Uncaught ErrorsString getText(String id, Link link) { String qStr; Query q;





return w.text;}

13




q.setParam(1, id); // q.setParam(2, link.id);


return w.text;}

14




q.setParam(1, id); // q.setParam(2, link.id);


return w.text;}

1. Unset parameter

15






return w.text;}

1. Unset parameter

16




q.setParam(1, new Weblog()); q.setParam(2, link.id);


return w.text;}

1. Unset parameter

2. Unsafe param type

17






return w.text;}

1. Unset parameter


18





Warranty w = (Warranty) q.execQuery();

return w.text;}

1. Unset parameter


3. Unsafe downcast

19






return w.text;}

1. Unset parameter


3. Unsafe downcast

Java compiler does not reason about the query strings; cannot typecheck.

20

Refactor: Weblog.id Weblog.nameString getText(String id, Link link) { String qStr; Query q;





return w.text;}

1. Unset parameter


3. Unsafe downcast

21






return w.text;}

1. Unset parameter


3. Unsafe downcast

Refactor

Don’t Refactor

Need to know type of w and w.link to

refactor safely.

22






return w.text;}

1. Unset parameter


3. Unsafe downcast

4. Refactoring difficult

23

String Based Query ChallengesString getText(String id, Link link) { String qStr; Query q;





return w.text;}

1. Unset parameter


3. Unsafe downcast


24

Contributions

Deep Typechecking: All parameters set Parameters correctly set Results safely downcast

Deep Refactoring: Builds on Deep Typechecking Enables class and field renaming

1. Unset parameter


3. Unsafe downcast


25

Outline

1. Introduction and Motivation2. Deep Typechecking and Refactoring

Straight Line Code Control Flow Multiple Methods

3. Experimental Results4. Related Work and Conclusion

26

Deep Typechecking ExampleString getText(String id, Link link) { String qStr; Query q;





return w.text;}

Query safety:

• All params set

• Params safely set

• Result safely downcast

Is this query exec safe?

Need to know:

1. query string

2. param types

27

Deep Typechecking ExampleString getText(String id, Link link) { String qStr; Query q;





return w.text;}

Bound Query:• query string• param types

query : “SELECT … ?1 … ?2 … ?3 …”?1 : String?2 : Weblog?3 : unknown

Example :

At each program point map each var to a set of BQs.

28

Bound Query AnalysisString getText(String id, Link link) { String qStr; Query q;





return w.text;}

29






return w.text;}

qStr = “SELECT … ?2”

30






return w.text;}


31






return w.text;}


32






return w.text;}


query : “SELECT …”?1 : unknown?2 : unknown

33






return w.text;}



query : “SELECT …”?1 : String?2 : unknown

34






return w.text;}



query : “SELECT …”?1 : String?2 : unknown

query : “SELECT …”?1 : String?2 : int

35






return w.text;}

Checking param types:

1. Parse query string

2. Check all params set

3. Check param types

Bound Queries:


36






return w.text;}

Checking result type:

1. Infer result type

2. Propagate result type

3. Check downcasts

Bound Queries:


query : “SELECT …”?1 : String?2 : intresult : Weblog

37






return w.text;}

If a query passes Deep Typechecking, then it will not cause an error at runtime.

Therefore, Bound Query Analysis has no silent failures.

39

Deep Refactoring ExampleString getText(String id, Link link) { String qStr; Query q;





return w.text;}

40






return w.text;}

Refactor Weblog field:

id name

41






return w.text;}


id name

42






return w.text;}


id name

43

Deep Refactoring ExampleRefactor Weblog field:

id name

String getText(String id, Link link) { String qStr; Query q;





return w.text;}

query :

SELECT w FROM Weblog w WHERE w.id = ?1 AND w.link.id = ?2?1 : String?2 : int

44






return w.text;}

query :

SELECT w FROM Weblog w WHERE w.id = ?1 AND w.link.id = ?2?1 : String?2 : int


id name

1. Refactor full query

45






return w.text;}

query :

SELECT w FROM Weblog w WHERE w.name = ?1 AND w.link.id = ?2?1 : String?2 : int


id name


46






return w.text;}

query :



id name


2. Propagate changes

47


qStr = “SELECT w FROM Weblog w ”; qStr += “WHERE w.name = ?1 ”; qStr += “AND w.link.id = ?2”;




return w.text;}

query :



id name


2. Propagate changes

48

Outline




49

Flow SensitivityString getText(String id, Link link) { String qStr; Query q;





return w.text;}

50

Flow SensitivityString getText(String id, Link link) { String qStr; Query q; qStr = “SELECT w FROM Weblog w ”; qStr += “WHERE w.id = ?1 ”;

qStr += “AND w.link.id = ?2”; q = createQuery(qStr); q.setParam(1, id); q.setParam(2, link.id);

Weblog w = (Weblog) q.execQuery(); return w.text;}

51


qStr += “AND w.link.id = ?2”; q = createQuery(qStr); q.setParam(1, id); q.setParam(2, link.id);


52


if(link != null) { qStr += “AND w.link.id = ?2”; q = createQuery(qStr); q.setParam(1, id); q.setParam(2, link.id); } else {

}


53


if(link != null) { qStr += “AND w.link.id = ?2”; q = createQuery(qStr); q.setParam(1, id); q.setParam(2, link.id); } else { q = createQuery(qStr); q.setParam(1, id); }


54




55





56




query : “SEL … ?2”?1 : String?2 : int


57







58







59







60





query : “SEL … ?1”?1 : String



61




query : “SEL … ?1”?1 : String

“SEL … ?2”?1 : String?2 : int

“SEL … ?1”?1 : String




62




“SEL … ?2”?1 : String?2 : int

“SEL … ?1”?1 : String

As before, for each bound query:

1. Check param types

2. Check result type

In general, we express Bound Query Analysis as a dataflow analysis.

64

LoopsString getText(String id, Link link) { String qStr; Query q;

qStr = “SELECT w FROM Weblog w ”; qStr += “WHERE w.id = ?1 ”; qStr += “OR w.link.id = ?2”;




return w.text;}

65

LoopsString getText(String id, List<Link> links) { String qStr; Query q;





return w.text;}

66






return w.text;}

67


qStr = “SELECT w FROM Weblog w ”; qStr += “WHERE w.id = ?1 ”;

for(int i = 0; i < links.size(); i++) { qStr += “ OR w.link.id = ?” + i; }


...}

68

String getText(String id, List<Link> links) { String qStr; Query q;




...}

Loops

qStr = ???qStr = “SELECT … w.id = ?1” ( “ OR w.link.id = ?#” )*

69





...}

Loops

qStr = “SELECT … w.id = ?1” ( “ OR w.link.id = ?#” )*

70





...}

Loops

qStr = “SELECT … w.id = ?1” ( “ OR w.link.id = ?#” )*

Deep Refactoring:• know query structure• know fragment locs• refactor across loops

Deep Typechecking:• unknown # of params• do not check params• can still check result

71

Outline




72

Multiple MethodsString mainQueryStr() { return “SELECT ... ?1”;}

Object getMain() { String qStr = mainQueryStr();

Query q = createQuery(qStr);

q.setParam(1, “main”);

return q.execQuery();}

String mainId() { return ((Weblog) getMain()).id;}

String Analysis : • interprocedural• compute regexps

Bound Query Analysis : • intraprocedural• no complex aliasing

Result Analysis :• interprocedural• propagate result type

73

Outline




74

Results Implementation: Quail

5700 lines of Java in Eclipse plugin

Benchmarks:Application LOC DescriptionRoller 82,900 Blogging frameworkPlanet 14,500 News aggregatorPetStore 5,440 Example JPA applicationCaveatEmptor 3,370 Online auction systemTotal 106,000

Create Query Set Param Exec Query163 199 136

JPA Calls:

Tested,Stable,

Deployed

75

Deep Typechecking ExperimentCheck for all query execution sites:

All parameters set Parameters safely set Query results safely downcast

Show 84% of query executions are safe

Remaining 16% due to imprecision: Queries built with loops or data structures Path sensitivity Reflection

76

Deep Refactoring Experiment Rename 16 most frequently appearing classes / fields Correctly refactor 98% of refactorable locations Remaining 2% from heap based strings in query

String[] blacklist = getBlacklist();

String qStr = “SELECT w FROM Weblog w WHERE w.id = ?1 ”;

for(int i = 0; i < blacklist.length; i++) {

qStr += “AND w.id != ” + blacklist[i] + “ ”;

}

...

77

Results

Typechecking and Refactoring Experiments: No silent failures Show user exactly where to check 2% vs 16% imprecision:

Refactoring does not depend on parameter types

78

Outline


Straight Line Code Control Flow Mutliple Methods


79

Related Work

Query Extraction [Wiedermann, Ibrahim, Cook 08] Infer query from Java code that uses DB root object Not applicable to legacy code

Static Query Checking [Gould, Su, Devanbu 04] Typechecks JDBC queries against DB schema Not applied to source language type system

80

Related Work

Language Based [Matthes 95, Schmidt 94] Syntax extension for queries Not applicable to legacy code

Orthogonal Persistence [Atkinson 96, Liskov 96] Map DB to collection of persisted objects Must express queries in non-query language

81

Conclusion

String based queries can be safe and flexible.

Deep Typechecking ensures: All parameters are set Parameters are set to correct type Query results are safely downcast

Deep Refactoring enables: Class and field renaming

82

Thank You

Deep Typechecking and Refactoring Zachary Tatlock, Chris Tucker, David Shuffleton, Ranjit Jhala,...

Documents

Transcript of Deep Typechecking and Refactoring Zachary Tatlock, Chris Tucker, David Shuffleton, Ranjit Jhala,...