ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial...
-
Upload
blackberry -
Category
Technology
-
view
180 -
download
0
Transcript of ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial...
![Page 1: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/1.jpg)
1
Detecting Problems in the Database Access Code of Large Scale Systems
An industrial Experience Report
Mohamed Nasser, Parminder Flora
Tse-Hsun(Peter) Chen Ahmed E. HassanWeiyi Shang
![Page 2: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/2.jpg)
Existing static analysis tools focus on language-related problems
2
Coverity PMD Google error-prone
Facebook InferFindBugs
However, many problems are related to how developers use different frameworks
![Page 3: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/3.jpg)
Over 67% of Java developers use Object-Relational Mapping (Hibernate) to access databases
3
Existing static analysis tools provide mostly rudimentary support for JDBC!
22%67%
![Page 4: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/4.jpg)
Over 40% of Java web application developers use Spring
4
Developers use Spring to manage database transactions in web applications
None of the static analysis tools support Spring!
![Page 5: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/5.jpg)
There is a huge need for framework-specific tools
5
Developers leverage MANY frameworks, but existing tools only support detecting language-related problems.
![Page 6: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/6.jpg)
An example class with Java ORM code
6
@Entity@Table(name = “user”)@DynamicUpdatepublic class User{
@Column(name=“id”)private int id;
@Column(name=“name”)String userName;
@OneToMany(fetch=FetchType.EAGER)List<Team> teams;public void setName(String n){
userName = n;}
… other getter and setter methods
User.javaUser class is
mapped to “user” table in DB
id is mapped to the column “id” in the
user table
A user can belong to multiple teams
Eagerly retrieve associated teams when retrieving a
user object
Performance-related configs
![Page 7: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/7.jpg)
Accessing the database using ORM
7
User u = findUserByID(1);
ORMdatabase
select u from user where u.id = 1;
u.setName(“Peter”);
update user set name=“Peter” where user.id = 1;
Objects SQLs
![Page 8: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/8.jpg)
Transaction management using Spring
8
@Transaction(Propogation.REQUIRED)getUser(){ … updateUserGroup(u) …}
By using ORM and Spring, developers can focus more on the business logic
and functionality
Create a DB transaction
Entire business logic will be executed with the same DB transaction
![Page 9: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/9.jpg)
Implementing DBChecker
9
Source code
• DBChecker looks for both functional and performance bug patterns
• DBChecker is integrated in industrial practice
![Page 10: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/10.jpg)
Overview of the presentation
10
Bug patterns Lessons learned when adopting the tool in practice
![Page 11: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/11.jpg)
Overview of the presentation
11
Bug patterns Lessons learned when adopting the tool in practice
More patterns and learned lessons in the paper
![Page 12: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/12.jpg)
ORM excessive data bug patternClass User{
@EAGERList<Team>
teams;}
User u = findUserById(1);u.getName();EOF
12
Objects
SQL
Eagerly retrieve teams from DB
User Table Team Table
join Team data is never used!
![Page 13: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/13.jpg)
Detecting excessive datausing static analysis
13
First find all the objects that eagerly retrieve data from DB
Class User{@EAGERList<Team>
teams;}
Identify all the data usages of ORM-managed objects
User user = findUserByID(1);
Check if the eagerly retrieved data is ever used
user.getName();
user team
user team
![Page 14: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/14.jpg)
Nested transaction bug pattern
14
@Transaction(Propogation.REQUIRED)getUser(){ updateUserGroup(u) …}
Create a DB transaction
@Transaction(Propogation.REQUIRES_NEW)
Create a child transaction, and suspend parent transaction until child is finished
Misconfigurations can cause unexpected transaction timeout, deadlock, or other
performance-related problems
![Page 15: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/15.jpg)
Detecting nested transaction bug pattern
15
@Transaction(Propogation.REQUIRED)getUser(){ … updateUserGroup(u) …}
Parse all transaction configurations
Identify all methods with the annotation
Propogation.REQUIRED
Propogation.REQUIRS_NEWcalls
Traverse the call graph to identify potential misconfigurations
![Page 16: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/16.jpg)
Limitation of current static analysis tools
16
Annotations are lost when converting source code to byte code
Do not consider how developers configure frameworks
@Transaction(Propogation.REQUIRED)@EAGER
Many problems are related to
framework configurations
Many configurations are
set through annotations
![Page 17: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/17.jpg)
Overview of the presentation
17
Bug patternsLessons learned when
adopting the tool in practice
Most discussed bug patterns are related to
incorrect usage of frameworks
![Page 18: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/18.jpg)
Overview of the presentation
18
Bug patternsLessons learned when
adopting the tool in practice
Most discussed bug patterns are related to
incorrect usage of frameworks
![Page 19: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/19.jpg)
Handling a large number of detection results
19
• Developers have limited time to fix detected problems
• Most existing static analysis frameworks do not prioritize the detected instances for the same bug pattern
![Page 20: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/20.jpg)
20
Prioritizing based on DB tablesUser
Time zone
• Problems related to large or frequently-accessed tables are ranked higher (more likely to be performance bottlenecks)
• Problems related to highly dependable tables are ranked higher
![Page 21: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/21.jpg)
Developers have different backgrounds
21
• Not all developers are familiar with these frameworks and databases
• Developers may not take the problems seriously if they don’t understand the impact
![Page 22: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/22.jpg)
Educating developers about the detected problems
22
• We hosted several workshops to educate developers about the impact and cause of the problems
• Walk developers through examples of detected problems
• May learn new bug patterns from developers
![Page 23: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/23.jpg)
Overview of the presentation
23
Bug patternsLessons learned when
adopting the tool in practice
Most discussed bug patterns are related to
incorrect usage of frameworks
We prioritize problems based on DB tables, and
educate developers about the problems
![Page 24: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/24.jpg)
24
![Page 25: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/25.jpg)
25
![Page 26: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/26.jpg)
26
![Page 27: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/27.jpg)
27
![Page 28: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/28.jpg)
28
![Page 29: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/29.jpg)
29
![Page 30: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/30.jpg)
30
![Page 31: ICSE2016 - Detecting Problems in Database Access Code of Large Scale Systems - An Industrial Experience Report](https://reader036.fdocuments.in/reader036/viewer/2022081605/58f0f6fa1a28ab4a3f8b4663/html5/thumbnails/31.jpg)
31
Tse-Hsun (Peter) Chen http://petertsehsun.github.io