Recommending Proper API Code Examples for Documentation Purpose
-
Upload
- -
Category
Technology
-
view
152 -
download
2
description
Transcript of Recommending Proper API Code Examples for Documentation Purpose
Recommending Proper API Code Examples for Documentation Purpose
Lee Wei Mar, Ye-Chi Wu, and Hewijin Christine Jiau
DBSE Lab @ Inst. CCE & Dept. EE National Cheng Kung University (NCKU), Taiwan
Outline
• Introduction
• Code Search Engines for Code Example Collection
• The Proposed Methodology - PropER-Doc
– Execution Process
– Recommendation Assistants
• Evaluation
• Conclusion
Introduction
• Nowadays, object-oriented frameworks & libraries offer powerful APIs
that facilitate programmers in accomplishing their works
Introduction
• Nowadays, object-oriented frameworks & libraries offer powerful APIs
that facilitate programmers in accomplishing their works
Power ↑ API Complexity ↑ Learning Barrier ↑
Introduction
• Nowadays, object-oriented frameworks & libraries offer powerful APIs
that facilitate programmers in accomplishing their works
Power ↑ API Complexity ↑ Learning Barrier ↑
Hamper
Introduction
• Nowadays, object-oriented frameworks & libraries offer powerful APIs
that facilitate programmers in accomplishing their works
Power ↑ API Complexity ↑ Learning Barrier ↑
To ensure the facilitation, offering effective API learning resources is essential for API providers
Hamper
Introduction (cont.)
• Code examples (CEs) are effective resources for API learning
Introduction (cont.)
• Code examples (CEs) are effective resources for API learning
public class Snippet26 {
public static void main (String [] args) {
Display display = new Display ();
Shell shell = new Shell (display);
Combo combo = new Combo (shell, SWT.READ_ONLY);
combo.setItems (new String [] {"Alpha", "Bravo", "Charlie"});
Rectangle clientArea = shell.getClientArea ();
combo.setBounds (clientArea.x, clientArea.y, 200, 200);
shell.pack ();
shell.open ();
….
}
Eclipse SWT
Independent CE repository
Introduction (cont.)
• Code examples (CEs) are effective resources for API learning
public class Snippet26 {
public static void main (String [] args) {
Display display = new Display ();
Shell shell = new Shell (display);
Combo combo = new Combo (shell, SWT.READ_ONLY);
combo.setItems (new String [] {"Alpha", "Bravo", "Charlie"});
Rectangle clientArea = shell.getClientArea ();
combo.setBounds (clientArea.x, clientArea.y, 200, 200);
shell.pack ();
shell.open ();
….
}
Eclipse SWT Eclipse JDT
Independent CE repository Embedding CE in API reference
Introduction (cont.)
• Code examples (CEs) are effective resources for API learning
Guide programmers to properly use API by demonstration
Introduction (cont.)
• Code examples (CEs) are effective resources for API learning
• However, many APIs fail to offer sufficient CEs
Guide programmers to properly use API by demonstration
Introduction (cont.)
• Code examples (CEs) are effective resources for API learning
• However, many APIs fail to offer sufficient CEs
API providers must invest large human effort for CE
construction from scratch
Guide programmers to properly use API by demonstration
CSEs for Code Example Collection
• Code Search Engines (CSEs) are potentially useful in CE collection
query
source files potentially use the API element
API provider
CSEs for Code Example Collection
• Code Search Engines (CSEs) are potentially useful in CE collection
The signature of a specific API element
is given as the query keyword
query
source files potentially use the API element
API provider
CSEs for Code Example Collection
• Code Search Engines (CSEs) are potentially useful in CE collection
The signature of a specific API element
is given as the query keyword • These files contain “client code” that demonstrates the use of the API element
• The client code can be treated as “candidates” of CEs for documentation
query
source files potentially use the API element
API provider
CSEs for Code Example Collection - Challenges
• Code Search Engines (CSEs) are potentially useful in CE collection
API providers must manually
query
source files potentially use the API element
API provider
CSEs for Code Example Collection - Challenges
• Code Search Engines (CSEs) are potentially useful in CE collection
API providers must manually
query
source files potentially use the API element
API provider
1. Locate API usage code scattered in source files as candidates
2. Reorganize candidates according to different usages
3. Check appropriateness of candidates for documentation
CSEs for Code Example Collection - Challenges
• Code Search Engines (CSEs) are potentially useful in CE collection
These manual efforts must be reduced to make such practice applicable
API providers must manually
query
source files potentially use the API element
API provider
1. Locate API usage code scattered in source files as candidates
2. Reorganize candidates according to different usages
3. Check appropriateness of candidates for documentation
The Proposed Methodology – PropER-Doc
• Proper code Example candidates Recommendation for Documentation
– Use API element links to guide the recommendation from CSE results
The Proposed Methodology – PropER-Doc
• Proper code Example candidates Recommendation for Documentation
– Use API element links to guide the recommendation from CSE results
Links between API elements to indicate the joint use relationship
• Structural Links from structural dependencies in API implementation
• Conceptual Links from cross references in API documentation
The Proposed Methodology – PropER-Doc
• Proper code Example candidates Recommendation for Documentation
– Use API element links to guide the recommendation from CSE results
– Offer recommendation assistants for candidate inspection & selection
Links between API elements to indicate the joint use relationship
• Structural Links from structural dependencies in API implementation
• Conceptual Links from cross references in API documentation
PropER-Doc Execution Process
Candidate Collection
Candidate Annotation
Candidate Recommendation
targetType query perform
code search
check API call significance
candidates
API Provider
candidate suggestion
annotated
candidates
PropER-Doc Execution Process
Candidate Collection
targetType query perform
code search
API Provider
PropER-Doc Execution Process
Candidate Collection
targetType query perform
code search
API Provider
The API provider gives a name
of the API type that needs CEs
PropER-Doc Execution Process
Candidate Collection
targetType query perform
code search
API Provider
The API provider gives a name
of the API type that needs CEs The query is delegated to CSEs
PropER-Doc Execution Process
Candidate Collection
targetType query perform
code search
candidates
API Provider
The API provider gives a name
of the API type that needs CEs The query is delegated to CSEs
• PropER-Doc collects candidates from CSE results
• Candidate: a method implementation that refers targetType usage
PropER-Doc Execution Process
Candidate Collection
Candidate Annotation
targetType query perform
code search
check API call significance
API Provider
candidates
PropER-Doc Execution Process
Candidate Collection
Candidate Annotation
targetType query perform
code search
check API call significance
API Provider
candidates
For each API call, annotate it with a
significance degree
6
(relevant) 0 (irrelevant)
PropER-Doc Execution Process
Candidate Collection
Candidate Annotation
targetType query perform
code search
check API call significance
API Provider
annotated
candidates
candidates
For each API call, annotate it with a
significance degree
6
(relevant) 0 (irrelevant)
PropER-Doc Execution Process
Candidate Collection
Candidate Annotation
Candidate Recommendation
targetType query perform
code search
check API call significance
candidates
API Provider
annotated
candidates
candidate suggestion
PropER-Doc Execution Process
Candidate Collection
Candidate Annotation
Candidate Recommendation
targetType query perform
code search
check API call significance
candidates
API Provider
annotated
candidates
candidate suggestion
with recommendation assistants
Candidate Recommendation
Four Recommendation Assistants are offered to guide the recommendation
Candidate Recommendation
Grouping for
distinguishing API usages
[All candidates]
Four Recommendation Assistants are offered to guide the recommendation
Candidate Recommendation
Grouping for
distinguishing API usages
[All candidates] [A selected group]
Diagramming API
types interaction
Four Recommendation Assistants are offered to guide the recommendation
Candidate Recommendation
Grouping for
distinguishing API usages
[All candidates] [A selected group]
Diagramming API
types interaction
[Candidates in a group]
Ranking based on
appropriateness metrics
Four Recommendation Assistants are offered to guide the recommendation
Candidate Recommendation
Grouping for
distinguishing API usages
[All candidates] [A selected group]
Diagramming API
types interaction
[Candidates in a group] [A selected candidate]
Ranking based on
appropriateness metrics
Presenting the
candidate for inspection
Four Recommendation Assistants are offered to guide the recommendation
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[All candidates] – Distinguish different API usages
Recommendation - Clustering Grouping Diagramming Ranking Presenting
Rationale: In different usages, targetType tends to interact with different API types
[All candidates] – Distinguish different API usages
Recommendation - Clustering Grouping Diagramming Ranking Presenting
Rationale: In different usages, targetType tends to interact with different API types
[All candidates] – Distinguish different API usages
• PropER-Doc extracts iTypes of each candidate
– iTypes: the set of API types where targetType directly interacts with
• Candidates with the same iTypes tend to demonstrate the same usage and are grouped together
Recommendation - Clustering Grouping Diagramming Ranking Presenting
Rationale: In different usages, targetType tends to interact with different API types
[All candidates] – Distinguish different API usages
• PropER-Doc extracts iTypes of each candidate
– iTypes: the set of API types where targetType directly interacts with
• Candidates with the same iTypes tend to demonstrate the same usage and are grouped together
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage
• Visual Grammar
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage
• Visual Grammar
– Box: API Type
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage
• Visual Grammar
– Box: API Type
– Edge / Label: interaction / API calls
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage
• Visual Grammar
– Box: API Type
– Edge / Label: interaction / API calls
– API call popularity
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage
• Visual Grammar
– Box: API Type
– Edge / Label: interaction / API calls
– API call popularity
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage
• Visual Grammar
– Box: API Type
– Edge / Label: interaction / API calls
– API call popularity
• Selection of interested API calls for
filtering candidates
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[A set of similar candidates demonstrating the same usage is chosen]
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[A set of similar candidates demonstrating the same usage is chosen]
Which one is more appropriate for documentation and should be inspected first?
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[A set of similar candidates demonstrating the same usage is chosen]
Define three appropriateness metrics to rank the candidates
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[A set of similar candidates demonstrating the same usage is chosen]
Define three appropriateness metrics to rank the candidates
Significance
Importance of referred API calls
>
Density
Portion of relevant API calls
>
Statements with significance degree
Candidate
6 0
Cohesiveness
Degree of relevant call aggregation
> more
representative more
comprehensible easier extraction
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[A set of candidates is chosen]
Rank function: f(c) = Significance(c) + Density(c) + Cohesiveness(c)
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[A candidate is selected for inspection & CE construction]
Recommendation - Clustering Grouping Diagramming Ranking Presenting
Highlight relevant code place to facilitate code inspection & extraction
[A candidate is selected for inspection & CE construction]
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[A candidate is selected for inspection & CE construction]
Highlight relevant code place to facilitate code inspection & extraction
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[A candidate is selected for inspection & CE construction]
Important code blocks are highlighted
Highlight relevant code place to facilitate code inspection & extraction
Recommendation - Clustering Grouping Diagramming Ranking Presenting
[A candidate is selected for inspection & CE construction]
Important code blocks are highlighted
Key API elements are marked as bold
Highlight relevant code place to facilitate code inspection & extraction
Evaluation
• Objective: check PropER-Doc effectiveness on
A. Distinguishing different API usages among candidates
B. Suggesting proper candidates for CEs construction
• Procedure
1. For an API type, manually identify its typical usages (U1) that need CEs
2. Use PropER-Doc to collect candidates and distinguish the API type usages (U2)
3. Map between U1 and U2 (objective A)
4. For each usage in U1, use PropER-Doc to select the top-ranked candidate
5. Evaluate the effort for constructing CE from top-ranked candidate (objective B)
• Subject: ASTParser class (A complex API type in Eclipse JDT framework)
Evaluation Result – Mapping Result
U1
Get AST from ICompilationUnit
Get AST from a string of class body
Get AST from a string of statements
Get AST from a string of expression
Get AST from ITypRoot
Get IJavaElement binding info.
Get AST from source code
Get ASTs from ICompilationUnits
Get AST from IClassFile
U2 (GID)
1
2
3
4
5
6
7
8
9
Evaluation Result – Mapping Result
U1
Get AST from ICompilationUnit
Get AST from a string of class body
Get AST from a string of statements
Get AST from a string of expression
Get AST from ITypRoot
Get IJavaElement binding info.
Get AST from source code
Get ASTs from ICompilationUnits
Get AST from IClassFile
U2 (GID)
1
2
3
4
5
6
7
8
9
• Mostly 1 : 1 mapping
Evaluation Result – Mapping Result
U1
Get AST from ICompilationUnit
Get AST from a string of class body
Get AST from a string of statements
Get AST from a string of expression
Get AST from ITypRoot
Get IJavaElement binding info.
Get AST from source code
Get ASTs from ICompilationUnits
Get AST from IClassFile
U2 (GID)
1
2
3
4
5
6
7
8
9
• Mostly 1 : 1 mapping
– Confirm the effectiveness of the
grouping assistant in PropER-Doc
Evaluation Result – Mapping Result
U1
Get AST from ICompilationUnit
Get AST from a string of class body
Get AST from a string of statements
Get AST from a string of expression
Get AST from ITypRoot
Get IJavaElement binding info.
Get AST from source code
Get ASTs from ICompilationUnits
Get AST from IClassFile
U2 (GID)
1
2
3
4
5
6
7
8
9
• Mostly 1 : 1 mapping
– Confirm the effectiveness of the
grouping assistant in PropER-Doc
• For the 3 : 1 mapping, the
diagramming assistant is useful in distinguishing different usages
Evaluation Result – Mapping Result
U1
Get AST from ICompilationUnit
Get AST from a string of class body
Get AST from a string of statements
Get AST from a string of expression
Get AST from ITypRoot
Get IJavaElement binding info.
Get AST from source code
Get ASTs from ICompilationUnits
Get AST from IClassFile
U2 (GID)
1
2
3
4
5
6
7
8
9
The Grouping and Diagramming assistants are useful in distinguishing different API usages
• Mostly 1 : 1 mapping
– Confirm the effectiveness of the
grouping assistant in PropER-Doc
• For the 3 : 1 mapping, the
diagramming assistant is useful in distinguishing different usages
Evaluation Result – Constructing Effort
U1 ∆Stmt (+/-) Coh.
Get AST from ICompilationUnit 0 / 0 1.00
Get AST from a string of class body 0 / 0 1.00
Get AST from a string of statements 0 / 0 1.00
Get AST from a string of expression 4 / 0 1.00
Get AST from ITypRoot 0 / 1 1.00
Get IJavaElement binding info. 3 / 0 1.00
Get AST from source code 0 / 0 1.00
Get ASTs from ICompilationUnits 16 / 0 0.63
Get AST from IClassFile 0 / 0 1.00
∆Stmt (+/-): #(extra statements) / #(missed statements)
Coh.: Cohesiveness (between 0 and 1)
Evaluation Result – Constructing Effort
U1 ∆Stmt (+/-) Coh.
Get AST from ICompilationUnit 0 / 0 1.00
Get AST from a string of class body 0 / 0 1.00
Get AST from a string of statements 0 / 0 1.00
Get AST from a string of expression 4 / 0 1.00
Get AST from ITypRoot 0 / 1 1.00
Get IJavaElement binding info. 3 / 0 1.00
Get AST from source code 0 / 0 1.00
Get ASTs from ICompilationUnits 16 / 0 0.63
Get AST from IClassFile 0 / 0 1.00
∆Stmt (+/-): #(extra statements) / #(missed statements)
Coh.: Cohesiveness (between 0 and 1)
• 5 of 9 top candidates can be
directly used as CEs
Evaluation Result – Constructing Effort
U1 ∆Stmt (+/-) Coh.
Get AST from ICompilationUnit 0 / 0 1.00
Get AST from a string of class body 0 / 0 1.00
Get AST from a string of statements 0 / 0 1.00
Get AST from a string of expression 4 / 0 1.00
Get AST from ITypRoot 0 / 1 1.00
Get IJavaElement binding info. 3 / 0 1.00
Get AST from source code 0 / 0 1.00
Get ASTs from ICompilationUnits 16 / 0 0.63
Get AST from IClassFile 0 / 0 1.00
∆Stmt (+/-): #(extra statements) / #(missed statements)
Coh.: Cohesiveness (between 0 and 1)
• 5 of 9 top candidates can be
directly used as CEs
• Only 1 top candidate misses 1
statement for CE construction
– Top candidates contains almost all required API calls for CE construction
Evaluation Result – Constructing Effort
U1 ∆Stmt (+/-) Coh.
Get AST from ICompilationUnit 0 / 0 1.00
Get AST from a string of class body 0 / 0 1.00
Get AST from a string of statements 0 / 0 1.00
Get AST from a string of expression 4 / 0 1.00
Get AST from ITypRoot 0 / 1 1.00
Get IJavaElement binding info. 3 / 0 1.00
Get AST from source code 0 / 0 1.00
Get ASTs from ICompilationUnits 16 / 0 0.63
Get AST from IClassFile 0 / 0 1.00
∆Stmt (+/-): #(extra statements) / #(missed statements)
Coh.: Cohesiveness (between 0 and 1)
• 5 of 9 top candidates can be
directly used as CEs
• Only 1 top candidate misses 1
statement for CE construction
– Top candidates contains almost all required API calls for CE construction
• High cohesiveness in all top
candidates
– Effort for relevant code extraction from top candidates is quite low
Evaluation Result – Constructing Effort
U1 ∆Stmt (+/-) Coh.
Get AST from ICompilationUnit 0 / 0 1.00
Get AST from a string of class body 0 / 0 1.00
Get AST from a string of statements 0 / 0 1.00
Get AST from a string of expression 4 / 0 1.00
Get AST from ITypRoot 0 / 1 1.00
Get IJavaElement binding info. 3 / 0 1.00
Get AST from source code 0 / 0 1.00
Get ASTs from ICompilationUnits 16 / 0 0.63
Get AST from IClassFile 0 / 0 1.00
PoperER-Doc suggests high quality candidates for CEs construction
• 5 of 9 top candidates can be
directly used as CEs
• Only 1 top candidate misses 1
statement for CE construction
– Top candidates contains almost all required API calls for CE construction
• High cohesiveness in all top
candidates
– Effort for relevant code extraction from top candidates is quite low
Conclusion
• PropER-Doc is proposed to assist API providers in constructing proper API code examples using code search engines
1. Candidate grouping algorithm
2. API-type interaction diagram
3. Metrics-based ranking mechanism
Conclusion
• PropER-Doc is proposed to assist API providers in constructing proper API code examples using code search engines
1. Candidate grouping algorithm
2. API-type interaction diagram
3. Metrics-based ranking mechanism
Distinguish different API usages
Conclusion
• PropER-Doc is proposed to assist API providers in constructing proper API code examples using code search engines
1. Candidate grouping algorithm
2. API-type interaction diagram
3. Metrics-based ranking mechanism
Distinguish different API usages
Inspect specific usage & filter candidates
Conclusion
• PropER-Doc is proposed to assist API providers in constructing proper API code examples using code search engines
1. Candidate grouping algorithm
2. API-type interaction diagram
3. Metrics-based ranking mechanism
Distinguish different API usages
Inspect specific usage & filter candidates
Select high quality candidates
Conclusion
• PropER-Doc is proposed to assist API providers in constructing proper API code examples using code search engines
1. Candidate grouping algorithm
2. API-type interaction diagram
3. Metrics-based ranking mechanism
• Evaluation on Eclipse JDT framework has been conducted to confirm the effectiveness of PropER-Doc
Distinguish different API usages
Inspect specific usage & filter candidates
Select high quality candidates
Thank You
Lee Wei Mar (馬立偉)
Database and Software Engineering Laboratory,
Institute of Computer and Communication Engineering &
Department of Electrical Engineering,
National Cheng Kung University, Tainan, Taiwan