Introduction to Amazon CodeGuru
Transcript of Introduction to Amazon CodeGuru
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Introduction to Amazon CodeGuru:Automate Code Reviews and Application Performance Recommendations
Srinivasan H Sengamedu Daniela Tzvetkova
Senior ML Manager Senior Product Manager
Amazon Web Services Amazon Web Services
A day in the life of Lynn
• Lynn is tech lead working on Java projects in an ecommerce company
• part of a distributed development team
• responsible for the backend services (search, order, and shipping) of her company’s high volume
site
• Her responsibilities span the entire application development and operations cycle
• D: We found a data corruption issue in production.
• L: Let’s find the root cause.
D: I think it is due to a data race.
Could we have caught it during code reviews? I wish we had someone who really understands concurrency.
• O: The site latency is increasing. I just got paged!
• L: Let’s find the root cause.
O: The CPUs are overloaded. Can we increase the fleet size?
• L: We increased the fleet size last month. The traffic is pretty much the same. What’s going on?
• O: ???
• L: OK, let’s increase the fleet size.
How do we find out what’s actually going on? I wish we’ve a performance expert in our team!
What’s on Lynn’s mind?
How can we improve code
quality?
Are we giving lowest latency to our customers?
Are our infrastructure
costs just bloating?
Lynn’s ecosystem
Write +
Review
Build +
TestDeploy Measure Improve
What’s missing in Lynn’s ecosystem?
• Detection of code defects early in the cycle
• Keeping up with coding best practices
• Identifying performance bottlenecks and linking them to code
• Tools for visualizing application performance
• Availability of expertise
• Faster time to resolution and remediation
• Developers need a truly integrated tool.
• The tool should provide actionable recommendations across phases in the life cycle.
What is Amazon CodeGuru
Amazon CodeGuru Reviewer
Amazon CodeGuru Profiler
Demo
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Introducing Amazon CodeGuru
• Machine learning service for automated code review and application performance profiling
• Trained on decades of knowledge and experience at Amazon
• Evolves with user feedback
• Searches for optimizations continuously, even in production
• Provides actionable recommendations to fix identified issues
• Automatically inspects code for hard to find defects
• Helps you find the most promising methods for optimization in your running application
It is like having a distinguished engineer on call 24x7
Amazon CodeGuru: Using ML to Code Review and Optimize High-Performing Applications
Easily identify performance
and cost improvements in
production environment
CodeGuru Profiler
Detect and optimize
the expensive lines
of code pre-prod
Built-in code reviews
with actionable
recommendations
CodeGuru Reviewer
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon CodeGuru Reviewer
• Provides automated code review comments
• Supports Java applications
• Integrated with GitHub and AWS CodeCommit source code repositories
• Leverages Pull Request-based code review workflow
Pull Request-based Code Review Process
1. Lynn creates a branch.
2. Lynn makes code changes.
3. She creates a Pull Request.
4. Code reviewers provide
comments. Lynn provides
responses.
5. The code changes are merged after approval.
Pull
Request
Approval
Merge
Code
Review
Branch
Make
changes
locally
Code Review Key Challenges
• Expertise• Availability, Compliance and Correctness aspects often do not get addressed because of
lack of expertise.
• Senior Talent• Code reviews often demand a senior engineer to be involved. Teams may not have the
right individuals or they may be focused on other high value tasks.
• Multiple functional areas• The number of topics which require expertise, e.g., AWS API use and concurrency, is
increasing
• Human code reviews often focus on business logic and less on functional correctness.
• Number and size of source code repos increasing• Reviews often require inspecting a large amount of source code for context
Amazon CodeGuru Reviewer
Flags critical defects and reliability issues in source code.
Amazon CodeGuru Reviewer augments human code review process and does not replace it
Pull
Request
Approval
Merge
Code
Review
Branch
Make
changes
locally
Amazon CodeGuru Reviewer
Code Areas addressed by CodeGuru Reviewer
AWS Best Practices: Correct use of AWS APIs
Incorrect use results in performance (e.g., polling) or correctness and completeness (e.g., pagination) issues.
Concurrency: Correct implementation of concurrency constructs.
Incorrect use results in correctness (e.g., missing synchronization) or performance issues (e.g., excessive synchronization) and hence impact availability.
Resource Leaks: Correct resource handling
Incorrect handling (e.g., not releasing database connection) results in slowdown and impacts availability.
Sensitive Information Leak: Leakage of Personally Identifiable Information
Leakage of sensitive information (e.g., logging of credit card number) leads to compliance issues.
Code defects discovered by mining data: Hard to find defects
Correcting issues (e.g., not creating a client for each lambda invocation) improves code quality.
CodeGuru Reviewer Workflow
Code
Repository
CodeGuru
Reviewer3. Recommendation
4. Developer
Feedback
1. Repository
Association
Repo Admin2. Pull
Request
Developer
Amazon CodeGuru Reviewer – How it Works
Customer performs
Pull Request
Input:
Source Code
try (GZip gzip =
GZIPInputStream.create(
url.openStream())) {
use(gzip);
} catch (Exception e) {
handle();
}
Extract semantic
features/patterns
Feature Extraction
gzip =
GZIPInputStream
.create(stream)
use(gzip)
ENTRY
EXIT
stream =
url.openStream()
gzip.close()
handle()
throw
Exception
ML algorithms + Program
analysis identify code defects
Machine Learning
Code
corpus
Customers see
recommendations as
Pull Request comments
Output:
Recommendations
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon CodeGuru Reviewer Demo
CodeGuru Reviewer – Concurrency
public String get(final String ip) {
if(!IP_PATTERN.matcher(ip).matches()) {
return ip;
}
if (repo.containsKey(ip)) {
return repo.get(ip);
}
…
}
Code
Recommendation“repo” is a ConcurrentHashMap and your usage of containsKey() and get() may not be thread-safe. In between containsKey()
and get(), another thread can remove the key and the get() will return null. Consider calling get() and using its result.
Good catch of a potential race.
Developer Feedback
public String get(final String ip) {
if(!IP_PATTERN.matcher(ip).matches()) {
return ip;
}
str = repo.get(ip);
if (str) {
return str;
}
…
}
Fix
CodeGuru Reviewer – Concurrency
synchronized (orderObject) {
obj = orderObject.get(name);
if (obj == null) {
obj = new orderObjectMarkdown(name, category);
orderObject.put(name, obj);
}
}
Code
Recommendation
Correct.
Developer Feedback
synchronized (orderObject) {
obj = orderObject.get(name);
if (obj == null) {
obj = new orderObjectMarkdown(name, category);
orderObject.putIfAbsent(name, obj);
obj = orderObject.get(name);
}
}
Fix
CodeGuru Reviewer – Looping vs Waiting
This code appears to be waiting for a resource before it runs. You could use the waiters feature to help improve
efficiency. Consider using TableExists, TableNotExists. For more information,
see https://aws.amazon.com/blogs/developer/waiters-in-the-aws-sdk-for-java/
Recommendation
do {
DescribeTableResult describe = ddbClient.describeTable(new DescribeTableRequest().withTableName(tableName));
String status = describe.getTable().getTableStatus();
if (TableStatus.ACTIVE.toString().equals(status)) {
return describe.getTable();
}
Thread.sleep(10 * 1000);
elapsedMs = System.currentTimeMillis() - startTimeMs;
} while (elapsedMs / 1000.0 < waitTimeSeconds);
Code
We should use waiters instead - will help remove a lot of this code.Developer Feedback
Fix:
Feedback from Amazon Developers
“Amazon CodeGuru Reviewer allowed us to catch a long-standing race condition lurking in a critical piece of code”
“We were using an Amazon DynamoDB table in a way that we don’t typically use and we missed this in the code review. Amazon CodeGuruReviewer identified that the operation was returning paginated results instead of all results. Great catch and great recommendation.”
“Amazon CodeGuru Reviewer recommended I use a try-with-resources to close the resource, and I implemented it immediately. +1.”
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Key challenges with poor application performance
• Poor end-user
experience
• Rise of performance problems:
Troubleshooting of distributed applications is
challenging
• Not enough performance engineers: Scarcity
of performance engineering expertise
• Higher cost of
compute
infrastructure
• Impact on mission
critical systems
Business impact Causes & contributing factors
• Losing customers
• Performance optimization is challenging:
not a domain expertise for most developers
CodeGuru Profiler finds your most expensive lines of codein your live application
• Trained to find methods with high-potential for performance optimization• High latency & low throughput
• High CPU utilization
• Recommends how to fix your code
• Intelligent profiler trained by many years of performance engineering experience at Amazon
• Interactive visualizations
Built for production systems
• Low overhead (~1%)
• Continuously runs on production
• Continuously analyzes performance
• Main use cases• Troubleshoot a bottleneck• Proactively improve latency and cost• Performance test• Learn how your application works
• Currently supports applications written in Java
Onboarding
Create Profiling Group
1
Update the IAM role used by the Profiler agent
2
Set Java application dependencies
3
Start Profiler agent in your application
4
Amazon CodeGuru Profiler Demo
How it works
CodeGuru Profiler – How it Works
Customer’s application
Profiler thread
Customer’s application
Profiler thread
Customer’s application
Profiler thread
Customer’s application
Profiler thread
Actionable recommendations
Amazon Developer Feedback on Profiler
Chris Butterfield, SDE
CodeGuru Profiler’s recommended fix removed the thread contention which was using 55.97% of CPU time. After the fix a single host could now serve ~7.5x more traffic than before. We reduced our number of instances by ~75% while still handling the same traffic
Rajesh Konatham, SDE
After following Profiler’s recommendation to remove these clones, we saw huge reductions in CPU utilization – a 40% reduction on the synchronous fleet and 67% reduction on the asynchronous fleet
Next steps
• Onboard CodeGuru Profiler
• https://aws.amazon.com/codeguru/
• Download the sample app
• https://github.com/aws-samples/aws-codeguru-profiler-sample-application
• Start profiling!
Summary
• Amazon CodeGuru makes it easy to improve code quality and application performance
• Amazon CodeGuru continuously learns and improves.
• Amazon CodeGuru Reviewer
• Identifies code defects. Provides actionable recommendations on code fixes
• Easily integrates with pull-request based code reviews
• Amazon CodeGuru Profiler
• Actionable recommendations on performance issues
• Rich visualizations to help troubleshoot issues
Get started at aws.amazon.com/codeguru
Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Srinivasan H Sengamedu Daniela Tzvetkova
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.