FaultHunter workshop (SourceMeter for SonarQube plugin module)

29

description

A workshop presentation about FaultHunter, a programming mistake locator module for SourceMeter source code analysis engine compatible with SonarQube.

Transcript of FaultHunter workshop (SourceMeter for SonarQube plugin module)

Page 1: FaultHunter workshop (SourceMeter for SonarQube plugin module)
Page 2: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2

Coding Rule Violations

Gábor Szőke

PhD Student

University of Szeged

www.sed.hu

2014.04.02. FrontEndART.com

Page 3: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 3

What do we call Coding Rule Violations?

FrontEndART.com

• Source code patterns which indicate potential faults• These patterns come from actual bugs and include• Common programming mistakes• Bad programming habits• Performance issues

• The patterns are gathered by developer communities• Documented Eclipse bug fixes• Chapters of the Effective Java book• Blog posts on the http://thedailywtf.com website

Page 4: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 4

Bugs? Come on!

FrontEndART.com

• Developers are smart.

• Smart people don’t make dumb mistakes.

• WRONG!

• Smart people do make dumb mistakes.

• Common issues:• Wrong boolean operator, missing parenthesis, etc.

• Misunderstood class or method invariants

Page 5: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 5

Can You Find The Bug?

FrontEndART.com

JDK 1.6.0, b105, sun.awt.x11.XMSelection

if (listeners == null)

listeners.remove(listener);

Page 6: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 6

Control Flow Conventions Performance

Open Stream Suspicious Equals Method Use Arrays As List

Avoid Catching NPE Naming Conventions Unused Constructor

StringBuffer Instead Of

Concat

Npath Complexity Unused Imports

Empty Catch Block Cloneable Implementation Too Many Fields

FrontEndART.com

Some Frequent Anti-Pattern Categories

Page 7: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 7

Example

FrontEndART.com

• libsecurity_ssl/lib/sslKeyExchange.c

OpenSSL bug

Page 8: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 8

Example

FrontEndART.com

• libsecurity_ssl/lib/sslKeyExchange.c

IESMUB – If-Else Stmts Must Use

Braces

#gotofail

Page 9: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 9

Popular Solutions

FrontEndART.com

• CheckStyleCoding standard checkerHighly configurable×Small amount of implemented rules×Rules are mostly for style conventions,

not for fault prevention

• FindBugsLooks for bugs in Java codeFinds NPEs, typos, inconsistencies×A lot of irrelevant warnings×Does not show the roots of the problems

• PMDOpen-source code violation checkerMany rules×A lot of irrelevant warnings×Several significant warning are not found

Page 10: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 10

Pros and Cons

Relevant warning

Issues are found during development

Trains developers (lots of skills can be learned from following coding rules!)

Reduces project costs

Increases efficiency

Irrelevant warning

Creates work overhead

Developers will start ignoring coding issue warnings

Causes stress

Convinces developers that the code lacks actual coding issues even when there are a lot.

FrontEndART.com

Page 11: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 11

Can You Find The Bug?

FrontEndART.com

// com.sun.xml.internal.txw2.output.XMLWritertry { … }catch (IOException e) { new SAXException(”Server side Exception:” + e);}

// com.sun.xml.internal.txw2.output.XMLWritertry { … }catch (IOException e) { throw new SAXException(”Server side Exception:” + e);}

Page 12: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 12

False Positive and True Negative hits

FrontEndART.com

Found FaultsFaults

Not Faults

True Negative

False Positive

Solution?

Page 13: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 13

False Positive and True Negative hits

FrontEndART.com

Found FaultsFaults

Not Faults

True Negative

False Positive

We need a bigger boat!

Page 14: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 14

False Positive and True Negative hits

FrontEndART.com

Found Faults

Faults

Not Faults

True Negative

False Positive

We need a bigger boat!

Page 16: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 16

FaultHunter

FrontEndART.com

• Advanced static analysis engine• Re-implements inaccurate

PMD rule violations• 62 implemented PMD rule definitions

• Defines new rules• 8 new rule definitions

• Precise and efficient• A module of SourceMeter• Supports Ant and Maven integration• Is part of the SourceMeter for SonarQube plugin

Page 17: FaultHunter workshop (SourceMeter for SonarQube plugin module)

Advanced Static Analysis

source code

analysis

source code

abstract syntax tree

coding issue warnings

analysis

modeling

modeling

PMD

FaultHunter

Few warnings found

More true positivewarningsabstract semantic graph

Page 18: FaultHunter workshop (SourceMeter for SonarQube plugin module)

PMD Shared FaultHunter

ArgoUML 484 31% 1,086 531 34%Cobertura 195 5% 4,121 7,689 178%Eclipse 6,867 19% 28,661 65,773 185%GWT 1,166 17% 5,533 2,021 30%

Hibernate 560 17% 2,829 2,170 64%Jenkins 430 24% 1,399 2,649 145%Log4j2 120 22% 431 145 26%PMD 113 6% 1,815 2,112 110%

Spring-Framework 1,206 26% 3,497 1,219 26%Tomcat 859 19% 3,728 4,241 92%

… … … … … …Total 51,107 14% 308,339 358,543 100%

• 217 open source software systems analyzed

Results

Page 19: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 19FrontEndART.com

ResultsPMD

FH

PMD

FH

PMD

FH

PMD

FH

PMD

FH

PMD

FH

PMD

FH

Hib

ern

ate

Log4j

Tom

cat

-100% -50% 0% 50% 100% 150% 200% 250% 300% 350%

SharedPMDFaultHunter

Page 20: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 20FrontEndART.com

Rule  FP TN  Rule FP TN

AvoidCatchingNPE 3% 0% CloseResource 27% 45%

AvoidCatchingThrowable 0% 2% DefaultLabelNotLastInSwitchStmt 0% 2%

AvoidDeeplyNestedIfStmts 99% 21% EmptyCatchBlock 49% 0%

AddEmptyString 0% 0% EmptyFinallyBlock 0% 175%

AvoidInstanceofChecksInCatchClause 1% 1% EmptyIfStmt 14% 6%

ArrayIsStoredDirectly - - EmptySynchronizedBlock 0% 0%

AvoidPrintStackTrace 12% 2% EmptySwitchStatements - -

AvoidRethrowingException 51% 0% EmptyTryBlock 0% 0%

AvoidThrowingNullPointerException 2% 4% EmptyWhileStmt 60% 0%

AvoidThrowingRawExceptionTypes 2% 32% ForLoopsMustUseBraces - -

BooleanGetMethodName 0% 0% IfElseStmtsMustUseBraces 25% 397%

BooleanInstantiation 1% 1% IntegerInstantiation 2% 1%

BigIntegerInstantiation 1% 42% LooseCoupling 16% 134%

ByteInstantiation 0% 0% LongInstantiation 1% 0%

Results by rule violation types

Page 21: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 21FrontEndART.com

Results by rule violation types Rule FP TN  Rule FP TN

MissingBreakInSwitch 88% 67% UseArrayListInsteadOfVector 27% 98%

MethodWithSameNameAsEnclosingClass 0% 0% UncommentedEmptyMethod 28% 3%

MethodNamingConventions 0% 2% UseEqualsToCompareStrings 2% 105%

PositionLiteralsFirstInComparisons 6% 75% UnnecessaryLocalBeforeReturn 8% 55%

PreserveStackTrace 1% 35% UnusedLocalVariable 2% 551%

ReplaceHashtableWithMap 7% 7% UseLocaleWithCaseConversions 12% 77%

ReplaceVectorWithList 5% 30% UselessOverridingMethod 29% 42%

SimplifyConditional 13% 10% UnusedPrivateField 3% 439%

SimpleDateFormatNeedsLocale 6% 2% UnusedPrivateMethod 2% 235%

SuspiciousHashcodeMethodName UseStringBufferForStringAppends 21% 7%

ShortInstantiation 0% 0% UnusedImports 0% 25%

ShortMethodName 24% 0% WhileLoopsMustUseBraces

SwitchStmtsShouldHaveDefault 0% 3%

Page 22: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 22

Precise and Efficient

FrontEndART.com

Uses 50% more memory than PMD on the same source set

Runs 50% longer than PMD on the same source set

BUT!

• Finds 99.97% more faults than PMD

• Skips 14.92% of false positive warnings

Page 23: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 23

Examples

FrontEndART.com

• org.eclipse.ui.forms/src/org/eclipse/ui/forms/DetailsPart.java

EIF – Empty If StatementPMD misses it, FaultHunter doesn’t.

True Negative

What was the programmer’s intention? Did they simply forget to code the instructions? Potential

threat!

Page 24: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 24

Examples

FrontEndART.com

• org.eclipse.ui.workbench/Eclipse UI/org/eclipse/ui/internal/WorkbenchWindow.java

BGMN – Boolean Get Method NamePMD misses it, FaultHunter doesn’t.

True Negative

• org.eclipse.ui.ide/src/org/eclipse/ui/internal/ide/misc/ProjectCapabilitySelectionGroup.java

Issue: JavaEE, SpringFramework searches for getters like

isShellActivated() or isModified() Error!

Page 25: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 25

Examples

FrontEndART.com

• Tomcat/java/org/apache/catalina/core/ApplicationFilterConfig.java

USBFSA – Use String Buffer For String

AppendsPMD finds it, FaultHunter

doesn’t.False Positive

Totally irrelevant warning

Page 26: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 26

Examples

FrontEndART.com

• org.eclipse.core.resources/src/org/eclipse/core/internal/events/NotificationManager.java

MBIS – Missing Break In SwitchPMD misses it, FaultHunter doesn’t.

True Negative

Why is the break statement missing? It was intentional, as

the comment indicates.

Page 27: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 27

Examples

FrontEndART.com

• Log4j\src\main\java\org\apache\log4j\NDC.java

Is it really necessary to use the thread-safe Vector class?

If this is not the case, we can use the ArrayList class which offers a much faster implementation with the

same functionality.(Collections.synchronizedList(…) can be used in

a multi-threaded environment too!) Comment indicates that developers see the problem

as well.

UALIOV – Use Array List Instead Of Vector

PMD misses it, FaultHunter doesn’t.True Negative

Page 28: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 28

Examples

FrontEndART.com

• openejb-4.5.2/org/apache/openejb/server/cxf/rs/CustomProviderTest.java

ACNPE – Avoid Catching NullPointerExceptionPMD misses it, FaultHunter doesn’t.

True Negative

• xalan-2.7/src/org/apache/xpath/jaxp/XPathImpl.java

SMN – Short Method NamePMD misses it, FaultHunter doesn’t.

True Negative

• Spring-Framework/spring-beans/src/test/java/org/springframework/tests/sample/beans/TestBean.java

ACNPEAvoid Throwing Raw Exception TypesPMD misses it, FaultHunter doesn’t.

True Negative

Page 29: FaultHunter workshop (SourceMeter for SonarQube plugin module)

2023.04.13. 29

Examples

FrontEndART.com

• ApacheDS/core-pi/src/test/java/org/apache/directory/server/core/api/changelog/TagTest.java

Position Literals First In ComparisonsPMD finds it, FaultHunter doesn’t.

False PositiveTotally irrelevant warning.