Towards Tooling; A Look at What is Missing From the Ruby Toolbox

Towards Toolingwhat is missing

from our toolbox?Loren Segal

@lsegal

Friday, November 8, 13

Are Rubyists good at testing because they

have good tools?Friday, November 8, 13

Do Rubyists have good tools

because they are good at testing?


Tools are important


We have good tools


...sometimes.


This talk is about the

not-so-good tools


GoalsFriday, November 8, 13

1. Introduce

different toolsFriday, November 8, 13

2. Find out which

tools we are missing


3. Write these

tools plz thx!Be a garbage collector


Note:

GoogleTOOL NAME + LANGUAGE

You should find the tools referenced in this talk


Kinds of Tools


Deployment / OpsDocumentation

TestingVisualization

DebuggingLinting

Static Analysis

High Level

Low Level


Visualization


Some ofthe most

important toolsare visualization tools


Know what your code

is doingFriday, November 8, 13

Thread in a sealed box.Is it dead or alive?


Visual Studio


XCode


VisualVM


Discoverability


Call references

Implementors ECLIPSE


Not just IDEs


I’ll prove it...


Firebug


Do you remember web development before Firebug?


Before: no visibility.


Ember Inspector


SmalltalkFriday, November 8, 13

InherentlyVisual


Where isRuby viz?


RubyMine


Profilers?Friday, November 8, 13

memprofJoe Damato

github/ice799/memprofFriday, November 8, 13

https://github.com/ice799/memprof

https://github.com/ice799/memprof

perftools.rb


NetBeans / JRuby


Use theJVM


Lintng��


Lintdivide by zero: checkinitialized vars: check

...style: check (last!)


Ruby?


Reek/Flog/FlayDoes: detect code smellsDoes not: find common errors


Assumption:Pretty code iscorrect code


Ugly.Not “correct”.


github.com/lsegal/my_fake_project


PS. I ♡Code

ClimateFriday, November 8, 13

Understandyour tools


Code Climate does not replace testing


ruby-lintYorick Peterse

but it’s newFriday, November 8, 13

Nothing comes

standardFriday, November 8, 13

Other languages?


JSHint (JavaScript)pylint (Python)

FindBugs (Java)FxCop (C#)


Widely used.


Why notRuby?


StaticAnalysis

lint++Friday, November 8, 13

is ahuge field


Types of “static analysis”- Defect Finding

- Memory Checking / Fuzz Testing

- Extended Static Checking

- Model Checking / Data Flow Analysis

- Symbolic ExecutionFriday, November 8, 13

Defect Finding


is basically lint,


but with less emphasis on syntax.


The Usual Suspects


BrakemanJustin Collins

brakemanscanner.org(Ruby on Rails)


Finds common flawsin Rails code

XSS, SQL injection, mass assignment


Static detection of security vulnerabilitiesin scripting languages

https://www.usenix.org/legacy/event/sec06/tech/full_papers/xie/xie_html/




Fuzz Testing


garbage in...


Lots of tools.

C, Java, JS, Python, etc.


Lots of papers.


“Automated Whitebox Fuzz Testing”

Microsoft Research(used in SAGE)

http://research.microsoft.com/en-us/um/people/pg/public_psfiles/ndss2008.pdf






What about us?


HeckleRyan Davis, Kevin Clark


MutantMarkus Schirp

github/mbj/mutantFriday, November 8, 13

We could use a real fuzz testing tool.


FuzzBert?Martin Bosslet

github/krypt/FuzzBertFriday, November 8, 13

lots of papers out therewith algorithms to implement


LET’S GET


Symbolic Execution


Run your codewith no immediate values


Similar to Extended Static Checking

but...


Contracts not required

and

Can tell you which inputs generated valid or invalid state


Think:

Automatic Test Case Generation


// @example pow(2, 8) == 256 int pow(int x, int n) { int v[32] = {x}, result = 0; for (int i = 1; i < n; i++) { v[i] = x * v[i-1]; } return v[n-1]; }


SymExe report:

x=1,n=5,result=1x=2,n=8,result=256x=1,n=0,error: array out of bounds ← x=1,n=33,error: array out of bounds ←


// @example pow(2, 8) == 256 // @requires n > 0 // @requires n < 32 int pow(int x, int n) { int v[32] = {x}, result = 0; for (int i = 1; i < n; i++) { v[i] = x * v[i-1]; } return v[n-1]; }


Tools?


KLEE (LLVM)Kudzu (JavaScript)

Kiasan (Java, SPARK)


Nothing for Ruby*

(*) “Automatic Program Verification and Test Case Generation of Ruby Programs”


Ruby doesn’t really have a scientific community.


Chicken and egg.


Python vs Ruby?Big boy language?


RECAP


We are greatat testing,

deployment,web frameworks


Not so good atvisualization,

linting,static analysis


We attractweb developersbecause we have good

web tools


Could webuild toolsfor other

communities?science, engineering, math


Take responsibility.


Great tool ideas arewaiting to be implemented


Tons of research papersin fields I mentioned

scholar.google.com


I had a whole section on my favourite research papers.


Come find me if you want titles.


Thank you.

Slides will be linked on Twitter@lsegal


Towards Tooling; A Look at What is Missing From the Ruby Toolbox

Technology

Transcript of Towards Tooling; A Look at What is Missing From the Ruby Toolbox