OPENERP BENCHMARK How$to$test$performance$and$robustness$against$your$own$volumes$?$

Cyril$MORISSE$($@cmorisse$)$$

Introduc3on%

!  At%Audaxis,%we%decided%to%invest%in%a%methodology%to%answer%3%kinds%of%ques3ons:%!  Is%OpenERP%able%to%manage%my%business%!  Is%this%hardware%configura3on%able%to%sustain%our%opera3ons%load%?%Un3l%when%?%

! What%hardware%should%we%buy%to%run%OpenERP%for%the%next%3%years%?%

What%to%measure%?%

!  You%must%find%the%most%challenging%business%processes%and%condi3ons%then%express%them%in%terms%of%what%you%can%measure%!  Eg.%Can%OpenERP%handle%240.000%orders%per%years%with%40%users%?%%

!  You%can’t%benchmark%on%this%base%%!  240.000%orders%per%year%is%2,5%orders%per%hour%per%user%(on%a%240%

working%days%of%10%hours%per%year%base)%"  Note:%that%this%is%one%order%every%36s%too%!%

!  What%you%can%measure%is:%!  Can%my%system%maintains%response%3me%<%1%s%per%transac3on%over%10%

hours%with%:%"  40%users%entering%1%order%every%16%minutes%during%10%(peak%hours%intensity)%%%

"  That’s%1%order%every%24s%

What%to%measure%(con3nued)%

!  What%does%«good%response%3me»%technically%means%?%

%

!  JSONbRPC%requests%response%3me%must%be%kept%around%1s%for%the%system%to%be%felt%as%responsive.%!  Funkload%measure%responsiveness%for%you%using%%APDEX%

OpenERP Database

OpenERP Application Server

User OpenERP Web Client

Enter fields, Press buttons JSON-RPC SQL

Responsiveness

1 Sale Order with 5 lines, “Confirmed” then “Invoiced”

~ 20 ~ 150 ~ 6700

How%to%measure%it%

!  Use%Funkload%!  Develop%a%test%case%which%simulates%user%interac3ons%(via%XMLbRPC…)%

!  Install%«%Funkload%monitor%»%on%your%servers%!  Define%and%run%Funkload%benchmarks%with:%

"  A%test%case%"  List%of%cycle%defining%the%number%of%concurrent%users%

"  Eg.%100,100,100,100%"  Cycle%dura3on%"  Frequency%(as%a%period%:%one%test%every%x%seconds)%

!  You%will%get%a%(quite)%fancy%report%with:%!  Test%execu3on%3me%!  Individual%request%execu3on%3me%!  Main%server%metrics%(CPU,%RAM,%Network%Load)%

%

Funkload%Sample%Report%(con3nued)%

Funkload%Sample%Report%(con3nued)%

Funkload%Sample%Report%(end)%

Database%size%really%maAers%

!  You%must%bench%with%real%volume%of%data%!!!%

0

1

2

3

4

5

6

7

8

0 100000 200000 300000 400000 500000

OpenERP Response Time evolution over Sales Order count

Response Time (s)

Database%size%really%maAers(con3nued)%

!  In%fact,%you%will%need%several%benchmarks:%!  Benchbfiller%

"  Use%these%benchmarks%to%fill%up%your%database%(customers,%products,%…)%

!  Benchbreference%"  2%hours%with%target%ac3vity%level%

!  Benchbrobustness%"  10%(several)%hours%with%target%ac3vity%level%

!  Benchblimit%" We%run%this%one%at%the%end%of%the%bench%to%determine%how%many%users%more%the%system%will%support%at%target%ac3vity%level%%

%

Plan%your%architecture%but%focus%on%the%database%

!  Configure%your%database%!%At%least%use%pgtune%!!!%!  ORM%may%be%your%enemy%;%think%about%the%roundtrips%involved%by%your%code%!  Don’t%split%Applica3on%and%Database%servers%too%early%!%!  If%your%database%is%not%too%big%:%think%SSD%(From%20x%to%250x%more%IOPS%than%HDD)*%

OpenERP Database OpenERP

Application Server OpenERP

Web Client

JSON-RPC SQL

Horizontal Scaling

Vertical Scaling

Tools%&%References%

!  Funkload%(%hAp://funkload.nuxeo.org%%)%!  Benchmark%(and%Test)%your%OpenERP%Server%and%others%

!  Muppy%(%hAp://bitbucket.org/cmorisse/muppy%)%!  “Make%your%OpenERP%Servers%behave%as%puppets”%!  Install%OpenERP%plamorms%everywhere%(in%really%alpha%stage)%

!  Pgbadger%(%hAp://dalibo.github.io/pgbadger%)%!  Analyse%SQL%requests%processed%by%your%PostgreSQL%Server%

!  Pgtune%(%hAps://github.com/gregs1104/pgtune%)%!  Helps%you%to%configure%your%PostgreSQL%(basically)%

!  Pg_ac3vity%(hAps://github.com/julmon/pg_ac3vity%)%!  Top%like%applica3on%for%PostgreSQL%Server%

!  OpenERP%buildout%Recipe%(%hAps://pypi.python.org/pypi/anybox.recipe.openerp%)%!  A%set%of%Ugly%scripts%to%automate%the%launch%of%Funkload%Filler%benchmarks%%

!  Theses%tools%allows%to%measure%performance%over%database%size%

!  hAp://www.apdex.org%%!  SQL%Databases%Don’t%Scale%by%Adam%WIGGINS%

!  hAp://adam.heroku.com/past/2009/7/6/sql_databases_dont_scale/%

!  HDD%vs%SSD%by%IBM%!  hAp://wwwb03.ibm.com/systems/power/hardware/peripherals/ssd/hdd_ssd.html%%

More%informa3on?%

www.audaxis.com%%

Contact%us%openerp@audaxis.com%

%

%

%

14

Visit our booth at OpenERP Community Days in Hall K