1

Real WorldPerformance Tuning

Ask Bjørn Hansen

OSCON 2001

2

Performance Tuning

• Show more pages

• Faster

• With less resources

• Design the architecture right

• Optimize the code

• (in that order!)

3

Memory usage

• N connections = N fat mod_p erl processes

• Fat processes doing little other than buffering to slow clients

4

Memory usage

20 connections per second

+ Each request takes 3 seconds to write to the network

= 60 active mod_perl processes

+ 15 spare processes for peaks

= 75 active mod_perl processes

* 20MB-12MB shared = 8MB memory

= 600MB memory

5

Reverse proxy

• Offload the buffering to a reverse proxy• Can

– Do caching– Serve static content– Distribute requests to different backend processes

• All in a “slim” process

6

Reverse proxies

• squid– best caching– not as flexible for a "reverse proxy" as mod_proxy

• apache/mod_proxy – simple to configure– known environment– can cache– mod_rewrite

7

apache/mod_proxy

• Specify what to pass through

RewriteRule ^/(foo/.*) http://localhost:8001/$1 [P]

• Specify what NOT to pass through

RewriteCond %{REQUEST_URI} !^/images/

RewriteRule /(.*) http://localhost:8002/$1 [P]

• Allow fix up of $r->connection->remote_ip

LoadModule proxy_add_forward_module modules/mod_proxy_add_forward.so

8

Memory Usage with apache/mod_proxy

• mod_proxy20 connections per second

+ Each request takes 3 seconds

to write to the network

= 60 active mod_proxy processes + 15 spare

= 75 running mod_proxy processes

75 mod_proxy processes (300KB each)

= ~25MB memory

9

Memory Usage with apache/mod_proxy

• mod_perl20 connections per second

+ Each request takes <.05 seconds

to write to the proxy

= 1-2 active mod_perl processes + 3 spare

= 5 running mod_perl processes

5 mod_perl processes (8MB non-shared each)

= 40MB memory

10

Memory Usage with apache/mod_proxy

25+40MB

= 65MB total memory usage

• >500MB less than the mod_perl alone!

11

Basic httpd.conf tuning

• mod_proxyMaxClients 512StartServers 50MinSpareServers 20MaxSpareServers 100

• mod_perlMaxClients 5StartServers 3MinSpareServers 1MaxSpareServers 5Port 80Listen 8001

12

mod_status

ExtendedStatus on

<Location /server-status>

SetHandler server-status

Order deny,allow

Deny from all

Allow from 1.2.3.5

</Location>

13

<VirtualHost> configurationNameVirtualHost 1.2.3.4

<VirtualHost 1.2.3.4> ServerName jobs.perl.org ... RewriteCond %{REQUEST_URI} !.*\.(gif|png|jpg)$ RewriteRule /(.*) http://localhost:8010/$1 [P]</VirtualHost>

<VirtualHost 1.2.3.4> ServerName onion.perl.org ServerAlias learn.perl.org ... RewriteRule /(.*) http://localhost:8020/$1 [P]</VirtualHost>

14

URI configuration

RewriteRule /(bar/.*) http://localhost:8030/$1 [P]

RewriteRule /(foo/.*) http://otherhost:8040/$1 [P]

• Each backend process can run as a different user– A process per developer– A process per customer– A process per site

• Backends can run on different machines

15

Focus on code quality

• The mod_perl guide recommends not using the IO:: modules or use vars qw(%foo);

• I say use them if you would like to– far fewer mod_perl processes

• use Exporter to export your function names as needed

• Of course, don't go crazy and use POSIX and CGI.pm everywhere

16

Load balancing

• mod_rewrite can do simple load balancing• the mod_perl processes can be behind a load

balancer, RewriteRule /(foo/.*)

http://modperl.farm.foo.com:8030/$1 [P]

• mod_backhand

17

Caching

• Browsers/end user proxies can cache from servers– set the right headers, content-length, last-modified

• Reverse proxies – you set the rules, if complete documents can be

cached

• Application can cache from other parts of the system (eg database)– even the database can cache some of what it has

done

18

HTTP headers

• Expires: • Content-Length:• Cache-Control:• Pragma: (old "Cache-Control")

• When the complete documents can be cached.

• If-Modified-Since:– 304 response

19

Application caching

• Generate static content every N minutes– for small set of files

• Save results from the SQL database– in a local BerkeleyDB or similar

• Generate a Storable or BerkeleyDB file centrally and rsync it to each mod_perl server

• Create summary tables in the database– to lighten the load of heavy queries

• Mix and match for fragments

20

Caching summary

• Decide for how long data can be considered "fresh enough"

• Cache fragments of pages where possible• Make each part of the system cache and

aggregate as much as possibly• Make SQL queries as simple and fast as

possibly

21

Databases

• Databases are hard(er) to scale• Reverse proxy minimizes the need for many

concurrent database connections• Apache::DBI minimizes the number of new

connections made• Caching minimizes the number of lookups• Summary tables can make the lookups faster

22

Summary

• Architecture more important than code

• Use many proxies– Allowing fewer heavy backends

• Caching is fundamental

23

Resources

• The mod_perl guide's performance section– http://perl.apache.org/guide/performance.html– lot's of nitty gritty details

• DBI and mod_perl– http://www.saturn5.com/~jwb/dbi-performance.html

• Database performance– Tim’s Advanced DBI Tutorial– http://www.cpan.org/authors/id/TIMB/DBI_Talk5_2001.tar.gz

• These slides– http://develooper.com/modperl/