2006/07/12 ICPADS 2006@Minneapolis 1

Secure and High-performance Web Server System

for Shared Hosting Service

Daisuke Hara and Yasuichi Nakayama

The University of Electro-Communications, Tokyo, Japan

2006/07/12 2

Outline

Introduction Background

– Problems of large-scale hosting service and web server

Proposal - Hi-sap– Design– Implementation

Evaluation Conclusions

2006/07/12 3

Introduction Problem of existing web servers

– Server embedded interpreters cannot be used safely in large-scale environments like a shared hosting service.

Proposal - Hi-sap– Web objects that are stored in a server are divided into

partitions*. – Server processes run under the privilege of different users in

every partition. Achievement

– Hi-sap solves the problem.– It achieves high performance & scalability.

(*) “partition” is a unit of division of web objects. (e.g. site, content, QUERY_STRING)

2006/07/12 4

Background

More people are creating their own websites as the Internet grows in popularity.– weblog, wiki, CMS

Shared hosting services are widely used.– Many customers share a server.

100s - 1000s sites/server

– low price & flexible custom CGI, etc.

2006/07/12 5

Server embedded interpreters

e.g. PHP, mod_ruby, mod_perl

Because they have server processes including interpreters of language processors,

they can improve performance in processing dynamic content like weblogs and wikis.

2006/07/12 6

Problem of existing web servers

A’s websiteA’s website B’s websiteB’s website

C’s websiteC’s website

Server

Internal users can steal & delete authentication content withoutauthentication (cp, rm commands or malicious CGI scripts).

browser

authentication

auth contentauth content

steal & delete

ID & Pass

It is required to grant read permission to an other.

(rw-r--r--)

2006/07/12 7

Problem of existing web servers (cont.)

Existing solution: POSIX ACL & suEXEC– CGI scripts run under the privilege of the site owner by

using suEXEC.– Permissions of public access files are granted only to

the dedicated user* by using POSIX ACL.

– It is not required to grant read permission to an other.

(*) “dedicated user” is user account that runs server processes.

e.g. www, apache, www-data

2006/07/12 8

Problem of existing web servers (cont.)

Even if POSIX ACL & suEXEC is used, the problem occurrs when server embedded interpreters are used.– Dynamic content that use server embedded in

terpreters (e.g. PHP, mod_ruby, mod_perl) also run under the privilege of a dedicated user.

– Malicious PHP scripts can steal & delete authentication content.

2006/07/12 9

Harache ([13][14]) Predecessor of Hi-sap Server processes run under the privilege of the site

owner.

rootroot

root①②

④

browser

GET /~userA/

① A browser sends request to the user A's website.② The privilege of the server process is changed to

user A.③ The server process processes the request.④ It returns a response to the browser.

Harache

Server Process

userA③

2006/07/12 10

Harache (cont.)

Server embedded interpreters can be used safely.– File permissions to a dedicated user are not

necessary.– It is required to grant permissions only to the site

owner. But, it cannot fully use the increased speed of

server embedded interpreters.– Server processes terminate after each session. (=

CGI)

Hi-sap solves Harache’s performance problem.

2006/07/12 11

Goal

Realization of secure, high-performance, and scalable web server system, Hi-sapSecure: Scripts of a partition cannot access

other partitions.High performance: Dynamic content can be

processed at high speed by fully using the increased speed of server embedded interpreters.

Scalable: A number of partitions can be housed in a server.

2006/07/12 12

Design

Security– Server processes run under the privilege of different

users in every partition. (= Harache)– The system brings access control into operation with

a secure OS. Performance

– The system pools server processes that run under the privilege of the different users. (!= Harache)

Scalability– The system controls the creation and termination of

server processes.

Content Access Scheduler

2006/07/12 13

Content Access Scheduler

Web-server level scheduler– [aim] It enhances the scalability of the number

of partitions in a server.– [method] It controls the creation and

termination of server processes.

By using the suitable scheduler for the purpose, it achieves high-scalability.

2006/07/12 14

Implementation

OS: Linux OS with SELinux dispatcher

– reverse proxy server– Apache 2.0.55 + mod_hisap

workers– Each worker runs under the privilege of a different

user and processes requests for a specific dedicated partition.

– Apache 2.0.55 x 1000 Any web server software can be used.

hisapd– Content Access Scheduler

2006/07/12 15

Overview of request processing

B

workersworkers

……

GET / HTTP/1.1Host: www.C.net

terminating worker A

www

www

A

A

A

BB

BBC

CC

C

hisapdhisapd

asking to activate worker C

root

root

worker A has no

requests

HTTPHTTP

UNIX Domain socket

sending the response

process the request

reverse proxy

activating worker C

confirming if worker C is active

dispatcherdispatcher

OK

BrowserBrowser

Server

heavy load

2006/07/12 16

Scheduling algorithm

We developed Content Access Scheduler to avoid thrashing.– Thrashing decreases the performance of web

servers dramatically. Algorithm of worker activation

– hisapd dynamically activates workers after requests from the dispatcher.

Algorithm of worker termination– When thrashing seems to occur, hisapd

terminates workers that have not been requested recently.

2006/07/12 17

Scheduling algorithm (cont.)

Conditions for which hisapd judges that thrashing seems to occur– A swap-in occurs.– A swap-out occurs.– Memory use is 99% or more.

Conditions for which hisapd chooses workers to terminate– The worker is active.– The worker is not recorded in the most recent

10,000 requests.

2006/07/12 18

Evaluation

Experimental environmentsNetwork

Switching HubDELL PowerConnect 2724

1000 BASE-T x 24

Server

CPUAMD Opteron 240EE

1.4 GHz x 2

Memory 4 GB (swap 8 GB)

OSFedora Core 4

(kernel 2.6.14)

NICBroadcom BCM5704C

1 Gbps

Client

CPUIntel Pentium III Xeon

500 MHz x 4

Memory 256 MB (swap 512 MB)

OSFedora Core 4

(kernel 2.6.14)

NICIntel PRO/1000XT

PWLA8490XT 1 Gbps

Gigabit Ethernet Gigabit Ethernet

2006/07/12 19

Evaluation (conf.)

Basic performance evaluation– We evaluated the basic performance in processing

dynamic content. Scalability evaluation

– We evaluated the scalability of the number of partitions in a server in processing dynamic content.

☆ Target content– We sent requests to a PHP script that calls phpinfo().

The script displays the system information of the PHP language processor. (40 KB per request)

2006/07/12 20

Basic performance evaluation

Aim– to determine useful performance of our system

Systems for comparison– Apache– One-to-one

It uses networks with a reverse proxy, and has a dispatcher and many workers that are dedicated to process requests for each partition.

Although it is similar to our system, mod_hisap and hisapd are not installed.

– Apache with suEXEC Benchmark

– httperf benchmark ver. 0.8

2006/07/12 21

Basic performance evaluation (cont.)

The system loses an avg. of 28.0% of the throughput relative to Apache.– The overhead of the system is because of a reverse proxy.

However, the system has high throughput relative to suEXEC. The system loses an avg. of 1.0% of the throughput relative to One-to-one.

– The overhead of mod_hisap & hisapd is very low.

0

100

200

300

400

500

600

700

800

100 200 300 400 500 600 700 800 900 1000Request frequency (#N/ s)

Thr

ough

put

(#N

/s)

ApacheOne-to-oneHi-sapsuEXEC

2006/07/12 22

Scalability evaluation

Aim– to determine the effectiveness of Content Acc

ess Scheduler Comparison system

– One-to-one mod_hisap and hisapd (Content Access Schedule

r) are not installed.

Benchmark– Apache benchmark ver. 2.0.41-dev

2006/07/12 23

Scalability evaluation (cont.)

Our system’s scalability is high.– The throughput decrement due to an increase in the number of partitions was low.

For One-to-one, the OS crashed due to a memory shortage when the number of partitions was 600.

0

50

100

150

200

250

300

350

400

450

500

100 200 300 400 500 600 700 800 900 1000Number of partitions (#N)

Thr

ough

put

(#N

/s)

Hi-sap

One-to-one

2006/07/12 24

Scalability evaluation (cont.)

The swap use of One-to-one dramatically increases due to an increase in the number of partitions. – This is the reason of the OS crash.

Our system does not use swap space as much because of Content Access Scheduler.

0102030405060708090

100

100 200 300 400 500 600 700 800 900 1000Number of partitions (#N)

Mem

ory

use

(%)

One-to-one memoryHi-sap memoryOne-to-one swapHi-sap swap

2006/07/12 25

Comparison of approachesSecurity in a Server

Basic Performance

Scalability Generality

Apache very poor excellent good good

suEXEC & POSIX ACL good very poor good good

　 Sandbox / VM　 excellent excellentpoor /

very poorgood

PHP safe mode good excellent good very poor

Apache perchild MPM good － poor good

One-to-one good good poor good

Harache good poor good good

Hi-sap 　 excellent good good good

2006/07/12 26

Conclusions

Proposal: Hi-sap– Secure and high-performance web server

system Implementation:

– On a Linux OS with SELinux. Achievement:

– High performance– High scalability

2006/07/12 27

Future Work

Creating various Content Access Schedulers– for wiki– for weblog– for CMS, etc.

Evaluating these schedulers

2006/07/12 28

Thank you.

Any questions/comments?