Wcw2004 Overhaul
-
Upload
mahanbhavsar143 -
Category
Documents
-
view
231 -
download
0
Transcript of Wcw2004 Overhaul
-
8/9/2019 Wcw2004 Overhaul
1/25
Overhaul: Extending HTTP to
Combat Flash CrowdsJay A. Patel & Indranil Gupta
Distributed Protocols Research Group
Department of Computer Science
University of Illinois at Urbana-Champaign (UIUC)
Urbana, Illinois, USA
-
8/9/2019 Wcw2004 Overhaul
2/25
-
8/9/2019 Wcw2004 Overhaul
3/25
Distributed Protocols Research Group, Department of Computer Science, University of I ll inois at Urbana-Champaign 3
Example: MSNBC
MSNBC home pageDecember 14, 2003
-
8/9/2019 Wcw2004 Overhaul
4/25
Distributed Protocols Research Group, Department of Computer Science, University of I ll inois at Urbana-Champaign 4
Motivation Problem
Unpredictable, yet frequent
Brief period of time
Thousand-fold increase in traffic
Two nave solutions
Overly insure on resources Shut down web site
-
8/9/2019 Wcw2004 Overhaul
5/25
Distributed Protocols Research Group, Department of Computer Science, University of I ll inois at Urbana-Champaign 5
Current Solutions Architectural Changes
SEDA
Capriccio ESI
Protocol Modifications
DHTTP
Web Booster
Cooperative Sharing
Squirrel
Kache Backslash
BitTorrent
-
8/9/2019 Wcw2004 Overhaul
6/25
Distributed Protocols Research Group, Department of Computer Science, University of I ll inois at Urbana-Champaign 6
HTTP: RegularInteraction
Client Server
GET RequestResponse
DocumentHeader
-
8/9/2019 Wcw2004 Overhaul
7/25
Distributed Protocols Research Group, Department of Computer Science, University of I ll inois at Urbana-Champaign 7
Overhaul: Overview Protocol change
HTTP extension, no modification 5 new tags added, 1 slightly modified
Backwards compatible Key concept: chunking
Characteristic of the web applied to individual documents
m chunks per document
P2P distribution framework Voluntary
Ad hoc, not DHT based
Key benefit:parallel resource discovery
-
8/9/2019 Wcw2004 Overhaul
8/25
Distributed Protocols Research Group, Department of Computer Science, University of I ll inois at Urbana-Champaign 8
Overhaul: Design
Client
Server
Client
Client
Client#1
#2
#4
#3
HTTP Request
with Overhaul support tag
Chunked Response
with Overhaul headers
Peers exchange chunks
to fetch the complete document
Ad hoc peer network
-
8/9/2019 Wcw2004 Overhaul
9/25
Distributed Protocols Research Group, Department of Computer Science, University of I ll inois at Urbana-Champaign 9
Details: Client/ServerInteraction Initial request by client
Supports: Overhaul $port $speed
Response by server in Overhaul mode
ith chunk transmitted in sequential order
Signatures of otherm-1 chunks for verification
Initial Overhaul network membership list n most-recent Overhaul clients
List maintained at server (updated with every request)
-
8/9/2019 Wcw2004 Overhaul
10/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 10
Details: Peer Clients Interaction Clients contact other peer members
To fetch remaining chunks
To discover new peers
Aggregate membership list by swapping information
1-hop random walk discovery process
Resource discovery
Lookup documents on a busy Overhaul server
Contact peers randomly on membership list
INFO $host.tld
-
8/9/2019 Wcw2004 Overhaul
11/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 11
Implementation Server
Apache/2.0 HTTP server
Module:mod_overhaul
Client
Java HTTP Proxy
Cross platform Universal client support
-
8/9/2019 Wcw2004 Overhaul
12/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 12
Testing Methodology: Server Server machine
2.5 GHz AMD Athlon XP+
1 GB RAM Client machine
650 MHz Pentium III
320 MB RAM
Same network equipment
25 concurrent fetches
ApacheBench utility
-
8/9/2019 Wcw2004 Overhaul
13/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 13
Results: Chunking (Fixed Size)Document: 10 KB
Concurrency: 25
Regular HTTP
512-byte chunks
2048-byte chunks
Overhaul mode requires the server to send only a single chunk
-
8/9/2019 Wcw2004 Overhaul
14/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 14
Results: Chunking (Maximum Count)
Regular HTTP
6 chunks
12 chunks
24 chunks
Document: 50 KB
Concurrency: 25
-
8/9/2019 Wcw2004 Overhaul
15/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 15
Results: Overhaul vs. Regular
Regular HTTP
6 chunks
12 chunks
Concurrency: 25
Minimum chunksize: 512-bytes
-
8/9/2019 Wcw2004 Overhaul
16/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 16
Testing Methodology: Client Cluster of workstations
25 homogenous PCs
2.8 GHz Intel Pentium 4
1 GB RAM
Same network equipment
Two experiments
Concurrent: single document
Staggered: multiple documents
-
8/9/2019 Wcw2004 Overhaul
17/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 17
Results: Single Document Large document: 50
KB (12 chunks)
Server condition: 150-250 concurrent fetches
+ competition
Overhaul requests:
concurrently only using 24
Overhaul-aware clients
Regular
requests
Overhaul
mode
Fastest 1 sec 6 secs
Slowest 32 secs 9 secs
Average 9 secs 7 secs
Server bandwidth usage in Overhaul mode: 1/12th of regular requests
-
8/9/2019 Wcw2004 Overhaul
18/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 18
Results: Multiple Documents 8 documents: 110 KB total
(12 chunks)
Server condition: 150-250
concurrent fetches +
competition
Overhaul requests staggered
1st stage: 12 concurrent
fetches, fetch all documents 2nd stage: 12 concurrent
fetches, fetch index document
only
Regular
requests
Overhaul
mode
Fastest 1 sec 14 secs
Slowest 28 secs
Average 23 secs* 18 secs
Server bandwidth usage in Overhaul mode : 1/18th of regular requests
* indicates completed requests
-
8/9/2019 Wcw2004 Overhaul
19/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 19
Limitations Both client and server must be Overhaul
aware
Requires critical mass to be maintained toremain effective
n clients > m chunks
More responsibilities for the client Possible security implications
-
8/9/2019 Wcw2004 Overhaul
20/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 20
Conclusion Saves resources
Bandwidth
The bigger the crowd, the lower the per capita usage Response time
Faster turnaround for both server and client
Getting wide spread acceptance
Marginal cost Protocol extension requires industry and
standards push
-
8/9/2019 Wcw2004 Overhaul
21/25
-
8/9/2019 Wcw2004 Overhaul
22/25
-
8/9/2019 Wcw2004 Overhaul
23/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 23
Heterogeneous Networks Problem
Connections are
heterogeneous
Solutions
Clustering of clients
Super nodes
Client
Client
Client
Client
-
8/9/2019 Wcw2004 Overhaul
24/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 24
Document Selection Only a partial set of
documents are affected
by a flash crowd Must implement
selective Overhaul
mode
Automatic selection Active monitoring
Server
Large collection of documents
reside on the server
Documents fetched
by a flash crowd
-
8/9/2019 Wcw2004 Overhaul
25/25
Distributed Protocols Research Group, Department of Computer Science, University of Ill inois at Urbana-Champaign 25
Dynamic Documents Flash crowds especially frequent during big
events and news
Characteristic: rapidly, changing data
Solutions
Time stamping
Expiration of chunks Inter-network refresh from peers