IWMW 1997: WWW Caching
30
Transcript of IWMW 1997: WWW Caching
Overview of Presentation
• Why caching?• Caching Infrastructures.• National Caching.• Caching hardware and software• Implementation of caching• Non-Technical Issues
Why Caching?
• 1000s of users ‘surfing’ the Internet each with their own browser.
• Users and browsers are ‘independant’ resulting in a large amount of replication of information carried over the network.
• Popular Web sites may have many simultaneous connections transmitting identical copies of a single item over the same network trunk routes. This state of affairs is highly undesirable because...
Why Caching?
• Bandwidth - especially international bandwidth - is very expensive, and must be used cost-effectively.
• Web ‘hot-spots’ are created.
• Web object retrieval times are increased.
Why Caching?
• Caching, or Web Caches are an attempt to:
– Minimise bandwidth wastage.
– Decrease object retrieval times.
– reduce number of ‘Hot-Spots’
Caching Infrastructures
• Caches may be implemented:
– Within departments– Within Institutions– Nationally– Internationally
• Caches can co-operate. So we have meshes of caches or caching infrastructures.
Caching Infrastructures
• Caching infrastructures are developing at every level.
– Quite a few departmental caches.– Many Institutions now operate
caches.– Within the UK a National caching
infrastructure is developing.– International infrastructures in
place and developing.
Caching Infrastructures
• Cooperation between caches.– Achieved by the ICP cache
communication protocol in one of two modes:
• Unicast mode - individual connections established to interrogate caches.
• Multicast mode - an ICP multicast packet is ‘multicast’ to a group of cooperating caches.
– Intuitively the multicast approach should be more efficient - reduce bandwidth, etc.
Caching Infrastructures
• For example at Manchester:
– Central campus cache and several departmental caches use it in unicast mode.
– Parent relationships with other caches in the UK, Europe and USA.
National Caching
• HENSA pioneered caching with their Public Caching Proxy Server. Initiated around 1992.– Used Lagoon initially– Then the CERN server– Then Netscape Proxy– And some Squid
• Details described at First International WWW Conference: http://www.hensa.ac.uk/www94
National Caching
• The existing service is hosted by University of Kent at Canterbury and University of Leeds.
• From 1st August 1997 it will be hosted by the University of Manchester and Loughborough University.
• Selection by a recent competitive tendering process.
National Caching
• The situation so far.
– Service still at HENSA and Leeds. We are preparing for the transition.
– Initially exisiting equipment will be used.
– Projection of demand performed and hardware upgrade path budgeted for.
National Caching
• The ‘new’ service will have:– a service ‘arm’– a development ‘arm’
• The National service will be directed by a steering committee and will be, as far as possible, user driven.
• National Caching Web site, regular newsletter, mailing lists, help desk system, fault reporting mechanism, etc, etc.
Benefits of National Caching
• Trans-Atlantic bandwidth and bandwidth to Europe are both very expensive and in great demand. Caching reduces bandwidth consumption. The resulting cost savings can be used to fund other things.
• Faster document retrieval time - in theory!
National Caching - Useful addresses and URLs
• Email addresses:– [email protected]
general mailing list for users.– [email protected] mailing
list for Special Interest Group.– [email protected], National
Cache Joint Team mailing list.
• Some URLS:– http://www.hensa.ac.uk– http://www.net.lboro.ac.uk/caching/– http://www.mcc.ac.uk/Cache/
Caching Hardware
• Any Unix platform• Linux• FreeBSD
Caching Software
• Lagoon• CERN• Netscape• Harvest• Squid
Using Caches
• Users interact with caches directly using their favourite browser.
• Caches interact or co-operate with other caches using ICP.
• Browser - cache interaction is a ‘client-server’ type interaction.
Implementation -Browsers
• Netscape– Manual configuration - Select
network preferences from Options menu...
– Automatic configuration - proxy configuration can be automated with Javascript...
• Others: Lynx, Mosaic, Microsoft Internet Explorer.
Implementation - caches
• With reference to Squid– Installation– Configuration– Operations
• Some problems– disk space– discarding documents
Implementation -Installation
• Retrieve from: – http://squid.nlanr.net/Squid/– Decompress and extract.– configure– compile – install
• Operating Systems– Unix, AIX, FreeBSD, HP-UX,
IRIX, Linux, OSF/1, Solaris, SunOS
Implementation - Configuration
• Configuration file– http_port– icp_poty– mcast_groups– Cache_host– cache_host_domain– cache_swap– cache_swap_low– cache_swap_high– cache_dir– cache_access_log
Implemetation - configuration
• Configuration file continued...– pid_filename– debug_options– ftpget_program– negative_ttl
• Access Control lists– http_access allow– htp_access deny
Implemetation - configuration
• Administration parameters– cache_mgr– cache_announce– logfile_rotate– minimum_direct_hops– and so on...
Operation
• Parent or sibling?• Log files• Statistics• Number of requests per day• Machine loading• Network loading• Improvement in latency?• Reduction in bandwidth usage?
Other Issues
• Copyright• Pornography• Log statistsics• Data protection act.
Should I run a cache?
• Should I run a:– Departmental cache?– Institutional cache?
• Should I link together departmental caches?
• Should I link departmental caches to my Institutional cache?
• Should I link my institutional cache to the National Cache?
Should I run a cache?
• There are no hard and fast rules. Clearly caching saves bandwidth and improves latency, but it is not obvious how best to construct a hierarchy to achieve this.
• We are are at the learning stage. Part of the remit of the National Web Network Caching Service will be to investigate this and produce guidelines and recommendations for individual sites.
Should I run a cache?
• The answer is yes!• Consider
– number of users– Type of work– Local Area Network
• loading• Bottlenecks
– Expected demand• Analyse statistics
Futures
• The National WWW Network Caching Service will be involved in the development of caching in the UK. Will investigate hardware and software. Findings will be published on the National Cache Web site:
URL: http://www.wwwcache.ac.uk
