mod-geocache / mapcache - A fast tiling solution for the apache web server
-
Upload
tbonfort -
Category
Technology
-
view
13.323 -
download
1
description
Transcript of mod-geocache / mapcache - A fast tiling solution for the apache web server
14 / 09 / 2011 Thomas Bonfort (Terriscope) Stephen Woodbridge (iMaptools)
MapServer MapCache (f.k.a mod-geocache)
A Fast Tile-Caching Solution for the Apache HTTP Server
What’s a tile cache?
Services
Cache
Tileset
Source
Grids requests
Yet Another Tile Cache ?
• Mature / feature-full solutions already out • High performance needed • Started out as a small project, to validate the
concept • Integrated into the MapServer stack for next
release
Apache module
• Module = code run by the apache processes that treat requests
• Native code • No overhead (e.g. CGI, FastCGI) • Does not require spawning an interpreter per
concurrent request • Caveats:
– Thread/process synchronization – Memory management – Security
Features
• Metatiling – Cross process/thread locking ensures you
can enable metatiling on an unseeded tileset
• Image recompression / optimization – JPEG quality – PNG compression level – PNG quantization – “Mixed” format: PNG+JPEG
Features (cont)
• Expiration handling – HTTP cache controlling headers – Last-Modified, If-Modified-Since – Automatically regenerate stale tiles
• Custom error reporting – Message – Empty image – Status code (+optional HTTP header)
• Watermarking
Features (cont)
• Dimensions – Single values – Intervals – Regular Expressions
• FeatureInfo forwarding • FastCGI / CGI option for use outside of apache
Services
• TMS • WMTS • WMS / WMS-C • Google Maps XYZ • VirtualEarth Quadkeys • KML SuperOverlays
“Demo” Interface
• Lists all active services • Easy way to check configured caches • Simple OpenLayers slippy-map • Cut-and-paste definitions for Layers • http://localhost:8081/mapcache/demo
Grids
• Extent • Projection • Resolution per level • Tile size
• Comes preconfigured with popular grids • Supports grid aliases • For limited areas, use grid subsets, not your
own grid!
Data Sources
• Extensible, anything able to provide an image for given: – Width, Height – Extent – SRS – Optionally dimension
• WMS is the only implemented source – Custom query parameters – Custom headers
Caches
• Extensible mechanism, backend must provide api to get/set a tile for a given grid, x, y, z (+dimension)
• Higher level locking mechanism allows on-demand cache generation
• Backends provide different performance / manageability tradeoffs
• Currently implemented: filesystem, sqlite, memcached
Disk based caches
• Tilecache hierarchy: – /tmp/osm/g/17/000/027/304/000/081/334.png
• Custom hierarchy: – /tmp/{tileset}/{grid}/{x}-{y}-{z}.{ext}
• Support for symlinking blank tiles • Watch out for filesystem limitations !
– Max files per directory – Max number of inodes – Blocksize
Sqlite caches
• Store tile data as blobs in sqlite db • Slower than disk caches, but prevents
filesytem headaches • Flexible storage options:
– Provided custom schema – MBTiles schema – Custom schema, provide your own queries:
• select tile_data from tiles where tile_column=:x and tile_row=:y and zoom_level=:z");
• insert or replace into tiles(tile_column,tile_row,zoom_level,tile_data) values (:x,:y,:z,:data)"
Vertical Assembling
Save bandwidth, request a single tiled layer !
&LAYERS=OSM&… &LAYERS=NEXRAD&… &LAYERS=OSM,NEXRAD&…
Horizontal Assembling
Support non-tiled clients
Tile Assembling
• CPU bound operation: image format (PNG/JPEG) encoding and decoding
• CPU acceleration (MMX,SSE,…) of pixel manipulation operations (scaling, blending)
• Configurable resampling • No reprojection support • Missing spec for TMS and WMTS support
Proxying support
• Transparently add tiling / fast WMS support to existing services
• Intercepts GetTile / GetMap requests • Configurable forwarding to other services
based on request parameters
Seeder
• Use multiple threads to load the source WMS • Reseed tiles older than a specified date • Seed only tiles inside given geometry
– OGR for data access: filter based on SQL queries, e.g. FIPS_A1=’USA’ , pop_density>1000
– GEOS Prepared Geometries for fast intersection calculation
• Delete mode • Specify dimension values
Benchmarks
• Server: 4-core, 8GB ram, SSD storage • ab tool used over Gigabit Ethernet • “warm” filesystem • All requests on exact same image data
TMS requests / sec
Throughput very dependent on image size
Concurrent requests Geowebcache MapProxy
wsgi MapCache
1 1 280 800 1 780
10 17 000 9 000 30 000
20 18 000 8 000 40 000
100 17 000 7 100 40 000
250 16 500 6 800 40 000
SQLite backend, TMS
• MBTiles backend • 10 threads:
– TileStream: 900 – MapCache: 8300
Horizontal assembling
• Workload: ~20 png decompressions + 1 jpg compression
• 10 threads • With bilinear resampling
– MapProxy: 29 – MapCache: 43
• With nearest neighbor resampling – Geowebcache: 35 – MapProxy: 40 – MapCache: 48
Vertical assembling
• Workload: 2 png decompressions, 1 jpeg compression
• 10 threads: – Geowebcache: unsupported – MapProxy: 435 – MapCache: 640
What’s next?
• MapServer Stack integration – Docs – Code sharing (image I/O, build tools)
• Ideas: – Source assembling at the tileset level – OGC ServiceExceptions – HTTP options:
• HTTPS • Authentication • POST
Want More?
• Soon up to date project page: http://www.mapserver.org/trunk/mapcache/index.html
• Soon defunct project page: http://code.google.com/p/mod-geocache/
• Get the code: – Mapserver trunk svn – Mapserver 6.2
• Contribute ! • Twitter: @tbonfort