Caching Maps and Vector Tile Layers: Best Practices€¦ · -spatial index, compact FGDB, copy data...
Transcript of Caching Maps and Vector Tile Layers: Best Practices€¦ · -spatial index, compact FGDB, copy data...
Tommy Fauvell
@CartoRedux
Caching Maps and Vector Tile Layers: Best Practices
Session Goals
• Introduction to Caching and terminology
• Differences between raster tiles and vector tiles
• Sharing / Publishing best practices
• Most efficient way to generate cache
• Framework for tackling basemap projects
Cache smarter, not harder…
• Informative
• Responsive
• Flexible
- Combine with other data / basemaps
- Usable across various clients (mobile / web / desktop)
• Current with timely updates
• Cartographically rich
Fast and responsive maps keeps eyeballs on your content and users engaged
Overview | What makes a good web map?
WebmapFile
Shapefile
Tables
Raster
ArcGIS Data Store
Dynamic Data can take a long time to render
Caching is an investment of time and infrastructureto deliver cartographically rich data in an efficient manner
Tile Cache Webmap
Raster
ArcGIS Data Store
File
Shapefile
Tables
Overview | Raster Tiles and Vector Tiles
Raster Tiles
• Pre-rendered snap-shots of your data
- JPEG’s, PNG’s, LERC
• Smart tile types:
- Mixed and PNG
• Most flexible client support
• Compute and storage intensive
Vector Tiles
- Tiled containers of data
- Features and attributes
- Rendering instructions
- Style file
- Additional map components
- Fonts and Icons
• Very fast to cook at a fraction of raster tile storage costs
• Only supports VECTOR datasets
• Rendered by client devices
- Not ArcMap…
MIXED or JPEG-65Mosaic Dataset footprints
LERC-0.1Mosaic Dataset Footprints
JPEG-90Custom AOI
Vector Tile
ArcGIS Server ArcGIS Enterprise ArcGIS Online
Federated Server Hosting Server
ArcGIS Pro
Overview | Software and Architecture Terminology
Overview | Software and Architecture Terminology
By Reference
You keep using that word…I don’t think it means what you think it means
Navigating the Esri CacheEcosystem
Overview | Cache is everywhere!
Design
• Requirements
• Datasets
• Intended use
• Constraints
• Deadlines / Milestones
Author
• ArcGIS Pro Project
• Prototype
• Finalize design
• Cartography / Multiscale Mapping
Cache
• Publish to Development
• Test Cache
• Generate Cache
• QC
Deploy
• Publish to Staging and validate
• Publish to Production
• Feedback
Overview | Basemap Project Lifecycle
Overview | Raster Tiles - Create Map Tile Package
• Improved at Pro 2.5
- Parallel Processing Factor
• Good for:
- Direct to TPK / TPKX
- Regular extents
• Limitations:
- Uses envelope extents vs detailed AOI
- No partial updates
Overview | Raster Tiles - Manage Tile Cache
• Improved at Pro 2.5
- Better use of Parallel Processing Factor
• Good for:
- ArcGIS Server or Enterprise unavailable
- AOI / irregular extents
- Partial updates
• Warning:
- Uses current map extent!
Overview | Raster Tiles - Manage Map Server Cache Tiles
• Good for:
- Caching on a deadline
- Most scalable option
- AOI / irregular extents
- Partial updates
- Doesn’t tie up workstation / runs asynchronously
• Warning:
- Will make servers cry, scale responsibly
Overview | Vector Tiles – Create Vector Tile Package
• Good for:
- DIY approach to creating vector tile tilesets
- Short to no-notice deadlines
- Mobile / offline support
- Only uses local system resources (single process)
- Does not require parallel processing
• Warning:
- Modern web browsers and supported clients
- Not ArcMap
- Requires ArcGIS Enterprise or ArcGIS Online
Overview | Vector Tiles – Create Vector Tile Index
• Good for:
- DIY approach to creating vector tile tilesets
- Use first for complex and vast basemap projects, then use the output in Create Vector Tile Package
• Warning:
- Don’t modify max vertex count…unless you have a really good reason
Cache smarter not harder…
Optimizing Cache Generation
Optimizing | Data
• Use a local FGDB copy / extract of your data
• Clean your data
- Eliminate duplicates
- Check/fix geometry errors
• How dense is your data?
- Set reasonable scale dependencies
- Generalize
Optimizing | Cartography
• Don’t start from scratch, import your MXD
• Set your scales according to the tiling scheme you select
• Remember scale logic in Pro is different from ArcMap
• Convert representations to unique value symbols
• Limit…
- number of layers
- duplication of content
- inclusion of additional fields / data in the tileset
Optimizing | Cartography
• Avoid…
- group layers
- complex symbols and unsupported symbol effects: hatched / gradient fills
- unsupported layer types: annotations, basemaps
• Be mindful of users that want to re-style your maps
Cache smarter not harder…
Optimizing Raster Tile Cache Generation
• Don’t use Fine, Verbose, or Debug logging.
• Size your Caching Tools Instances:
- N = # of cores per machine
- Min and Max = N
- 2 – 4GB of RAM x N
- Decrease N if necessary
Raster Tiles | ArcGIS Server Configuration
• 10.7+
• Cached map services are good candidates for shared instances
• Most popular dynamic map services as shared instances could be considered for caching
• Monitor usage: server statistics / logs
• User feedback / complaints
Shared Instances
Raster Tiles | ArcGIS Server Configuration
• Only cache what is necessary
• Use AOI’s with decreasing coverage as you increaseLOD’s
• Break your basemap project into multiple cache jobs by bracketing LOD’s
- Each job can / should have a unique AOI
• Only update what has changed
- You don’t need to re-cache everything if you have partial updates to your data
147M72M36M18M9M4M2M1M
577K288K144K72K36K18K9K4K2K1K
147M72M36M18M9M4M2M1M
577K288K144K72K36K18K9K4K2K1K
L00L01L02L03L04L05L06L07L08L09L10L11L12L13L14L15L16L17
Raster Tiles | Cache Jobs
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
0 2 4 6 8 10 12 14 16 18
Number of Host Machines
Millions of Tiles Per Minute
Actual
Expected (90,000 T/m)
0
60
120
180
240
300
0 2 4 6 8 10 12 14 16 18
Number of Host Machines
Completion Time in Minutes
M900 Blade: Intel Xeon E7-4820, 32 Cores, 256GB Ram
Raster Tiles | ArcGIS Server Scalability
Summary | Optimizing Cache Generation
• Optimize your data:
- spatial index, compact FGDB, copy data local
• Optimize Map and Imagery Projects:
- analyzer results, scale dependencies, Maplex when needed, Mosaic Dataset tuning
• Configure ArcGIS Server Caching instances
• Optimize raster tile cache jobs:
- Multiple jobs, AOI per LOD’s per job, only cache what is necessary
• ArcGIS Server will scale and leverage system resources for raster tile generation
• Vector Tiles are created using local resources not ArcGIS Server
Vector Tiles
Vector Basemap
• Integrated Sharing Workflow:
- Share - Publish Web Layer
- Copy all data
- Vector Tiles
- Configure parameters
- Cooks, uploads, publishes the hosted tile layer
Caching | Vector Tiles
Author Map CachePublish Hosted
Tile Layer
• Geoprocesing Tools:
- Create Vector Tile Index
- Create Vector Tile Package
- Share Package (publish tile layer option*)
- Publish hosted tile layer (Web)
Raster Tiles
Vector Basemap
• Publish then Cache
• Share – Publish Web Layer
- Configure cache parameters*
- Always “cache manually”
• Cache
- Manage Map Server Cache gptool
Caching | Vector as Raster Tiles
Author Map
Publish Cache
Cooking | Manage Map Server Cache Tiles
• Only cache what is necessary
• Use AOI’s with decreasing coverage as you increaseLOD’s
• Break your basemap project into multiple cache jobs by bracketing LOD’s
- Each job can / should have a unique AOI
• Only update what has changed
- You don’t need to re-cache everything if you have partial updates to your data
147M72M36M18M9M4M2M1M
577K288K144K72K36K18K9K4K2K1K
147M72M36M18M9M4M2M1M
577K288K144K72K36K18K9K4K2K1K
Raster Tiles | Cache Jobs
ArcGIS Pro
Imagery Project
• Publish then Cache
• Share – Publish Web Layer
- Initiated from Mosaic Dataset / Raster Dataset
- Configure cache parameters
- Always “cache manually”
• Cache
- Manage Map Server Cache gptool
Caching | Imagery as Raster Tiles
Author Map or Mosaic Dataset
Publish Cache
ArcGIS Pro
Terrain Layer Project
• Publish then Cache
• Share – Publish Web Layer
- Initiated from Scene and TOC
- Configure cache parameters
- Always “cache manually”
• Cache
- Manage Map Server Cache gptool
Caching | Terrain as Raster Tiles
Author Scene
Publish Cache
Summary• Design:
- Operational / base map content
- Clients (desktop/ mobile)
- Updates
• Caching:
- Use Pro for Authoring basemaps
- Vector tiles for vector content
- Raster tiles for imagery/hill shades
- Raster tiles for vector data (if your clients do not support WebGL)
- Use hosted feature service for information
• Optimize cache generation
- Keep data close
- Tune platform / simplify data
- Use mosaic dataset for imagery
Print Your Certificate of Attendance
Print Stations Located in 150 Concourse Lobby
Tuesday12:30 pm – 6:30 pmExpoHall B
5:15 pm – 6:30 pmExpo SocialHall B
Wednesday10:45 am – 5:15 pmExpoHall B
6:30 pm – 9:30 pmNetworking ReceptionSmithsonian National Museumof Natural History
Download the EsriEvents app and find
your event
Select the session you attended
Scroll down to “Survey”Log in to access the
surveyComplete the survey and select “Submit”
Please Share Your Feedback in the App
And maybe some answers?
Questions?