How to pivot with FME and FME

ServerGuido Stein

Awaken your inner ETL superhero!

Guido Stein

Guido SteinGeospatial Data Alchemist Applied Geographics Inc.

Co-ChairFOSS4G Boston 2017

Department of Transportations

vs DataProjects and Surveys and Layers, oh my!

Sources (Digital & Paper)Route Logs

Project PlansRight of Way BookletsTown Road Reports

Feature AttributesFile Source

Feature CreationSource Level

Source Project

Process Management

Attribute Management

Reporting

Output Combined Data

Place your screenshot here

Extract, Translate, Load

FromDGN files

ToMicrosoft SQL

Repeat

Scale

Place your screenshot here

“WITH GREAT POWER THERE MUST ALSO COME--GREAT RESPONSIBILITY!”Amazing Fantasy #15 (August 1962)

ETL Take One:FME Desktop

Building a tool for FME Users to ingest data

Ingest Data

FME Desktop

Good• Flexible• Powerful

Bad• Training Required• No Process Management• Intimidating

● FMW Workspace Files● Published Parameters

Conclusion: FME Desktop ●Great for FME Users●Confusing for everyone else●May lead to broken models●Not scalable●No Process Management (who did what and when)●Scalability tied to desktop liscense

ETL Take Two:Access Front End

Access to expose data and process in MSSQL

Ingest Data

Access Front End

Good• Simpler interface• Standard workflow• Visibility of output• Auditable Processes

Bad• Requires Access• Difficult Configuration• Wonky UI

● Access file● Use Access buttons to call FMW files with command line

parameters

Conclusion: Access Front End●Less intimidating to non FME Users●Good separation between user tools and process

developer tools●Frustrating setup (windows pathing and database

connection fun)●Limited UI●Scalability tied to desktop liscense

ETL Take Three:Web App & FME

Server

Ingest Data

Web App & FME Server

Good• Standard Workflow• Centralized Process

Publishing

Bad• Requires web application

development• Number of users and cores

effects timing

● Web Application● Use Rest Services provided in FME Server

Conclusion: Web Interface ●Web Applications can really standardize business

decisions and workflows●Developers Needed●FME Server is not limited to a specific edition●Scalablabilty based on number of engines you pay

for

WITH GREAT FMEPOWER COMES GREAT RESPONSIBILITY!

FME models can be scary for end users. Building a front end interface will empower end users with the tools you create. Many solutions exist, try them all.

Awaken Your Inner ETL Superhero!

Guido Steingstein@appgeo.com

@guidos

August 14-19, 2017Seaport Hotel & World

Trade CenterBoston, MA

Thank You Safe Software for being a Bronze

Sponsor