Interface Control Document Mobilize Maintenance M20 with ...
Transcript of Interface Control Document Mobilize Maintenance M20 with ...
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 1 of 18
Interface Control Document Mobilize Maintenance M20 with Esri Portal
Table of Contents Section 1 Document Information ............................................................................................................................................ 3
1.1 Information .................................................................................................................................................................... 3
1.1.1 Functional Requirements – Data Management and Reporting on M-20 ............................................................... 4
1.1.2 Functional Requirements – Data Exporting ............................................................................................................ 5
1.1.3 Functional Requirements – User Interface ............................................................................................................. 5
1.1.4 Functional Requirements – System Interfaces ....................................................................................................... 5
1.1.5 Functional Requirements – System Access and Security ....................................................................................... 6
1.2 Version History ............................................................................................................................................................... 6
1.3 References ..................................................................................................................................................................... 6
1.4 Relevant Glossary........................................................................................................................................................... 7
Section 2 Integration Conceptual View ................................................................................................................................... 7
2.1 Diagram and Description ............................................................................................................................................... 7
2.2 Products, Platforms and Software Matrix ..................................................................................................................... 7
Section 3 Logical View.............................................................................................................................................................. 8
3.1 Diagram and Description ............................................................................................................................................... 8
3.2 Portal Content and Web Services .................................................................................................................................. 8
3.2.1 M20 Feature Services ............................................................................................................................................. 8
3.2.2 Portal Content ......................................................................................................................................................... 9
3.3 ETL ................................................................................................................................................................................ 10
3.3.1 Data Integration: Overview .................................................................................................................................. 10
3.3.2 M20 Data: Import from PMSS to GIS & Update (Tools A, B & C) ......................................................................... 11
3.3.3 M20 Data: Import from ArcGIS Enterprise Geodatabase to PMS Staging Table (Tools D & E) ............................ 14
3.3.4 M20 Data: Import from PMS Staging Table into PMS M20 Tables (Tool F) ......................................................... 16
Section 4 Physical View .......................................................................................................................................................... 17
4.1 Diagram and Description ............................................................................................................................................. 17
Section 5 Assumptions and Risks ........................................................................................................................................... 18
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 2 of 18
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 3 of 18
Section 1 Document Information This document includes the systems integration information and implementation in order to facilitate the flow of
data between PMSS (Pavement Maintenance Scheduling System), field collected data managed by Esri Portal
internally hosted by VDOT, and PMS (Pavement Management System). Generally, Systems integration is
defined as facilitating the communication of information between 2 or more different applications, systems,
platforms and/or products. This document includes the details for the following types of components in an effort
to provide sufficient to identify the method of information flow among systems:
• Web service integrations
• Data integrations
• Mobile field data integrations
However, this document does not include information for the following components as they are considered
Universal across most applications:
• User-specific logins
• Database connections made by an application to perform CRUD transactional operations as these
connections do not qualify as integration efforts among different systems
1.1 Information
The Pavement Management program requires a technology based solution for agency-wide data collection and
management system whose primary function is to expedite and streamline the process of collection, storage and
reporting on M-20 data.
To accomplish the above articulated need, VDOT requires an agency-wide data collection and management
system that includes the following functionality:
• M-20 Data Management Services
o tracking pavement maintenance work completion and associated business data (paving location,
material layers and thicknesses, contract data, and field notes)
o pre-populate pavement maintenance and associated pavement material layer data based on
pavement maintenance schedule contract data available in PMSS
o allow update to planned work location, material layers and thickness, contract data, and field
notes
o allow capture of unplanned or non-PMSS pavement maintenance work through addition of new
records during field collection or office data entry
o house incomplete or planned maintenance work records with a process for adding records to
primary database (PMS) when record is indicated complete
o load GIS data into PMS staging tables
o utilize mobile technologies for data collection
o capacity to export business data to support District information needs (e.g. planned work
completion vs. remaining)
• Exporting Capacity
o capacity to select from map and export the associated data
o access to image files and reports
o capacity to generate a map output of current extent
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 4 of 18
• User Interface
o provide a visual indication of required data elements in online screens
o provide a save option for data editing
• System Interfaces
o Pavement Management System
o Pavement Maintenance Scheduling System
o Portal for ArcGIS
o Collector for ArcGIS
o WebApp Builder
• System Access and Security
o Integrated technical safeguards to ensure a level of privacy and security, including
Administrator controlled user name and password access
Administrator controlled user level read, write, edit and delete capabilities
o Capable of supporting 25 concurrent users (collection at multiple paving locations across
multiple Districts, with simultaneous data entry/review in office )
1.1.1 Functional Requirements – Data Management and Reporting on M-20
Req. ID Requirement Definition
FR2.0 The system shall provide the ability to manage maintenance treatment
material layer and pavement schedule data as captured on VDOT Form
M-20
FR2.0.1 The system shall provide functionality to support communication of
segment specific decision making and details during field collection
within comments fields (e.g. location of misc. patching locations,
comments regarding field decisions)
FR2.1 The system shall provide functionality to pre-populate Form M-20
information based on PMSS contract data for field validation and update
FR2.1.1 The system shall provide opportunity for update of pre-populated
information (including location and business information) as well as
identification of planned work completion
FR2.1.2 The system shall provide opportunity to capture unplanned or non-
PMSS pavement maintenance work through addition of new records
FR2.2 The system shall provide functionality for data selection, identification,
query and exporting from a mapping interface.
FR2.3 The system shall provide functionality for data selection, identification,
query and exporting from a graphic interface.
FR2.4 The system shall provide method for data transfer of pre-populated
and/or field validated Form M-20 information into PMS pavement work
history
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 5 of 18
Req. ID Requirement Definition
FR2.4.1 The system shall store incomplete or planned work records with a
process for adding records to primary database (PMS) when record is
indicated as complete. This process will developed with consideration
of possibility for future PMS system functionality to allow pre-
population of “shell” M-20 directly from PMSS
FR2.5 The system shall provide the capability to convert data stored against
VDOT LRS into GIS and vice versa
FR2.5.1 The system shall represent VDOT pavement maintenance segments as
line features based on associated route geometry
FR2.6 The system shall provide the functionality for data entry, update, and
review through mobile technology
1.1.2 Functional Requirements – Data Exporting
Req. ID Requirement Definition
FR3.0 The system shall provide capability to select from map and export data
FR3.1 The system shall provide user access to image files and reports
FR3.2 The system shall provide ability to generate a map output from current
extent
1.1.3 Functional Requirements – User Interface
Req. ID Requirement Definition
FR4.0 The system shall provide a visual indication of required data elements
on screen in online and mobile screens during user data entry
FR4.1 The system shall provide a save option for data editing
FR4.2 The system shall provide a tool to collect data in the field
1.1.4 Functional Requirements – System Interfaces
Req. ID Requirement Definition
FR5.0 The system shall interface with the Pavement Management System to
provide completed M-20 data for import to PMS pavement work history
FR5.1 The system shall interface with the Pavement Maintenance Scheduling
System to capture planned pavement maintenance work for pre-
population of M-20 data for field validation, update, and approval.
FR5.2 The system shall interface ArcGIS Portal to exchange GIS data across
VDOT firewall
FR5.3 The system shall interface ArcGIS Server for extraction or load into
PMS and PMSS systems
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 6 of 18
1.1.5 Functional Requirements – System Access and Security
Req. ID Requirement Definition
FR6.0 The system shall provide integrated technical safeguards to ensure of
level of data privacy and security
FR6.0.1 The system shall provide Administrator controlled user name and
password access
FR6.0.2 The system shall provide Administrator controlled user level read, write,
edit, and delete capabilities
FR6.1 The system shall support 25 concurrent users (through both office and
mobile field data review, entry, and update)
1.2 Version History Version Date Author Description
1.0 3/24/17 Courtney Moore Beginning draft document
1.1 3/29/17 Rachel O’Neil Technical, logical and physical architecture
1.2 3/29/17 Matt McCracken Review
1.3 3/29/17 Rachel O’Neil Updated an error 1.4 3/29/17 Will Duke Updated functional requirements
1.5 5/19/17 Courtney Moore Splitting M20 from Pavement Core
1.6 6/12/17 Courtney Moore Updated ICD 1.7 6/22/17 Courtney Moore Updated ICD
1.8 9/21/17 Jarrett Green Updated ICD diagrams
1.9 9/25/17 Jarrett Green Updated ICD diagrams
1.10 9/27/17 Will Duke Review
1.11 10/1/17 Jarrett Green Updated ICD
1.12 10/30/17 Jarrett Green Updated Section 3.2
1.3 References Reference Name Link Description
<<Project Solution Architecture Document>>
Link to the SA Document Library [both document link and project View]
Will contain overall Project Architecture information if the integration is being created as a result of a project.
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 7 of 18
1.4 Relevant Glossary
Glossary Item Link
Section 2 Integration Conceptual View
2.1 Diagram and Description
The M20 data on ArcGIS Server, which will reside in locally hosted Portal for ArcGIS, will provide feature services for field
collection of data and will be used to update PMS. An ETL process initiated by an administrator will import data from
PMSS into the ArcGIS Enterprise Geodatabase in order to provide data for field crews to update through ArcGIS Server,
although they will also have the ability to create new records (records that did not originate in PMSS).
2.2 Products, Platforms and Software Matrix
No. Name Version Vendor Name Purpose Lifecycle Status
License Status
1 VDOT PMSS 5.172 VDOT Supports Pavement Maintenance Scheduling and the creation of contracts
Static Licensed
2 VDOT PMS 7.0.4.0 VDOT Provides Pavement management by collecting and retaining information on pavement management activities
Static Licensed
3 Portal for ArcGIS 10.5 Esri Will host WebMaps and WebApps to aid in field collection and to allow the viewing and editing of data via in office applications
Static with scheduled updates
Licensed
4 ArcGIS Server 10.3.1 Esri To provide feature services to facilitate and store geospatial and business data
Static with scheduled updates
Licensed
5 ArcGIS Enterprise Geodatabase
10.3 Esri All data will be stored in this GIS database
Static with scheduled updates
Licensed
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 8 of 18
Section 3 Logical View
3.1 Diagram and Description
The below sequencing diagram depicts a logical starting point for PMSS, Portal and PMS interactions. This diagram shows
the initiating process of exporting data from PMSS into ArcGIS Portal to be exposed as a published feature service. This
published feature service is then consumed by the Field Collector App where data collected is sent through the service
back to Portal. Upon completion of field collection at a given interval, data collected in the field collector and now stored
in Portal is then exported to PMS.
3.2 Portal Content and Web Services The following Portal content and web services are used in the creation and editing of field collected data. Please note
that all item URLs and service URLs are subject to change if/when moved up the environment stack. The feature services
are secure services published to VDOTGISAGSDEV. The M20_editor user is assigned and when the services were added
to Portal, the credentials were stored so users are not prompt when they access the item. VDOT’s system architecture
also allows users to view this service and data outside the VDOT network.
3.2.1 M20 Feature Services
The following feature services are consumed by Collector and the M20 Web AppBuilder.
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 9 of 18
Name: M20_MapService_DEV
Item URL: http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=caf933debc8a4e3f8737cac79c620d9d
Service URL:
http://vdotgisuportal.vdot.virginia.gov/portal/sharing/servers/caf933debc8a4e3f8737cac79c620d9d/rest/services/M20
/M20_MapService_DEV/FeatureServer
Purpose: This feature service contains the paving start and end points that users create in the Web AppBuilder and
Collector. A scheduled python script will generate the reference line between the start and end points and perform
validation against the data.
Name: M20_LRS_DEV
Item URL: http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=7b8b7d7a8df742c9a3d3e0adbc81f417
Service URL:
http://vdotgisuportal.vdot.virginia.gov/portal/sharing/servers/7b8b7d7a8df742c9a3d3e0adbc81f417/rest/services/M20
/M20_LRS_DEV/MapServer
Purpose: This map service contains the LRS layers for users to visualize their location and access route information.
Name: M20_Boundaries_DEV
Item URL: http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=f4edf0f5a20b4f21a629f95fb6ba6b62
Service URL:
http://vdotgisuportal.vdot.virginia.gov/portal/sharing/servers/f4edf0f5a20b4f21a629f95fb6ba6b62/rest/services/M20/
M20_Boundaries_DEV/MapServer
Purpose: This map service contains the District and County layers for users to visualize their location.
3.2.2 Portal Content
The following content is used to display GIS data to end users.
Name: M20 Collector DEV
Item URL: http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=1fb82aaf2f724c52bcf591c2d9b0a84e
Purpose: This web map is used by Collector. It is editable and references the M20_MapService_DEV feature service.
Name: M20 WebMap DEV
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 10 of 18
Item URL: http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=188e2797d9734946af365bcdc51a6e26
Purpose: This web map is used as the input for the web application below. End users will generally not need to access
this map but must be shared so they can access the data in the web app. Layer configurations and pop-ups will be set
here by the item owner.
Name: M20 WebApp DEV
Item URL: http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=2b02c00c481c414fac1eec7322d65a8e
Purpose: This web app is the primary desktop application used by end users. Users can query, edit and export table data
from this application. Users will also verify edits in the field and update any M20 Description records that have errors.
3.3 ETL
3.3.1 Data Integration: Overview
The process to prepopulate, field update, finalize and import M20s from VDOT’s PMSS to the GIS field data collection
tools and into the PMS is described below. This process consists of the following general steps:
1. M20 Pre-population from PMSS to GIS – executed on demand 1-2 times / year
2. Field Data Collection, Review and Approval – through Collector mobile app or Desktop web application
3. M20 Import from GIS to PMS – combination of GIS and PMS scripts and business processes executed on demand
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 11 of 18
3.3.2 M20 Data: Import from PMSS to GIS & Update (Tools A, B & C)
Import from PMSS to ArcGIS Enterprise Geodatabase: on-demand tool allows an administrator/user to import data
from PMSS to ArcGIS Enterprise Geodatabase. The tool will require the user to specify through a configuration file the
Dashboard Year, Schedule Start Year, and LRS data to use to generate the M20 data in GIS. The tool will then query from
two views in the PMSS database (a Contract-Level view like the existing vPerformance, and a Materials Layers-Level
view) using the specified Dashboard Year and Schedule Start Year.
The import process will be initiated by Central Office Pavement Management through submission of a Change
Management Request to VDOT Information Technology Division. This process will then be executed by ITD staff per
their routine procedures for such a request.
Update Tool: Scheduled process updates M20 Reference Lines to reflect changes to M20 Paving Start and M20 Paving End
points made by field users every one minute. Field users will not create reference lines (as creating M20 lines that match
the underlying route is tedious and/or impossible in the field). This process will also create M20 Reference Lines for
records that were created in the field. A secondary function of the update tool is to perform business validation of all
M20 business data. These checks fall into three (3) general categories:
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 12 of 18
1. Location Validation: application of user entered route information to select desired route from nearby routes,
flagging where there are no nearby routes matching the user provided route information
2. Contract Information Validation: verification that all required descriptive information is provided during field
update/entry of M20 records. Will flag incomplete or improperly entered data identified on a record
3. Material Information Validation: screening to ensure accuracy of Layer, Material, and Thickness information,
flagging records with improper Layer IDs, Thickness or lacking valid Material selection.
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 13 of 18
M20 GIS records are created when the following conditions are met in the PMSS database:
Field Description Use
Schedule_Start_Year Year when contract begins. User-selected parameter in Import Tool.
Only create M20s for contracts with the selected Schedule Start Year Value
Dashboard_Year Year mainline pavement will be paved at this contract location. User-selected parameter in Import Tool.
Only create M20s for contracts with the selected Dashboard Year
Include_Exclude
Include = location part of contract advertised Exclude = location not part of contract advertised Not a user-selected parameter.
Only create M20s for "Included" records
Route_in_RNS Indicates if Route exists in VDOT Location Referencing System. Not a user-selected parameter.
Only create M20s for routes that are in VDOT Location Referencing System
Validate_in_RNS
Indicates if Route (and MP) are validated against VDOT Location Referencing System. Not a user-selected parameter.
Only create M20s for routes that have validated locations
Route_Name
LRS validated route on which paving is occurring and along which the M20 line should be drawn. Not a user-selected parameter.
Only create M20s for routes that begin with “R-VA”
MP_From MP_To
Used, in combination with the selected LRS, to create M20 Reference Lines showing the location along the route that the M20 record pertains to. From the line, M20 Paving Start and M20 Paving End points are also created. Not a user-selected parameter.
Only create M20 for records with valid From and To information.
Project_Status_Cd Shows status of UPC (project) in the construction process. Not a user-selected parameter.
Only create M20s for contracts which are moving forward through advertisement/award/construction process. Do not create M20s for contracts with status “No Dates Yet Set”.
Contract Width Shows overall width of paving activities at the contract location
Only create M20s for contracts which involve a minimum 9’ width. Less than 9’ would be considered partial lane width paving/patching and will not be pre-populated
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 14 of 18
3.3.3 M20 Data: Import from ArcGIS Enterprise Geodatabase to PMS Staging Table (Tools D & E)
M20 Data: Import from GIS to PMS Staging Table tool runs on a scheduled basis to identify GIS data that is prepared and
ready for Migration from the GIS to a PMS staging table. This process is triggered by the GIS Administrator by updating
an administrative parameter which indicates the GIS to PMS import process will proceed. Once this parameter is updated,
the Python script updates all “Accepted” GIS data (records which have been approved by Pavement field staff, and pass
all business validation) to a “Ready to Migrate” status. Once data has been marked as “Ready to Migrate” by the tool, it
will be available to be imported to the PMS Staging tables by the PMS Administrator (note: “Ready to Migrate” will not be
editable in GIS).
The administrative parameter used to trigger the process must be updated by the GIS Administrator to flag the import
process as completed (or otherwise ended). This should only be updated after PMS Import scripts to import the GIS data
to staging have been executed. Until that point, the Python script will run in the background, comparing the “Ready to
Migrate” records to the PMS M20 table; when a match is found, the script will update the GIS data to “Migrated” status.
After the GIS Administrator has confirmed that PMS Import to Staging is complete, the GIS Administrator updates the
administrative parameter to indicate that the import process will no longer be running. At that time, the Python script
performance will revert any GIS records with “Ready to Migrate” status back to “Accepted” to allow continued editing.
Records with status “Migrated” will no longer be managed in the GIS, as PMS now will retain those records as the source
system of record.
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 15 of 18
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 16 of 18
3.3.4 M20 Data: Import from PMS Staging Table into PMS M20 Tables (Tool F)
M20 Data: Import from PMS Staging Tables into PMS M20 Tables is an import process that users interact with from the
PMS front end, like other existing import processes. This process pulls information from the PMS Staging Tables and into
the PMS M20 Tables. This utilizes existing PMS functionality. PMS Administrative staff may modify data in the Staging
tables as necessary to ensure proper import to the As-Built Paving Information tables.
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 17 of 18
Section 4 Physical View
4.1 Diagram and Description
INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI
PORTAL
October 30, 2017
Page 18 of 18
Section 5 Assumptions and Risks
Assumptions:
• VDOT Maintenance Division staff will be granted write access to M20 related GIS tables
• Timmons Group/Spy Pond Partners Team will be granted access to necessary data
• Timmons Group/Spy Pond Partners Team will be granted access to necessary systems
• VDOT ITD will support in an advisory and consulting role
Risks:
• Portal deployment will be complete at VDOT before required end date of contract
• ArcGIS Portal will be functional and available in VDOT Production environment