The ione infrastructure metric mapping system (ims) armando guevara

© The iOne IMS Visual Intelligence -- LiDAR News 1

The iOneTM

Infrastructure Metric-Mapping System A Paradigm Shift on Co-Mounting and Co-Registering Geoimaging Sensors with LiDAR

Dr. J. Armando Guevara

Visual Intelligence LP - President and CEO

[email protected]

ABSTRACT

Integration efforts of geoimaging sensors with LiDAR have demonstrated the need to

systematically improve the co registration of the imagery with the LiDAR data such that errors in the

imagery collected are greatly reduced by the sensors being rigidly mounted, share the geopositional

metadata and are registered to each other in a rigorously calibrated metric configuration.

At Visual Intelligence (“VI”) a committed pursuit with our customers is to increasingly enable

them with our ubiquitous metric geoimaging sensor technology to collect more, do more, for less; this

whilst abating the increasingly speed at which digital devices become obsolete. In this pursuit VI has

developed the iOneTM

Sensor Tool Kit Architecture (or iOne STKATM

) from which the iOne Infrastructure

Metric-Mapping System has been developed. The iOne IMS is a modular and scalable co mounted and co

registered (“CoCoTM

”) geoimaging sensor with LiDAR that can readily, efficiently and economically be

configured to fit a variety of infrastructure surveying applications. This paper describes first the iOne

STKA and the iOne IMS, its core design and the operational efficiencies it provides.

KEYWORDS: iOne, STKA, Iris One, Digital camera, Camera calibration, Co mounting and Co

registration of Sensors, infrastructure mapping and surveying

1. INTRODUCTION

Founded in 1997, Visual Intelligence (VI) has focused on research and development (R&D) to provide

a multipurpose metric digital geoimaging sensor technology with scalable sensor imaging arrays for

automated high-accuracy metric geoimaging for mapping, surveillance, ground and mobile applications.

The sensor architecture is designed to be economical (lowest cost of ownership), light, small, high

collection, high resolution, and fast in deployment. The multi-year R&D has resulted with various granted

patents that have provided the foundation to generate a Virtual Frame (VF) camera systems comprised of

multiple COTS camera modules arranged at certain angles to achieve flexible and rapid configurations as

different and distinct (sometimes conflicting) mission requirements may mandate.

The patents awarded along with the USGS Aerial Digital Sensor Type Certification received in 2009

for the Iris One 50 (now called the Iris One Ortho 19 kps); validate the uniqueness of VI’s intellectual

property, technological foundation, and its forthcoming potential transforming role in the digital

geoimaging industry. The current and evolving portfolio of VI IP has been casted into a sensor tool kit

architecture called the iOne Sensor Tool Kit Architecture or iOne STKA. The scalability of the sensors

built from the iOne STKA are based on the Angular Retinal Camera Array (ARCA); this scalability

property allows for both functional and collection scalability. Functionally the sensors can be configured to

have only or many features such as ortho, stereo, oblique, full 3D as well as CoCo -(the co mounting and co

registering of sensors; e.g. imagery fusion- such as LiDAR, thermal, SWIR, FLIR, multispectral and

hyperspectral among other types of passive (e.g. electro-optical) and active (e.g. LiDAR, radar) sensors.

The modularity of the iOne STKA allows flexibility and scalability to meet various customer needs and

applications within one single base sensor system (hence “iOne”).

Visual Intelligence among its sensor family built from the iOne STKA has brought to market the Iris

One Ortho 19 kps; the Iris One MS; the Iris One Stereo and recently the iOne Infrastructure Mapping

System or iOne IMS –the subject matter of this paper. The iOne family of digital geoimaging sensor

systems in general, and in particular the iOne Stereo, is designed to achieve and exceed the performance of

the film aerial cameras, in collection capacity and metric accuracy. For a detailed description of the Iris

One family of sensors created to-date from the iOne STKA please see Petrie, Gordon 2012.


2. iOne STKA ARCA BASED DESIGNS

The iOne STKA is an optimal set of software, hardware, methods and procedures geoimaging

components (”Lego®-like) that include advanced imaging processing algorithms for radiometric and

geometric accuracy, pixelgrammetry processing and image analysis (feature extraction, point cloud

generation); it is a flexible and modular set of components, all solid state that are all integrated by software.

The iOne STKA is based on the ARCA (Guevara, 2009), a patented angular cross eyed imaging array

that allows the system to be small, light and scalable in collection capacity, resolution, and functionally

using the same base architecture –i.e. One system for all applications. An advantage of the ARCA is that

camera modules are configured in a linear arrangement with an “hour glass imaging effect”, giving it the

advantage of imaging a larger swath while looking through a small aperture; the optical axis of each

individual CM in the array to intersect, passing through a single perspective center. The patented ARCA

design uses synchronously operating camera module heads to form a single virtual central-perspective

image.

With multiple ARCAs, Iris One system can be easily configured as multi-spectral sensor (double

ARCAs) and stereo system (triple ARCAs). The multi-spectral version of Iris One system allows the color

RGB images recorded by camera modules mounted on one ARCA to be co-registered with and

superimposed on the corresponding near infra-red (NIR) images collected by the cameras mounted on the

second ARCA. The Iris One stereo system, with three ARCAs, can be oriented either in the cross-track or

the along-track direction. Vary-format camera modules and different types of lens produce various ground

coverage. There are two typical settings for the Iris One stereo system. The 60 % longitudinal overlap along

the flight line that is produced when the cameras are programmed to expose overlapping sets of images in a

stereo convergent configuration gives a base: height ratio of 0.6 –unique in the industry. When the system

is equipped with 9x29 MPIx camera modules and 135mm lenses and rotated by 90 degrees into the cross-

track position, the system yields a 0.34 base: height ratio, similar to that achieved by the overlapping stereo

images that are produced by conventional large-format digital mapping cameras.

(a) (b)

Figure 1- (a) This figure shows the ARCA(s) into which varying-format camera modules can be inserted.

(b) The geometric arrangement of an ARCA configuration; each ARCA or set of ARCAs generates a single

metric frame ingestable by any 3rd party photogrammetric workflow.

As such the iOne STKA allows to configure combinations of camera modules (CMs) that yield cross

track collection efficiencies from 20,000 pixels upward of 60,000 pixels cross track, as well as frame sizes

along track in excess of 19,000 pixels (depending on how the arrays are aligned). Table 1 depicts part of the

developed Iris One systems family. For a detailed technical description of the ARCA and its functional and

collection efficiencies please see Petrie, Gordon 2012.


Iris One Ortho/MS 19 kps A highly agile and robust system for ortho wide area collection

Iris One Stereo with

configurable B/H .6 or .3

depending on ARCA cross

track or along track

orientation

Based on B/H .6, only sensor in the industry with engineering quality

imagery equivalent or superior to film. The same system can be “rotated”

to achieve higher collection efficiencies whilst achieving .3 B/H.

Iris One Infrastructure

Mapping System (IMS)

A powerful, very light and compact sensor that provides ortho, multi-

spectral, backward and forward oblique, all in one pass

Isis Earth™ Software Post processing software that is integrated with Iris One sensors used to

generate accurate ortho images.

Isis Sky™ Software Near real-time onboard (in-flight) ortho processing software that is

integrated with Iris One sensors.

Table 1- The Iris One family of sensors and software based on the iOne STKA is capable of handling

numerous collection scenarios.

For engineering-quality metric application, high accuracy and resolution requirement in both geometric

and radiometric aspects must be met (Cramer, 2006). The geometric accuracy of the Iris One digital

imaging sensor systems is achieved from laboratory calibration of each camera module, the arrays set, as

well as calibration flight using BINGO (Kruck, 2010; Hwangbo, 2012). With highly accurately determined

geometric properties for the complete camera module set in the ARCA array(s), the Iris One system is able

to produce virtual frames from each ARCA CM set which is defined as single central perspective,

distortion-free image. The simple geometry of the ARCA Virtual Frame image makes it compatible with

the traditional workflow of any photogrammetric software. Moreover, radiometric calibration explores

camera’s radiometric properties to naturally link image data with actual scene for high-quality imagery

production. For further in depth technical details on both geometric and radiometric calibration procedures,

please refer to Hwangbo 2012 and Guevara-Wang 2012).

Figure 2- VI’s geometric calibration facilities: a). Distribution of control points used for laboratory

calibration (red dots are located on a 2D calibration wall and blue stars are located on a 3D calibration

frame); b). Calibration field with distribution of ground control points.

3. INTRODUCING THE CONCEPT OF KPS

A kps is the number of pixels across track covered by a sensor on the ground, in other words, the

number of pixels in the swath. 1 kps = one thousand pixel wide swath. Therefore the number of pixels

allowed by an iOne sensor to collect is defined as kps, or kilo pixel swath. So depending of the application

the sensor can be as small as 7 kps to as large as it is required by the mission of the sensor, all using the

same base architecture –feature that yields very fast sensor deployment time.

(a) (b)


Why kps? It is often difficult to differentiate the competing claims of different digital aerial

camera manufacturers when it comes to efficiency. There are many aspects that contribute to the

efficiency, but one simple measure is for the same aircraft speed, how much area is collected per hour of

flying?

There are many factors that go into the design of a successful aerial metric geoimaging project,

including but not limited to camera focal length, CCD size and flying height. For digital cameras, the

project is designed for a specific nominal ground sample distance or GSD. No matter what flying height,

focal length, or CCD size, the amount of area covered is the nominal GSD times the number of pixels in the

selected width of the CCD array (x or y orientation – typically the largest number of pixels on the CCD is

in x so if more depth is desired the CCD can be rotated with x becoming y).

Normal block collection of a project will factor in a 30% side lap between flight lines. Examples of pixels

swath with “CCD configurations” of:

A. 7 kps will have an effective swath 5051' wide

B. 11 kps will collect a swath 8,043' wide

C. 19 kps will collect a swath 13,280' wide

D. …and so on.

kps Width

30% side

lap

7 5,051 2,165

11 8,043 3,447

19 13,280 5,691

In a project example with a flying speed of 150 miles an hour times 5,280 feet in a mile, the aircraft travels

approximately 792,000 feet an hour. Efficiency results for the above kps example are shown on the

following chart:

Traveled

Distance/hour Width sq feet sq miles Efficiency

A 792,000 5,051 4,000,550,400 144 1.00

B 792,000 8,043 6,370,056,000 228 1.59

C 792,000 13,280 10,517,522,400 377 2.63

Figure 3- Impact of kps in collecting at 1 foot pixels (30 cm)


4. INTRODUCING CoCo

In 1998 VI designed, built and operated its first generation of digital aerial imaging and mapping

sensor. In 2001 VI acquired its first LiDAR. Since then, VI has been at the leading edge of innovation by

improving the operational use of LiDAR technology tightly coupled (co mounted and co registered) with

aerial digital cameras with different kps and FOV according to mission. Recently VI integrated LiDAR

technology with its 3rd generation imaging sensor technology, the iOne IMS (infrastructure metric-

mapping system).

CoCo is a patented vehicle based data collection and processing system and imaging sensor

system and methods thereof. The apparatus’ and methods optimize the co-mounting and co-registering of

two or more sensors, for example, an EO camera system with a LiDAR rigidly mounted on a single plate

with IMU. The claims were directed towards the incorporation of the co-mounted and co-registered nature

into VI’s system for terrain mapping which obtained unprecedented sensor integration performance for

large scale and large geographic area mapping.

The integration of CoCo begins with mounting the two sensors together. The sensors must be

rigidly mounted on the same base plate so that aircraft flex is minimized. This flex needs to be less than

100th. The sensors must be mounted as close as possible and best practices have the two sensors mounted

over the same aperture. This is possible because the Iris One’s ARCA can operates without the need for a

gyro-stabilized mount. The nearness of the sensors is needed so that they can share the same ABGPS/IMU.

This is only possible also because the Iris One family of sensors have a light, compact and rigid design that

can be Co-mounted with out the need for an additional standard camera hole.

Figure 4- CoCo System Diagram

The lever arm measurements must be taken for each sensor so that when processed each sensor

has its own offsets. By sharing the IMU and GPS, all sensors are calibrated substantially the same epoch,

using the same GPS signal, ground targets and under the same atmospheric conditions. Basically this

means, each image pixel and each LiDAR pixel are referenced from the same location. This greatly reduces

compounded errors realized when calibrating each separately, using different GPS signals or under

atmospheric conditions. Separate sensor holes (with no co-mounting) with a shared IMU is not the same, as

the aircraft has its own flex that can change or move independent to each other which causes positional

errors which may work depending on accuracies needed. Separate IMUs is another approach but each IMU

will have its own biases and cause differences in yaw, pitch, and roll data.


4. INTRODUCING iOne IMSTM

The Iris One Infrastructure Metric-Mapping System (iOne IMS) camera system based on the iOne

STKA, integrated with a LiDAR system in CoCo mode, is an efficient, economical, one-pass, all feature

digital infrastructure capture system that can support numerous image data requirements using a helicopter

or fixed wing aircraft. From a single control interface, operators can capture and monitor imagery from

high-resolution oblique, wide swath multi-spectral, and optional thermal and video cameras referenced to a

single GPS/IMU reference system for common picture and overlapping display.

With its single pass, full capture capability, the system produces accurate, high-quality imagery

saving 50%-75% over less efficient aerial collection cost. With its multi-sensor ARCA based platform

architecture, the system can grow to accommodate additional sensors thereby further increasing

information value with little added data collection cost.

The iOne IMS configuration includes the following benefits:

• Two camera oblique images (fore/aft) assures full coverage

• Single wide-field NADIR cameras for full swath 4-Band coverage with high positional accuracy

• Single operator control with on-board Quick-Look quality review

• B/H = 32% at 700 ft AGL to support future stereo/corridor-wide DTM

Figure 5- (a) the iOne IMS dimensions –a small, light compact system that can be flown on rotary or fixed

wing aircrafts (b) iOne IMS capability to collect ortho, near infrared, backward and forward oblique –all in

one pass.

The iOne IMS sensor is designed to collect orthophotos (RGB and Near Infrared) and oblique

imagery (forward and aft) simultaneously with a LiDAR sensor. The collection of laser and imagery data is

being conducted to be able to support, for example, the generation of surveys of power line transmission

corridors.

The iOne IMS generated surveys support the following categories of analysis:

• Inspection of transmission hardware mounted on towers

• Right -of -Way Analysis

• Inspection of power and relay substations

• Power line sag and tension analysis

• Encroachment of man-made and natural vegetation (e.g. trees)


This system will support the same analysis functions in other transmission corridors such as railways,

pipelines and others as required. The nominal parameters for the oblique sensor system are defined below:

Figure 6- (a) Minimum Oblique Collection Geometry

(Nominal altitude = 700 ft, Range, 850 ft, GSD approx. 0.5 inches) (b) Nadir Ortho Collection Geometry

Features Analysis Factors

Insulators/Conductors Size, Texture, Condition, Nominal (0.44 inch) Max

GSD (1 inch)

Transformers Size, Condition, Nominal (0.44 inch) Max GSD (1

inch)

Transmission Lines Condition, Sag, Sway Distance, Nominal (0.44

inch) Max GSD (1 inch)

Transmission Towers Condition, Size Envelope, Max GSD

Ground Vegetation Location, Distance to Towers, Max GSD, Bands

Trees/Foliage Location, Distance to Sway Line Limit, Bands

Fences Location

Accuracy standards if processed with solid ground control

of Manmade Structures within Easement

Location

Roads/Access to Easement Location

Examples of Electrical Infrastructure Feature Types that can be Collected- Sizes and Characteristics

Nominal Distance between camera and target ft 850 ft

Estimated tower height (minimum) 60 ft

Estimated tower height, (maximum) 200 ft

Estimated tower height, (average) 120 ft

Tower covers % of image height (minimum) 75

Desired resolution of tower, inches/pixel 0.44


Fig. 7- (a) iOne IMS Oblique sample image (b) Oblique image and automated extraction of

features of interest for further detailed analysis.

The operational envelope of the iOne IMS is as follows:

Corridor (or Coverage) Min/Max for Nadir Ortho Camera

• Minimum Swath: 600 feet

• Nominal Swath: 750 feet

• Desired Swath: 900 feet

Corridor (or Coverage) Min/Max for Oblique Camera

• Minimum Swath: 100% of infrastructure feature surveyed Width & Height

• Nominal Swath: Minimum Swath to Support feature collected


GSD Min/Max

• Oblique Camera Minimum GSD: 0.44”

• Oblique Camera Maximum GSD: 1.0”

• Nadir Cameras Minimum GSD: 6”

• Nadir Cameras Maximum GSD: +- 10%

iOne IMS sensor operation

• Single Operator

• Minimal Operations Workload; Flight technician-level skills

Altitudes (MSL) and (AGL) min/max

• Minimum Altitude (AGL): 600 ft

• Nominal Operations Altitude (AGL): 700 ft

• Maximum Altitude (AGL): 3,000 ft

Environment/Sun Angle/Time of Day

• Operations within +/- 20 deg sun angle range such that features are interpreted (detected)

in shadows.

• Platform Angle Range

• +/- Roll : system controlled

• +/- Pitch: system controlled

• +/- Yaw: system controlled

• Ground Speed Velocity and Tolerance: system controlled

• Features are interpretable in shadows.

Typical Mission Day

• Pre-mission calibration (boresight)

• 4-6 hours of collection

• Camera system operation is automated. No more than 5 minutes/hour operations

required for system operations monitoring

• 2TB on-Board SSD Data Storage (1-1.5 TB typical)

• 1 Hour Post Mission Data Quality Analysis

• On-site Imagery Review with Post-flight previewer software

The Iris One IMS will produce multiple image products for use in utility corridor status analysis and

asset assessment. These products are summarized as:

� Image RGB Oblique of full features (e.g. Towers) -100% coverage- front/back (image pair)

o GeoTiff with lat/long location center, image scale

o One image per feature structure is generated – iOne IMS creates an oblique virtual frame

if the feature appears in two or more images. Feature virtual frames are minimized.

o Provides KML with camera orientation during exposure (meta data option)

o Google Earth KMZ file with image and camera locations (meta data option)

� Provides Multispectral Image -Four-Band Orthos- (metrically co registered at 1:1 resolution

RGB+NIR)

� The images are color balanced across for the mosaicing workflow.

o Oblique Images maintain color saturation, intensity, and hue across the mission

o Ortho images support color balancing and mosaicing methods


The iOne IMS was designed for ease of use for the operator and minimal training requirements to

be proficient at operating the system. VI includes a flight planning software tool called TopoFlight

Navigator that is bundled with the iOne Isis Earth orthophoto system.

TopoFlight Navigator is used to navigate the aircraft for image acquisition flights. A predefined

flight plan (e.g. provided using TopoFlight) is used as base data. The camera is triggered at the pre-defined

positions. The interface for every camera can be delivered or can be implemented by the operator. The

system consists of different modules to provide the capability to combine the actual TopoFlight Navigator

with any available GPS, IMU and camera system.

(a) (b)

Fig. 8 (a) The iOne IMS and RIEGL VQ-480 combination mounted side-by-side on a common base plate

which is placed on a set of anti-vibration dampers – as viewed from the side at left and from above at right.

(b) The iOne IMS system mounted in an Aerocommander aircraft.

5. OPERATIONAL EFFICIENCIES

For many projects collecting LiDAR and Imagery together can lead to less flight time or eliminate

the need for an additional aircraft with separate sensors. Over the years since VI established the CoCo

approach, we have created for our users innovative ways to collect the data simultaneously. Some examples

follow.

5.1 Forestry

CoCo collection was used in several forestry projects where the Imagery FOV was 70° and the

LiDAR FOV of 45° (due to point density needed). Since the LiDAR was the limiting factor, VI devised for

its customer the flight plans to maximize each sensor FOV. To solve for this the imagery and LiDAR were

collected on even numbered flight lines during prime sun angle and LiDAR only was collected on odd

numbered lines during times of less than optimal sun angle. This improved overall collection time by using

the least number of lines and only one aircraft with one flight and ground crew. This approach allowed for

the imagery to be flown with the minimal amount of flight lines, which relates to less data to process, faster

deliverables and overall operational savings.

5.2 Infrastructure Corridor Mapping

CoCo was used in a corridor mapping project where the LiDAR was flown to create more accurate

DEMs for the ortho imagery. Sample case project collected 500 miles of pipeline. The customer wanted a 1

mile swath of imagery and 3500ft swath of LiDAR. Instead of using two aircrafts, one with a LiDAR and

the other with a digital camera, the projects was flown with both simultaneously, and by using an Iris One

19 kps the project was flown more efficiently- having both sensors collecting concurrently reduced costs

by 50%.


6. CONCLUSIONS

This paper has described the iOne STKA, the CoCo technology and its new embodiment the Iris

One Infrastructure Metric-Mapping System or iOne IMS, an efficient, economical, one-pass, all feature

digital infrastructure capture system that can support numerous image data requirements using a helicopter

or fixed wing aircraft. The operational improvements (data collection, time, cost) obtained by flying in

tandem the iOne IMS in CoCo mode can lead to great operational efficiencies and cost savings such as less

flight time and/or eliminate the need for an additional aircraft with separate sensors.

With the iOne STKA VI has created a robust and solid software and hardware Lego®-like

foundation to design and deploy any type of EO sensor, and if required, fused (“CoCo” - co mounted and

co registered) with any other passive or active sensor type in the most effective and efficient manner, e.g.

LiDAR, thermal, video, UV. The iOne STKA is backed by numerous patents and IP (methods, procedures

and software) that yield a very powerful plug-and-play sensor foundation. Methods and procedures include

but are not limited to robust geometric and radiometric calibration; very large virtual frame generation that

is ingestible by any traditional photogrammetric workflow (the ARCA array set behaves like one single

camera); ortho direct positioning onboard processing software that is the platform for event driven report

generation and more.

REFERENCES

Cramer, M., 2006. Calibration and validation of airborne cameras. Proceedings ISPRS Commission I

Symposium “From Sensor to Imagery”, Paris – Marne Le Valle, July 4-6, 2006.

Guevara, A., 2009. The ARCA of Iris: a new modular & scalable digital aerial imaging sensor architecture.

ASPRS 2009 Annual Conference, Baltimore, March 9-13, 2012.

Guevara, A.; Wang, W 2013. The iOne STKA Foundation for the Iris One Sensor Family. ASPRS 2013

Annual Conference.

Hwangbo, J, 2012. Iris One Stereo System, ASPRS 2012 Annual Conference, Sacramento, March 19-23,

2012.

Kruck, E., 2010. Developments and challenges in bundle triangulation, ASPRS 2010 Annual Conference,

San Diego, April 26-30, 2010.

Petrie, Gordon 2012. Visual Intelligence’s Iris One Airborne Camera Systems - Based on its iOne Sensor

Tool Kit Architecture. Emeritus Professor of Topographic Science in the School of Geographical & Earth

Sciences of the University of Glasgow, Scotland, U.K. E-mail – [email protected] ; Web Site –

http://web2.ges.gla.ac.uk/~gpetrie/ - Geoinformatics Magazine (September 2012 issue no. 6/2012

http://www.geoinformatics.com).

The ione infrastructure metric mapping system (ims) armando guevara

Technology

Transcript of The ione infrastructure metric mapping system (ims) armando guevara