Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient...

13
Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway EIVA A/S Page 1 of 13 S-CAN in NaviModel3 Efficient Sonar Data Cleaning Implementation of the S-CAN Automatic Cleaning algorithm in EIVAs NaviModel3 By Lars Dall, Survey Manager, EIVA A/S Abstract The optimization of the post-processing environment within EIVAs 3D modelling tool, NaviModel3 has focused on two aspects primarily: Optimization of the visual environment in order to supply the operator with enhanced and improved background information for his decision making Speed-optimization and automation of the entire post-processing task One of the major components in the speed-optimization and automation has been the implementation of the S-CAN automatic data cleaning algorithm. Key words: Data modelling, optimization and practical efficiency, manual and automatic cleaning functionalities, performance 1. Introduction The objective of the optimization of the post-processing environment within NaviModel3 has been to supply the users with tools that facilitate the production of better and more unambiguous bathymetric data, faster and with reduced user intervention. For the optimization of the visual environment, a series of new features have been implemented. In the development phase of NaviModel3, it was regarded an important enhancement to the visual environment to be capable of displaying all data of relevance in an integrated 3D based DTM- window with video functionalities being integrated into the visualization of the data. Of specific interest in connection with pipeline inspection tasks is furthermore elements that will improve the visualization of the pipe object. Also the speed-optimisation and automation has had a series of focus areas. The fact that NaviModel3 has an unlimited model-size is expected to be a significant factor. Furthermore, with seabed mapping and investigations, such as pipe-line inspection, to a large extent depending on reliable and accurate seabed information, some kind of fast, consistent, yet user-friendly, automatic cleaning of the seabed scanning data has been regarded a necessity. The implementation of the S- CAN automatic cleaning algorithm is the response to this. In this context, it is worth noting, that cleaning can only take place within the framework of NaviModel3, i.e. in the modelling phase rather than in the editing phase of the post-processing, due to the fact that the raw observations are inherent in the terrain model. This introduces a new approach to the cleaning of multi-beam based bathymetric data that gives the user the possibility to observe the consequences of the cleaning, be it automatic or manual, instantly. The objective of this paper is thus to describe the environment in which the S-CAN cleaning algorithm has been implemented, to describe the actual implementation, and to give some estimates on the efficiency of the cleaning, mainly in order to substantiate that the objectives of the optimization have been met. The paper will furthermore consider potential future developments and enhancements to the automatic cleaning functionalities.

Transcript of Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient...

Page 1: Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient Sonar... · Efficient Sonar Data Cleaning ... 1. Introduction The objective of the

Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway

EIVA A/S Page 1 of 13 S-CAN in NaviModel3

Efficient Sonar Data Cleaning Implementation of the S-CAN Automatic Cleaning algorithm in EIVAs NaviModel3

By Lars Dall, Survey Manager, EIVA A/S

Abstract

The optimization of the post-processing environment within EIVAs 3D modelling tool,

NaviModel3 has focused on two aspects primarily:

Optimization of the visual environment in order to supply the operator with enhanced and

improved background information for his decision making

Speed-optimization and automation of the entire post-processing task

One of the major components in the speed-optimization and automation has been the

implementation of the S-CAN automatic data cleaning algorithm.

Key words: Data modelling, optimization and practical efficiency, manual and automatic

cleaning functionalities, performance

1. Introduction

The objective of the optimization of the post-processing environment within NaviModel3 has been

to supply the users with tools that facilitate the production of better and more unambiguous

bathymetric data, faster and with reduced user intervention.

For the optimization of the visual environment, a series of new features have been implemented. In

the development phase of NaviModel3, it was regarded an important enhancement to the visual

environment to be capable of displaying all data of relevance in an integrated 3D based DTM-

window with video functionalities being integrated into the visualization of the data. Of specific

interest in connection with pipeline inspection tasks is furthermore elements that will improve the

visualization of the pipe object.

Also the speed-optimisation and automation has had a series of focus areas. The fact that

NaviModel3 has an unlimited model-size is expected to be a significant factor. Furthermore, with

seabed mapping and investigations, such as pipe-line inspection, to a large extent depending on

reliable and accurate seabed information, some kind of fast, consistent, yet user-friendly, automatic

cleaning of the seabed scanning data has been regarded a necessity. The implementation of the S-

CAN automatic cleaning algorithm is the response to this.

In this context, it is worth noting, that cleaning can only take place within the framework of

NaviModel3, i.e. in the modelling phase rather than in the editing phase of the post-processing, due

to the fact that the raw observations are inherent in the terrain model. This introduces a new

approach to the cleaning of multi-beam based bathymetric data that gives the user the possibility to

observe the consequences of the cleaning, be it automatic or manual, instantly.

The objective of this paper is thus to describe the environment in which the S-CAN cleaning

algorithm has been implemented, to describe the actual implementation, and to give some estimates

on the efficiency of the cleaning, mainly in order to substantiate that the objectives of the

optimization have been met. The paper will furthermore consider potential future developments and

enhancements to the automatic cleaning functionalities.

Page 2: Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient Sonar... · Efficient Sonar Data Cleaning ... 1. Introduction The objective of the

Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway

EIVA A/S Page 2 of 13 S-CAN in NaviModel3

2. NaviModel3

The NaviModel3 DTM modelling software is a tool for the generation of and manipulation with

Digital Terrain Models (DTMs) on the basis of either single- or multi-beam based bathymetric data.

The modelling is founded on either Triangular Regular Network (TRN) or on Triangular Irregular

Network (TIN) algorithms. The TRN geometry type models consist of equally spaced triangular

cells, whereas for the TIN geometry type models, triangles are based on the raw data.

A series of dedicated add-on modules that are designed for specific tasks, have been developed for

inclusion in the modelling tool. These comprise:

Online 3D module. This module facilitates visualization in an online environment in which

various objects can be shown in real time and superimposed onto a DTM

Catenary module that facilitates a variety of catenary based tools, as well as calculations and

visualizations associated with pipe- and cable-laying jobs

Pipeline inspection module

Figure 1 The NaviModel3 Post-processing environment

Figure 1 shows the flow for a typical NaviModel3-based post-processing task. Input data is

originating from the on-line and editing phases. Furthermore video- and event-information,

originating from external sources, can be imported. NaviModel3 hosts different tools, for cleaning,

visualization, determination of the pipe etc., that are used in combination with external utilities for

offline eventing and automatic cleaning. Ultimately, in order to make the bathymetric data available

for further processing and documentation, a series of generic and predefined exporting

functionalities have been implemented.

3. The Quad Tree Principle

Within NaviModel3, data is, in general terms, organised in a so-called Quad Tree structure. A Quad

Tree is a tree-based data structure in which each internal node has up to four children. Quad Trees

are commonly used to partition a two dimensional space by recursively subdividing each level into

four quadrants or regions as visualised in the figure below. In a Quad Tree, records are stored in

locations called leaves. The name originates from the fact that records always exist at end points;

there is nothing beyond them. The 1st level is also sometimes termed the root. Branch points, on the

Page 3: Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient Sonar... · Efficient Sonar Data Cleaning ... 1. Introduction The objective of the

Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway

EIVA A/S Page 3 of 13 S-CAN in NaviModel3

other hand, are called nodes. The order of a tree is the number of branches per node. In a Quad

Tree, there are always four children per node, so the order is 4. The number of leaves in a Quad

Tree is consequently always a power of 4. The number of access operations required to reach the

desired record is called the depth of the tree. Figure 2 visualizes a Quad Tree of depth 4.

In a practical Quad Tree, there can be billions of records. As can be seen in the figure, not all leaves

necessarily contain a record and the same is actually the case for nodes. In the latter case, the node

does not have to be subdivided. When a leaf does not contain a record it is called a null record. In

the example shown here, seven out of 64 leaves are nulls, indicated by open circles.

Figure 2 Quad Tree with Leaf Level

Within NaviModel3, the higher orders of the Quad Tree are primarily used for visualization

purposes: the higher the scale the lower the requirements to the resolution and thereby the higher in

the Quad Tree the data can be assembled. When exporting from a DTM, the leaf level data are

normally used to define the various attribute values for the export. The cell size for the exporting is

however used to optimize the exporting: if for instance the cell size for the export is 2 times the cell

size of the model, then the exporting function will actually collect attribute values from the level

above the leaf level. This will potentially speed up the export by a factor 4.

In NaviModel3 a database that encloses the hierarchical organisation of the Quad Tree structure has

been entrenched in a single file solution. With this type of organisation of data, a high degree of I/O

efficiency is required from the system, with I/O efficiency defined as the speed by which data is

moved between internal memory and storage media, primarily when the user pans or zooms in the

model or when a new model is loaded. Whereas the loading of a model can be accomplished in less

than 1-2 seconds, irrespective of size, moving and panning can be conducted seamlessly, without

the user noting that the system accesses the disc in order to update the view in accordance with

changes in scale and position.

4. Geometry Types and Interpolation Methods

A Digital Terrain/Elevation Model (DTM/DEM) can be regarded a generalization of the observed

bathymetric data, with generalization being defined as the process of reducing the amount of detail

in a map in a meaningful way, with respect to scale.

At the same time, since total model coverage of the area of interest is often a requirement, the

model must often be extended beyond the observations.

Whereas different geometry types are used to generalize on the basis of the bathymetric

observations, interpolation methods are utilized to generate qualified information, based on the

observations available, in areas where no observations have taken place.

NaviModel3 supports two different geometry types:

The Triangulated Regular Network (TRN) geometry type

The Triangulated Irregular Network (TIN) geometry type

Page 4: Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient Sonar... · Efficient Sonar Data Cleaning ... 1. Introduction The objective of the

Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway

EIVA A/S Page 4 of 13 S-CAN in NaviModel3

Only the TRN geometry type is of relevance in the present context.

TRN Modelling

The TRN geometry type is based on square cells with a given cell size. The cell attribute value (z-

value or depth value) can either be arrived at by averaging all the observations within a single cell

or by taking the minimum or maximum depth value respectively. The TRN geometry method

produces all three model types by default. So when generating a DTM within the NaviModel3

framework, the first step is to generate these three model types on the basis of the input data and on

the specified cell size.

The TRN cell array looks as visualised in Figure 3, left. The centre of each cell is visualised with a

green dot. Each cell has its own attribute value not associated with the attribute of the neighbouring

cell. The model therefore will appear to have steps and will certainly not seem smooth. With

squares representing the attributes it is in other words not possible to make the seamless transition

between the cells that is often desired.

Figure 3 TRN Squared Array with primary cells as squares (left). Triangulated TRN on the basis of the squared cell model (right)

Each cell is therefore sub-divided into four triangles as it appears in Figure 3, right. The corners of

each cell (visualised with a yellow dot) is given an attribute value that is representing the attribute

values of the four neighbouring cells. For an average model, the point will be given the average

value of the neighbouring cells, whereas it will be given the maximum and the minimum of the four

neighbouring cells for maximum and minimum models respectively. By using triangles it is ensured

that each piece of the mosaic surface will fit with its neighbouring pieces since the surface of each

triangle is defined by the elevations of the three corner points.

With all three corners of each triangle having been assigned an attribute that is linked to the

neighbouring points, it is possible to create the desired continuous seamless transition between

cells, represented by the triangles, within the model.

The TRN-based model types can be used to generate and export gridded values even if the desired

cell-size and orientation is not identical to that of the model. The present TRN model type selected

can then be regarded a look-up table and the export routine will thus, by sending the XY-values,

make NaviModel3 return the associated attribute value from the present TRN model type.

The generation of contours is also based on the TRN model. For each depth value related to the

contour interval and the depth range of the model, the contour routine will search the model and

find places with attribute values equal to the desired contour values. For each contour the points

returned will be connected taking the basic requirements to contouring into consideration, such as:

Contour lines cannot cross each other

Contour lines cannot stop in the middle of the model (unless there are no data available)

Page 5: Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient Sonar... · Efficient Sonar Data Cleaning ... 1. Introduction The objective of the

Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway

EIVA A/S Page 5 of 13 S-CAN in NaviModel3

Contour curves cannot split in two

Contour curves cannot follow the top of a ridge or the bottom of a depression

Interpolation Methods

Interpolation is used to predict the values of attributes in areas with no observations available, but

within the area covered by observations. Predicting values outside this area is termed extrapolation.

When data is abundant, like in connection with areas observed with multi-beam techniques, most

interpolation techniques will yield (close to) similar results. When data are sparse, however, like in

connection with single-beam surveys, the method for interpolation can be critical.

Two different methods for interpolation/extrapolation are supported:

Interpolation, based on the average model or on the minimum model

TIN Modelling (which is then both a model type and an interpolation method)

Interpolated Models

NaviModel3 facilitates the method of performing ‘Interpolated’ models on the basis of a TRN

model. The phrase ‘Interpolated’ covers the fact that an interpolation is taking place internally in the

surveyed areas, whereas extrapolation is taking place beyond these areas.

NaviModel3 will, on the basis of primary cells (≈ cells containing observations), interpolate to the

neighbouring cells by utilizing a user-defined search radius. Where primary cells are completely

surrounding the secondary point in question, taking the search-circle radius into consideration, the

method will result in an interpolated result. Otherwise an extrapolated result will be the outcome.

Figure 4 Principle of Interpolation: Extrapolation (left) and Interpolation (right)

This is visualised in Figure 4 above. The primary cells are indicated with a green dot, whereas the

secondary cell in question is represented by a red dot. Around the secondary cell a circle is drawn

with the predefined search radius. This is called the search circle. The routine will search for

primary cells in all directions inside the search circle. If more than one primary cell is found in a

given direction, only the closest will be used to determine the attribute of the secondary cell. Once

all directions have been investigated and at least one primary cell has been found inside the circle,

the value for the secondary cell in question is found as a weighed average of the attributes of the

primary cells found. This weighing is performed as the inverse to the square of the distance between

each of the primary cells and the secondary cell in question. The attribute value of the secondary

cell is in other words calculated as:

Page 6: Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient Sonar... · Efficient Sonar Data Cleaning ... 1. Introduction The objective of the

Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway

EIVA A/S Page 6 of 13 S-CAN in NaviModel3

i

n

i i

n

i i

Apd

d

As *1

*

)1

1

12

12

, with

As = attribute value of secondary cell

Api = attribute value of primary cell i

n = number of primary cells found

di = distance from secondary cell to primary cell i

The first part of the equation is used to normalise the result for the attribute of the secondary cell,

whereas the second part constitutes the weighing of the observations relative to the inverse of the

square of the distances.

The method can be considered a dedicated case of the Inverse Distance Weighing (IDW) method

that by some is considered the workhorse of spatial interpolation. IDW achieves the desired

objective of creating a smooth surface whose value at any point is more like the values at nearby

points than the values at distant points. The method is thereby taking Tobler’s first law of

geography into consideration ‘Everything is related to everything else, but near things are more

related than those far apart’.

IDW is often described as an exact method of interpolation, since the results are true to the input as

opposed to an approximate method that allows the result to deviate from the input in the interest of

perhaps a higher degree of smoothness of the model. Consider however, if it was really used to try

to change the value of a primary cell: the method would then actually arrive at the input value of the

primary cell, because the weight would be infinite with a zero distance.

The IDW method is considered particularly useful with hydrographic survey data. A weighed

average that is never negative will always return a value that is between the limits of the measured

values. This means that the method will never generate new undesired highs or lows, not even when

extrapolating from the outer skirts of the model.

‘Interpolated’ models are particularly useful with multi-beam surveys but can also successfully be

used with single-beam data.

5. Cleaning Methods

Cleaning can take place within the framework of NaviModel3 due to the fact that the raw

observations are inherent in the DTM. A series of different cleaning methods are implemented,

primarily:

Point Edit 3D Cleaning

Histogram Plane Cleaning

In addition to the two primary built-in cleaning methods, NaviModel3 facilitates the inclusion of

dedicated cleaning tools, termed plug-ins. A plug-in tool is here defined as an executable file that

reads a point-based data-set and writes back the (edited) result to the same file and consequently

back to NaviModel3, once the dedicated cleaning has taken place. Of particular interest in the

present context is the S-CAN automatic cleaning plug-in. Even though the application is rightfully a

plug-in to NaviModel3, it is actually, by default, installed, when installing NaviModel3.

Point Edit 3D Cleaning

The ‘Point Edit 3D’ cleaning tool is a conventional manual editing tool that was originally

developed to be used in EIVAs editing software, NaviEdit. By most users it is here employed on a

file-to-file basis.

The 3D-based cleaning, in which the user can see portions of the model area includes some manual

cleaning and visualization tools. The user is given the possibility, on the basis of the 3D-view, to

perform manual cleaning of areas as well as of single points. The clipping plane feature, through

which the user can limit the view of the data to an area between two planes, is thought to be

Page 7: Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient Sonar... · Efficient Sonar Data Cleaning ... 1. Introduction The objective of the

Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway

EIVA A/S Page 7 of 13 S-CAN in NaviModel3

particularly useful in connection with pipe data cleaning. The user can thereby edit data in front of

and behind an object (pipe) without removing the actual object of interest. It is furthermore possible

to show deleted points superimposed on the accepted ones. This includes points deleted in previous

cleanings sessions. Once the cleaning is accomplished, the DTM is updated in accordance with the

revised information.

Figure 5 Point Edit 3D cleaning

Histogram Plane Cleaning

The ‘Histogram Plane Cleaning’ method is a semi-automatic cleaning tool that, on the basis of a

user-defined polygon, will generate a least squares adjusted horizontal plane through the corners of

the polygon. The tool uses the plane for the subsequent cleaning. The selection tool, shown below

in Figure 6, is an XY-window that in the X-direction shows the distance to the plane, whereas the

Y-axis illustrates number of points. As a guide, proposed maximum and minimum values are

visualized with red vertical lines. These lines can be moved forth and back, on top of the

distribution curve (histogram), by manual selection and movements of the cursor.

Figure 6 Plane Distance selection tool (left) and 3D visualization

Another important visual selection guide is given in the 3D window where the consequence of the

selection can be monitored, with red dots indicating points to be deleted below as well as above the

plane. When accepting the settings, the DTM is promptly re-indexed and thereby updated.

Page 8: Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient Sonar... · Efficient Sonar Data Cleaning ... 1. Introduction The objective of the

Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway

EIVA A/S Page 8 of 13 S-CAN in NaviModel3

The S-CAN Automatic Cleaning Method

NaviModel3 supports incorporation of dedicated, user-developed plug-ins for cleaning and anti-

noise determination. The S-CAN (SCALGO Combinatorial Anti Noise) cleaning tool is such a

plug-in, developed in close corporation with the ‘Center for Massive Data Algorithms’

(MADALGO) at the University of Aarhus.

The development of the tool focused on automatic cleaning of the densely populated multi-beam

point-clouds, typically associated with pipe line surveys. The S-CAN algorithm calculates an initial

so-called ‘Noise Score’ for each data point in the data-set, and the user can then easily and

interactively clean parts of (or the whole) dataset by selecting a region of the data and remove

points with high noise scores. Alternatively the user is given the opportunity to use a more

differentiated approach to the selection of points to delete.

The score value of each sounding is determined or indexed in an initial, relatively processing-heavy,

step. The subsequent step of selecting areas with different and dedicated threshold values is

developed for efficiency. Here manual intervention is required.

The S-CAN is capable of efficiently processing large datasets that do not match the limitations of

internal memory but must reside on disk. The constant movement of data to and from disc during

the cleaning is not a performance bottleneck and the algorithm can therefore be regarded I/O-

efficient. This is ideal in a NaviModel3 environment, since this is exactly the way data is organised,

both the model-based data, through the governing Quad Tree principle, and the raw data-points,

with the latter being the subject of the cleaning.

The S-CAN plug-in comes in two different variants: a) the ‘Score’ variant and b) the ‘Components’

variant.

The Components Variant

The ‘Components’ variant separates input observations into series of observations that fulfil an

internal requirement regarding maximum tolerable height difference (threshold) between

neighbouring points. These neighbouring series are termed ‘surfaces’. The minimum height

difference between points in a surface and the neighbours in adjacent surfaces is consequently

higher than the threshold. Similarly, a large threshold separates into surfaces with high internal

difference/noise, whereas a small threshold will divide the observation material into more surfaces

as visualised in Figure 7.

Figure 7 Principle of the ‘Components’ variant

Page 9: Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient Sonar... · Efficient Sonar Data Cleaning ... 1. Introduction The objective of the

Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway

EIVA A/S Page 9 of 13 S-CAN in NaviModel3

The largest surfaces, in terms of population, are listed, in sequence, in the user interface, as shown

in Figure 9. The user is thereby given an opportunity to choose which ones to keep, as shown to the

right in the figure. If the initial threshold is not acceptable for the cleaning, a new score indexing,

with a revised threshold, must take place.

Figure 8 The user interface of the 'Components’ variant, with the ‘Surface’ selection window

The Score Variant

Compared to ‘Components’, the ‘Score’ variant calculates for all thresholds once and for all. This

optimizes the testing of the best possible threshold value for a given area and it furthermore honours

the fact that different areas might ideally be the subject to different threshold values. In the latter

situation, the user will have to perform the subsequent cleaning selection for each of these areas.

Figure 9 Principle of the ‘Score’ variant

Page 10: Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient Sonar... · Efficient Sonar Data Cleaning ... 1. Introduction The objective of the

Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway

EIVA A/S Page 10 of 13 S-CAN in NaviModel3

Figure 10 The user interface of the 'Score' variant, with the ‘Threshold’ selection window

Even though the ‘Score’ variant, as a consequence, is often substantially faster than the

‘Components’ variant, it can only be used in situations where one surface must be determined. A

pipe and a seabed can in many instances be regarded one surface. This is actually the case in the

examples given here. Similarly, ‘Components’ should be used when there is a larger variety in the

seabed features.

Most often however, combining the two variants will be an ideal solution, with ‘Score’ being used

as priority 1, because of its effectiveness, and ‘Components’ used in the remaining, more complex

areas.

Compared to other automatic cleaning algorithms commonly used today (usually based on TVU

(Total Vertical Uncertainty) assessments (TPE or CUBE variants)), S-CAN does not take the

conditions during acquisition, such as the characteristics of each instrument and the overall

geometry of the setup, directly into account. It is therefore not possible to make estimates of the

theoretical accuracy of each observation point and thereby use those as basis for the cleaning. The

method employed in S-CAN, where only the actually observed data are used as the basis for

cleaning, has however proven to be capable of efficiently separating noise clusters from points on

top of physical objects like pipelines.

Having said this however, it is anticipated that future developments of the S-CAN algorithm will

utilize some of data quality parameter (TVU), when determining the initial score value. This could

then possibly be regarded an addendum or an alternative to the two variants presently implemented.

6. Evaluation of the Efficiency of the S-CAN Cleaning Algorithm

The evaluation of the efficiency of the S-CAN cleaning has been based on the time-consumption

associated with the initial determination of the score values. For the evaluation, different hardware

platforms have been investigated, however the same dataset, originating from a dual head Reson 7K

ROV-based pipeline inspection survey, was used throughout. With 1024 points per scan and

approximately 10 scans per second, the dataset has been acquired at a rate of approximately

600.000 points per minute.

The figure below displays the relationship for the Score variant, between time consumption and

number of points on different hardware platforms, with the X-component showing the time

consumption in minutes and the number of points displayed as the Y-element.

Page 11: Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient Sonar... · Efficient Sonar Data Cleaning ... 1. Introduction The objective of the

Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway

EIVA A/S Page 11 of 13 S-CAN in NaviModel3

Figure 11 Performance of the ‘Score’ algorithm on different platforms

The characteristics of the various platforms associated with the colours of the figure are as follows:

the red line illustrates the performance of a Windows XP 64-bit operating system with 8 GB

usable memory, executed on a desktop computer with a 1.86 GHz Intel Core 2 Duo

processor and a 7200 RPM hard disc (≈ medium/low performance, 64 bit)

the orange line shows the results of performance of a Windows 7 32-bit operating system

with 3 GB usable memory, executed on a laptop computer with a 2.0 GHz Intel Core 2 Duo

processor and a 7200 RPM hard disc (≈ medium performance, 32 bit)

the green line demonstrates the results associated with a Windows 7 32-bit operating system

with 3 GB usable memory, executed on a desktop computer with a 2.67 GHz Intel Core I5

processor and a 7200 RPM hard disc (≈ high performance, 32 bit)

It appears from the figure, that the performance of all three configurations initially follows a linear

slope and then transforms into another linear slope, with the break-point between the two

inclinations being different for the three configurations. The inclination of the three lines as well as

the break-points, have been established as shown in the table below.

Break-point

Performance

before the break-point

Performance

after the break-point

64-Bit XP (8 GB RAM) 34 million points 1.8 million/minute 0.5 million/minute

32-Bit W7 laptop (3 GB) 12 million points 3.2 million/minute 0.6 million/minute

32-bit W7 desktop (3 GB) 14 (14) million points 6.9 (9.4) million/minute 1.7 (2.5) million/minute

The break-point for the various platforms is equivalent to 1.2 km (60 minutes of observation) of

data for the 64-bit and 500 m (25 minutes) for the 32-bit, with the dataset used for the evaluation.

The break-point is in other words a function of the RAM available for the algorithm. Once all points

to be cleaned can be contained in RAM, the algorithm is 4-5 times more efficient than when the

auxiliary memory on the swap-file must be applied to contain the points. Note that the values given

in parenthesis in the table above are associated with the ‘Components’ algorithm. The relationship

between the performances of the two algorithms on an identical platform is depicted below in

Figure 12.

Page 12: Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient Sonar... · Efficient Sonar Data Cleaning ... 1. Introduction The objective of the

Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway

EIVA A/S Page 12 of 13 S-CAN in NaviModel3

Figure 12 Performance of the ‘Score’ algorithm (green) against the ‘Components’ algorithm (blue)

The performance evaluation can be divided in two parts:

It has been verified that the S-CAN is actually capable of efficiently processing large

datasets that do not match the limitations of internal memory but must reside on disk and

that the constant movement of data to and from disc during the cleaning, that is a

consequence of this, does not appear to be a performance bottleneck. The algorithm can

therefore be regarded I/O-efficient

It is also clear, that for larger dataset it can sometimes prove beneficial to divide the initial

score determination into optimum parts, relative to the RAM available. When performance

is of outmost importance, substantial improvements can furthermore be achieved by

employing a 64-bit operating system with large amounts of RAM on a high performance

computer.

7. Summary

For the optimization of the post-processing environment within NaviModel3, a series of new

features have been implemented. The main contributor to the visual optimization is indisputably the

fact that all data of relevance within NaviModel3 is visualised in an integrated 3D based setting that

involves different views. Also the fact that video functionalities have been incorporated into the

modelling tool adds to an enhanced visual environment.

The speed-optimization and automation of the post-processing task has been concentrated on the

initial, processing-heavy fields, with a series of focal points.

With seabed mapping and investigations, such as pipe-line inspection, to a large extent depending

on reliable and accurate seabed information, some kind of efficient cleaning of the seabed scanning

data is considered a necessity. The cleaning of the multi-beam data has furthermore increasingly

become a topic where reliable automation is required. As a consequence of this, a series of cleaning

tools, manual as well as automatic, has been implemented in NaviModel3. For obvious reasons, the

automatic cleaning tools, the S-CAN variants, are the main contributors to the potential speed

increase, for the most part because they require a moderate user involvement. With the tools being

relatively simple to use, they can furthermore be regarded practically efficient.

Having said this, however, in some situations the manual and the semi-automatic cleaning tools

will, most often in combination with the automatic cleaning tools, provide optimum solutions.

Page 13: Efficient Sonar Data Cleaning - EIVAdownload.eiva.dk/online-training/NaviModel3 manuals/Efficient Sonar... · Efficient Sonar Data Cleaning ... 1. Introduction The objective of the

Seabed Mapping and Inspection 2011, February 9-11, Geilo, Norway

EIVA A/S Page 13 of 13 S-CAN in NaviModel3

Also the fact that NaviModel3 has an unlimited model-size, as a response to the vast increase in

data amounts over the last years, contributes noticeably to the speed-optimisation. The

implementation of a Quad Tree data structure as well as the fact that the raw data points are inherent

in the models, has resulted in an improved I/O efficiency, so that even though the data is residing on

larger and slower secondary storage devices, as opposed to having been loaded into the internal

memory, the speed of moving data to and from disc has not become a significant performance

bottleneck, not even in connection with the automatic cleaning sessions.

Having said this, however, it can also be concluded, based on the performance evaluation of the S-

CAN algorithm, that for larger dataset it is often favourable to divide the initial score determination

into optimum parts, relative to the RAM available. When performance is of outmost importance,

considerable improvements can finally be achieved by employing a 64-bit operating system with

large amounts of RAM on a high performance computer.

8. References

1. ‘Bathymetric Postprocessing Manual with NaviModel3’, EIVA A/S, 2010: http://download.eiva.dk/online-training/Bathymetric_Processing_Tutorial_wNM3.pdf

2. ‘Pipeline Inspection Tutorial with NaviModel3’, EIVA A/S, 2010: http://download.eiva.dk/online-training/Pipeinspection_Processing_Tutorial_wNM3_III.pdf

3. ‘Cleaning Massive Sonar Point Clouds’, L. Arge, K. G. Larsen, T. Mølhave and F. v.

Walderveen, ACM-GIS, 2010

4. Anti-noise plugin for NaviModel3, EIVA A/S, 2010. Press release at Hydro International: http://www.hydro-international.com/news/id3697-NaviModel_DTM_Software.html

5. Release of NaviModel3 DTM Software, EIVA A/S, 2010. Press release at Hydro

International: http://www.hydro-international.com/news/id3899-Antinoise_Plugin.html