Combined Acoustic and VideoCharacterization of Coastal Environmentby means of Unmanned Surface Vehicles

Elisa Fumagalli ∗ Marco Bibuli ∗ Massimo Caccia ∗

Enrica Zereik ∗ Fabrizio Del Bianco ∗∗∗ Luca Gasperini ∗∗

Giuseppe Stanghellini ∗∗ Gabriele Bruzzone ∗

∗ Consiglio Nazionale delle Ricerche - Istituto di Studi sui SistemiIntelligenti per l’Automazione, Via De Marini 6, 16149 Genova, Italy

(e-mail: elisa.fumagalli@ge.issia.cnr.it).∗∗ Consiglio Nazionale delle Ricerche - Istituto di Scienze Marine, Via

Gobetti 101, 40129 Bologna, Italy.∗∗∗ Proambiente Scrl, Via Gobetti 101, 40129 Bologna, Italy (e-mail:

f.delbianco@consorzioproambiente.it).

Abstract: The aim of this work is the exploitation of an Unmanned Surface Vehicle (USV)for visual and acoustic characterization of the seafloor in shallow water coastal environments,through combined acquisition of echograms and video images. To this purpose, the employedvehicle is composed by two systems: i) a leading USV, equipped with an advanced missioncontrol architecture, which performed a grid sampling over the study area; ii) a towed vehiclehosting an integrated system for acquisition of video and ultrasonic images, from the watercolumn and the seafloor, coupled with a GPS receiver, which provided the geo-referencing ofthe survey. Data acquisition was carried out over different seafloor types, which included rockoutcrops and sediments (mostly sand). The main key-points of employing such techniques tothis peculiar field of coastal area monitoring are the possibility to perform automated missionsand the capability to operate in very shallow water areas, usually not accessible to conventionalboats.

1. INTRODUCTION

Environmental monitoring of marine and oceanic areas isa fundamental step in order to guarantee sustainabilityand maintenance of ecosystems and habitats. These tasksare common for many fields of applications (including theevolution of flora and fauna habitats and their changesin response to natural or anthropogenic changes, and thestudy of marine erosion and deposition processes, whichcontrol redistribution of sediments along the coasts) andneed to be evaluated at different time-scale to suggeststrategies for coastal maintenance. Other fields of interestinclude detection of mineral deposits or morphologicalformations such as reef, underwater valleys, thermal vents,as indicators of geological processes that shape the seafloorin a given environment. More in general, identification ofphysical and biological connections is the key action toenhance policies of effective environmental management.In fact, the multidisciplinary knowledge of the natural en-vironments (in particular the coastal environment), whichis poorly investigated, heavily impacted by human activi-ties and prone to ecological crises, could guide protectionschemes based on effective exchanges between protectedareas; the European Project COCONET 1 is the actualframework whose main goal is the actual framework whosemain goal is the definition of guidelines to design, manageand monitor network of Mediterranean Protected Areas

1 http://www.coconet-fp7.eu/

(MPAs).Cultural aspects related to underwater archeology are alsoof great interest and include the discovery of ancient arti-facts, settlements or ship wrecks, as well as the protectionand maintenance of already known archaeological sites.For such reasons the continuous exploration and monitor-ing of underwater areas is an important task, which dealsmostly with observation and characterization of shallowwater areas. For instance, monitoring and safeguard ofPosidonia sea grass is of valuable interest; Posidonia mead-ows represent the ecosystem with the highest biodiversityin the Mediterranean Sea, providing also the most effectiveoxygen supply to superficial waters. Monitoring of erosionand deposition of marine sediments in coastal areas isfundamental to carry out effective maintenance, such asinstallation of protective barriers or sand replenishment.One of the most difficult tasks to be achieved is theperiodic observation and sampling of such areas, becausemonitoring technologies are expensive if performed on-board conventional oceanographic vessels. The introduc-tion of a robotic technology can overcome the problem ofassigning repetitive, tedious and expensive tasks to humanoperators, when such goals can be easily achieved by theemployment of highly autonomous marine platforms. Fur-thermore the access to many interesting shallow water ar-eas is usually dangerous or even impossible to conventionalvessels or boats; small robotic platforms characterized bya very small draught could access such difficult marineareas, reducing the risk of grounding.

Preprints of the 19th World CongressThe International Federation of Automatic ControlCape Town, South Africa. August 24-29, 2014

Copyright © 2014 IFAC 4240

Many studies and researches focusing on seafloor char-acterization can be found in the literature. The workin Waddington and Berman [2007] reports a method forbenthic habitat characterization. This work, after a briefoverview of a proposed acoustic technique used to gener-ate an initial broad-scale characterization of the seafloor,presents a detailed survey of the sediment-profile imagingtechnique that can be used as an efficient high-resolutiontool for ground-truthing the broad-scale acoustic data.In Tsemahman et al. [1997] a 50 Hz echo sounder com-bined with a digital wave form acquisition system anda seabed classification software were used to record andprocess echoes from a bay southeast Vancouver Island,Canada; each echosounding was preprocessed and severalstatistical and spectral algorithms were used to extractcharacteristic patterns. In Penrose et al. [2005] a review ofthe acoustic techniques for identification, classification andmapping of benthic habitats is given, as well as a report onthe commonly employed acoustic devices and subsurfacesensing techniques. The unmanned marine robotic vehicleROAZ, developed by the Instituto Superior de Engenhariado Porto, has been employed for bathymetric surveys inFerreira et al. [2009]; the aim of this work was collectingdata for risk assessment in shallow water environmentsand water-land transitional zones. The employment of anUnmanned Surface Vehicle (USV) for automatic bathy-metric survey is also presented in Yanfeng et al. [2011],furthermore including an adaptive isobath environmentalprofiles tracking. The task of collecting bathymetric datathrough autonomous vehicles in very shallow coastal, es-tuarine, and inland water environment is performed bymeans of a Small Waterplane Area Twin Hull (SWATH)vehicle, described in Kitts et al. [2012], which providesnatural platform stability for a multibeam sonar payload.The aim of this paper is to present how the robotictechnology represented by the Unmanned Surface Vehi-cles (USVs) can be effectively employed for biological orgeological observation and data acquisition of the coastalenvironment. In particular, the work focuses on the em-ployment of a double vehicle system, composed by a lead-ing vehicle in charge of navigating along a predefined sam-pling grid, towing a trimaran-shaped platform equippedwith video and echosounding devices. Video equipment isemployed to observe and characterize the seafloor environ-ment, recognizing sandy and rocky areas, evaluating areaextension of flora habitats, locating polluting objects andso forth. The echosounding device is firstly exploited toobtain a bathymetric map of the working area; throughthe analysis of raw acoustic signals and echo reflectionsand analyze reflectivity pattern as diagnostic of differentseafloor types.The test site for this combined visual and echographicsurvey was a shallow water coastal area (< 7m) closeto Murter Island (Croatia). Although, in general, suchsurveys are devoted to simply estimate the seafloor depth,digitally sampled echogram were used to infer the acousticproperties of the seafloor, i.e. the normal incidence reflec-tivity. An acoustic wavelet generated at the sea surfaceis partly reflected and scattered at the sediment/rock-water interface, and partly transmitted in the sub seafloor,generating further reflection within the marine sedimen-tary sequence. The returning echo contains componentsof the transmitted pulse with backscattering components

(see Hamilton [2011]). Typically, a flat and smooth seafloormay have a high percentage of energy reflected back to thetransducer, resulting in a trace with narrow peak with notail. Conversely a rough, complex seafloor may have highdegree of scatter, and traces could show peaks and tails.Nature of echoes are influenced by many aspects of seabedtopography such as grain size parameters and the presenceof benthic fauna and flora. All physical characteristics ofthe reflected acoustic waves depend on these properties.Analysis of such echoes may be crucial to determine physi-cal and biological nature of the seabed. This direct analysisof the seabed needs to be calibrated and ground-truthedby direct measures. Usually direct sampling of seabed isused for this purpose.

In this work, together with echosounding data, pictures ofseabed were collected using an underwater camera, whichallowed to qualitatively estimate the relationship betweenultrasonic echoes from the different seafloor types. Rate ofpicture acquisition and echosounding were synchronized inorder to have a perfect correspondence between position-ing of the two data-sets.The paper is organized as follows: Section 2 includes adescription of the employed robotic platforms and sen-sors, as well as a brief outline of the mission definitionand execution; Section 3 describes the mission design andexecution for video and acoustic data gathering, exploitingan autonomous robotic platform; the results obtained bythe data collection are reported and described in Section 4;conclusions and discussions on future steps are given inSection 5.

2. ROBOTIC SYSTEM

The combined sampling of video and acoustic data de-scribed in this work has been carried out through theemployment of a dual robotic platform system. It is com-posed by: i) the leading vehicle CART USSV, originallydeveloped within the European Project CART for shiptowing operation support tasks (an extensive description isreported in Bruzzone et al. [2013]); ii) the towed trimaran-like sampling platform, equipped with video and acousticdevices, that will be transformed into an autonomoussurface vehicle in a short-time upgrade.In the following of this section the characteristics of thetwo employed platforms will be described.

2.1 Leading vehicle

The CART USSV, developed by CNR-ISSIA and reportedin Figure 1, is a 0.9 m long and 0.75 m wide roboticplatform. The thrust is provided by four DC brushlessmotors coupled with 4-bladed propellers, capable of amaximum bollard pull of about 15 Kg. A central cylin-drical canister contains all the computation and sensingelectronic devices; in particular, the USSV is equippedwith a single board computer running a GNU/Linux basedreal-time control application, a GPS system providing theabsolute position fix and an AHRS (Attitude and HeadingReference System). Another cylinder contains a set ofLithium ions batteries providing energy for the vehicle for5÷ 6 hours of operations.For the goal of exploiting the CART vehicle as a testbedfor the development of advanced navigation, guidance and

19th IFAC World CongressCape Town, South Africa. August 24-29, 2014

4241

Fig. 1. CART USSV during operations

control techniques, as well as payload carrier in differentexperimental campaigns, the software control architectureas been upgraded porting an already extensively testedand exploited system. In particular the control architec-ture of the CNR-ISSIA Charlie USV has been customizedto be also able to run on the CART USSV. The port-ing operation basically regards the development of a newdriver layer, thus creating the connection between thehardware and logical variables; all the higher levels ofthe architecture do not require any adaption, because ofthe complete decoupling from hardware-related issues. Insuch a way the high level mission definition and executionmodule of the Charlie USV (described in Bibuli et al.[2008]) can be exploited to autonomously drive the CARTUSSV along an area coverage grid.The mission control subsystem is a high level moduleallowing the user to define and execute a completely cus-tomized mission plan. The key issue of this mission controlarchitecture is the high level of interaction between theuser and the mission controller itself. Apart from an off-linemission plan definition, the user is allowed to reconfigure oradapt the mission specifics during the on-line execution. Ahuman operator can add or update desired paths or motionprimitives, as well as control reference variables, e.g., cruisespeed. Moreover the mission behavior can be adapted asa function of environmental condition occurrences or usercommands, triggering the execution or the replanning ofparts of the original mission plan. This can be providedthanks to the intrinsic modular design of the missioncontrol module, which is composed of three main elements:i) data structures, from simple ones like booleans, integers,or real variables, to more complex ones like lists, queues,or stacks that can be used, for example, to store way-points or reference paths to be followed by the vehicle; ii)execution actions, like simple Petri nets whose markingdefines the actual state of the action, linked with semanticmodules that specify the primitives or commands relatedto the specific execution action; iii) control flow modules,also named containers, representing the topological inter-connection between the execution actions and thus theexecution flow, as for instance series, parallel, selectionand iteration flows. In the remainder of this section, acustom mission for a maritime security application will bedesigned. In this kind of framework, the operator coversthe role of the mission supervisor, monitoring the vehicle’sstate, evaluating time-changing operative condition (thatcan arise due to environment, traffic, unexpected situa-tions) and (semi-)automatic on-line mission replanning or

adaptation. The strong interaction between the humanoperator and the autonomous system, yields the roboticplatforms to be very effective tools in such a kind ofapplications.

2.2 Towed platform

The trimaran-shaped sampling platform is a robotic ve-hicle currently under development at CNR-ISSIA. Forthe task of a preliminary campaign of coupled video andacoustic data gathering, it was decided to exploit the stillnon-actuated platform mounting on that the video andacoustic devices, then linking the platform to a leadingvehicle with the task of towing the sampling platform.The trimaran-shaped sampling platform reported in Fig-ure 2, has a hull characterized by 1.20 m of length, 0.8m of width and 0.2 m of height (excluding electronic boxand antenna pole) dimensions. A single board computerrunning a GNU/Linux based real-time application col-lects the data gathered by the video and acoustic devices(mounted on the stern bottom side of the vehicle and thusnot visible in the picture) and integrates them with a geo-referenced fix obtained by a GPS system. The video ac-

Fig. 2. Trimaran sampling platform during pool trials

quisition system is based on an underwater camera, rigidlymounted on the USV hull, that was initially exploited formosaicking operations of shallow water seafloor by meansof an USV Ferreira et al. [2012]. The chosen camera ischaracterized by low weight (32g) and small dimensions(44 cm x 44 cm x 25.4 cm) implied neither a decreasein resolution (Wide-VGA with 752 x 480 pixels) nor alow frame rate (maximum 87 fps). The USB connectionthat serves as both power and data transmission erasesthe need of a separate power source. The lens used is an 8mm one. The focal length as well as the camera parameterswere calibration using a chessboard and the MATLABr

Camera Calibration Toolbox.

A vertical incidence echosounder, the PSA900 manufac-tured by Datasonics, was used to collect ultrasonic imagesof the water column and the seafloor. The PSA900 is par-ticularly suitable for shallow-water environments becauseis characterized by a high operating frequency (200 kHz),a narrow (8 , conical) beam width, a short pulse length(350 sec), and a minimum depth range of 0.75 m. The 200kHz echosounder signal was digitally within a constanttime window, and the data (the echograms) were stored inSEGY-format files (Barry et al., 1975).

19th IFAC World CongressCape Town, South Africa. August 24-29, 2014

4242

3. DATA COLLECTION MISSION

Relying on the mission definition and control architectureof the employed autonomous vehicle, an usual samplingmission (represented in Figure 3) over a lawn-mowersampling grid has been defined.Once the sampling grid has been defined (by settingdesired position, orientation, width, length and numberof transects), in the first phase of the mission all thegeometrical trajectories are computed and stored intoan internal data structure. As the main mission loopstarts, each reference transect is loaded and the vehicleis commanded to track it through a proper path-followingalgorithm (refer to Bibuli et al. [2009]). The reference pathis tracked until the end of the actual transect is reachedor if the remote human operator decides to switch to thefollowing reference line. The mission is completed whenall the reference sampling transects have been executed. A

Fig. 3. Mission scheme for autonomous data collection

simplified example of the motion of the vehicle executingthe sampling mission is reported in Figure 4, where it ispossible to notice the good tracking capabilities, keepinginto account that the area was subject to wind and seacurrent and that the leading vehicle was mostly affectedby the towed platform during the navigation. As shownin the picture, the vehicle, navigating at a speed of about0.4 ÷ 0.5 m/s, assumes an orientation in such a way tocompensate the effect of the external environmental andtowing-induced disturbances.

4. COLLECTED RESULTS

The first step of data processing was to estimate water-depths from the raw echograms. To this purpose, theprocedure described in GASPERINI 2005 was followed,computing an amplitude envelope trace by convolving thesquared values of the original data trace with a rectangularwindow of the same width of the source pulse. Conversionof travel-times into water-depth was performed assuminga constnt sound velocity of 1500m/s. The acquisition of

Fig. 4. Leading vehicle motion during lawn-mower missionexecution

the entire echogram at each sounding point, rather thanthe simple depth estimate, gives the opportunity to cal-culate. In the case of vertical incidence, neglecting theeffect of energy scattering due to the bottom roughness,it is possible to calculate the reflection coefficient (R)

using: R =Ay

Asz where Ar and As correspond to the

amplitude of the source and reflected signals respectively,and z is the water depth. In order to obtain an estimateof R from the finite-length echosounder pulse, As and Ar

are expressed as:As =∑W

i=0 |xi| and Ar =∑B+W

i=B |xi|where xi represents the digital sampled signal, W is thewidth of the source pulse and B is the bottom detec-tion time. Once reflectivity data were obtained a com-parison between reflectivity patterns and video imagingof the seafloor was performed. In fact although seafloorreflectivity is dependent on several physical and biologicalaspects, such as colonization by plants and animals, diage-nesis, etc. an interesting matching between reflectivity andseafloor geological properties (such as mean grain size ofthe sediment) has been observed in similar shallow-waterenvironments without visual inspections (e.g. Gasperini2005). Echograms were collected in depths ranging from1m to 7m over different environments characterized bysandy, rocky, or mixed seafloor types. Digital sampling ofthe ultrasonic signal was performed at constant samplingrate (NS = 10000) while the ping rate was 8 per secon.In this way an acquisition time window of 8ms per pingwas obtained, corresponding to a depth of 6m, assuming aconstant sound velocity of 1500 m/s. Echosounder profilesshown in figures 5 and 6 were obtained using Seisphro, aninteractive computer program for processing and interpret-ing high-resolution seismic reflection profiles (Gasperiniand Stanghellini [2009]).

This data representation is obtained by plotting signalamplitude during propagation time (y-axis) vs shot num-ber, represented on the x-axis. Amplitude of the ultrasonicsignal is displayed as colour variations along the y-axis.The two profiles presented in figures 5 and 6 are a selectionof 1500 pings collected using the same acquisition param-eters, allowing a qualitative comparison between different

19th IFAC World CongressCape Town, South Africa. August 24-29, 2014

4243

Fig. 5. Left: example of echographic profile in sand-dominated seafloor. Right: video image collected in the same area

Fig. 6. Left: example of echographic profile in rocks-dominated seafloor. Right: video image collected in the same area

acoustic responses of the seafloor, in terms of reflectivityand morphology. We assume that the intensity of the re-flections from the seafloor is related to physical characteris-tics (density and sound velocity) of the media. By this sim-ple approach it is possible to carry out a general qualitativeanalysis of seafloor types. However, since an estimate ofthe reflection coefficient at the seafloor was performed,this qualitative observation could provide a quantitativeevaluation of reflectivity coefficient represented by R. Fig-ure 5 displays an echogram obtained in a homogeneous,sandy seafloor. We observe a smooth morphology of theprincipal reflector representing the seafloor and relativelylow values of R. This ultrasonic pattern is ground-truthedby underwater imaging, which show in correspondence ofrelatively low R values a smooth, sandy, homogeneousseafloor. A different ultrasonic pattern is observed in figure6. This profile shows a high degree of signal scattered andreflected from the seabed, with a more marked reflector,characterized by a complex morphology and the reflectoris generally thicker. This setting corresponds to higheraverage values of R and a more ”patchy” variability ofseafloor reflectivity. It suggests the presence of a rockyseafloor, as confirmed by analysis of video imaging (figure6). In Figure 7 an echogram characterized by compositepattern (rocks and sand) is shown; pictures aside theacoustic profile are related to the points marked in theechogram. Every point corresponds to a set of about tenpings (echoes) and every ping is related to nearly aboutsix pictures collected with the camera.

In this profile (figure 7) characterized by high lateralvariation of seafloor reflectivity, the areas characterizedby sandy or rocky seafloor are marked by an numberedarrow, which corresponds to a picture to the right of theechogram. The normalized seafloor reflectivity is plotted atFigure 6 bottom.By the compared analysis of reflectivityand video imaging of the seafloor, we note that higherreflectivity values correspond to the rocky seafloor, whilethe sandy, relatively smooth, areas show lower values. Thegood fit between ultrasonic and visual imaging enable usto future extension of this techniques to a compositesuiteof seafloor types, producing a catalogue of case historiesthat could cover the geological/biological variability of thecostal environment. This could be eventually used in caseswhere visual inspection are not feasible due to the absenceof light or the presence of light-scattering particles in thewater column.

5. CONCLUSIONS

These preliminary tests of combined echosounder andvisual seafloor observation suggest the possibility of us-ing such techniques to perform habitat mapping of theseafloor in shallow-water environment using UnmannedSurface Vehicles. nvestigate seabed composition directlyfrom echograms. One of the future development will in-clude the analysis of the seabed response in terms of re-flectivity, backscattering and frequency analysis, to extractwave parameters that can be linked to physical properties.Comparison between underwater images and echograms

19th IFAC World CongressCape Town, South Africa. August 24-29, 2014

4244

Fig. 7. Echogram of the seafloor across different domains. Numbered arrows indicate the location of seafloor picturescollected at any given point. Reflectivity curve (red line, figure bottom) shows that higher R-values correspond toa rocky seafloor.

allowed to test the combined video and echosounding ac-quisition system and constitutes a first step to developmethods of nearly real-time analysis of seabed morphol-ogy. This work has shown the effectiveness of autonomousrobotic platforms for monitoring the shallow-water coastalenvironment, relying on an advanced automatic missionexecution system. Finally the real-time(or quasi real-time)knowledge of bathymetry and seafloor profile can be ex-ploited to feed the mission execution system and carry outadaptive missions to detect a particular seafloor pattern ora given marine habitat.

ACKNOWLEDGMENTS

Authors want to thank Giorgio Bruzzone, Mauro Gia-copelli, Edoardo Spirandelli for the technical work andFausto Ferreira for the development of image samplingsystem.This work has been supported by the project MORPH(EU FP7), Grant Agreement n. 288704, within the scopeof advanced methods and technologies for autonomous sea-bottoms bathymetric characterization.

REFERENCES

M. Bibuli, G. Bruzzone, and M. Caccia. Mission control forunmanned underwater vehicles: functional requirementsand basic system design. In Proc. of IFAC NavigationGuidance Control of Underwater Vehicles Conference,2008.

M. Bibuli, G. Bruzzone, M. Caccia, and L. Lapierre. Path-following algorithms and experiments for an unmannedsurface vehicle. Journal of Field Robotics, 26(8):669–688, 2009.

G. Bruzzone, M. Bibuli, M Caccia, and E. Zereik. Co-operative robotic maneuvers for emergency ship towingoperations. In MTS-IEEE Oceans, Bergen (Norway),June 2013.

F. Ferreira, M. Bibuli, G. Bruzzone, and M. Caccia.Enhancing autonomous capabilities and human-robotinteraction for unmanned surface vehicles. In Proc. of

IEEE 20th Mediterranean Conference on Control andAutomation, 2012.

H. Ferreira, C. Almeida, A. Martins, J. Almeida, N. Dias,A. Dias, and E. Silva. Autonomous bathymetry for riskassessment with roaz robotic surface vehicle. In Proc.of the MTS/IEEE Oceans Conference, 2009.

Luca Gasperini and Giuseppe Stanghellini. Seisprho: Aninteractive computer program for processing and in-terpretation of high-resolution seismic reflection pro-files. Comput. Geosci., 35(7):1497–1507, July 2009.ISSN 0098-3004. doi: 10.1016/j.cageo.2008.04.014. URLhttp://dx.doi.org/10.1016/j.cageo.2008.04.014.

L.J. Hamilton. Acoustic seabed segmentation forechosounders through direct statistical clustering ofseabed echoes. Continental Shelf Research, 31, Decem-ber 2011.

C. Kitts, P. Mahacek, T. Adamek, K. Rasal, V. Howard,S. Li, A. Badaoui, W. Kirkwood, G. Wheat, andS. Hulme. Field operation of a robotic small waterplanearea twin hull boat for shallow-water bathymetric char-acterization. Journal of Field Robotics, 29(6):924938,2012. doi: doi: 10.1002/rob.21427.

J. D. Penrose, P. J. W. Siwabessy, A. Gavrilov, I. Parnum,L. J. Hamilton, A. Bickers, B. Brooke, D. A. Ryan, andP. Kennedy. Acoustic techniques for seabed classifica-tion. Technical Report 32, Cooperative Research Centrefor Coastal Zone Estuary and Waterway Management,September 2005.

A. S. Tsemahman, W. T. Collins, and B. T. Prager.Acoustic seabed classification and correlation analysisof sediment properties by QTC VIEW. In Proc. of theMTS/IEEE Oceans Conference, 1997.

T. Waddington and G. Berman. The use of acoustic andoptical imagery data to characterize benthic habitats.In U.S. Hydro Conference, Norfolk, Virginia (U.S.A.),May 2007.

W. Yanfeng, J. Jiucai, G. Sheng, and Z. Jie. Simula-tion of adaptive isobath tracking for unmanned surfacebathymetry vehicle. In Proc. of the 4th InternationalConference on Intelligent Computation Technology andAutomation, 2011.

19th IFAC World CongressCape Town, South Africa. August 24-29, 2014

4245