Post on 30-May-2020
A Review of Calibration, Validation and 3D Measurements for
Underwater Stereo-video Systems Mark Shortis
School of Mathematical and Geospatial Sciences RMIT University, Melbourne, Australia
ISPRS / CIPA Workshop UNDERWATER 3D RECORDING & MODELING
16-17 APRIL 2015 Piano di Sorrento, Italy
Acknowledgements Euan Harvey and Nader Boutros
University of Western Australia/Curtin University
Alan Williams and Bruce Barker CSIRO Marine and Atmospheric Research
Mike Cappo Australian Institute of Marine Science
Why Bother with Calibration? • Unnecessary for low accuracy applications. • Use a correction lens (Ivanoff &Cherney, 1960). • Use the classic grid approach to take
measurements within a consistent XY system.
nauticalarch.org
Why Bother? 2D+ or Accuracy! • Refraction effects must be modelled to avoid
systematic effects and scale errors.
listverse.com
Why Accurate Size is Important - Age
Napier and Goodland, 1997 Snake River chinook salmon
Beer, 1998
Four sets, Sept 2002, Bowling Green Bay; stereo-rig - 248 measurements
Lower end of 100mm length class
-20 80 180 280 380 480 580 680 780 880 980 2000 2200
Freq
uenc
y
0
10
20
30
40
50Carangids, Nemipterids
Spanish mackerel
School mackerel
Serranids, lutjanids, lethrinids, pomacanthids
Maori wrasse, tuskfish
shark ray
Balistids, lutjanids
A Concise History
1850
1915
1930
1945
First underwater images using glass plates
First underwater movie
O-ring seals make water-tight housings possible
AquaLung invented by Coutseau
1949
1957
1960
Rolleimarin underwater housing
Waterproof 35mm camera invented
First seabed and archaeological mapping
1950s Surges in SCUBA diving, photography, underwater feature movies, exposure of marine environment
1980s
2006
2000s
Stereo-photos used for quantitative analysis and underwater stereo-video adopted by marine science
GoPro revolution starts
First solid state video recording systems
1970 Film stereo cameras with correcting lenses
1990s Digital still cameras arrive and DV tape released
Self or System? Pre- or In-situ? • Relatively few examples of self-calibration using
the object of interest (Schewe et al, 1996). • Pre-calibration with a calibration fixture (Harvey
& Shortis, 1996) or a ‘Manhattan’ object (Turner, 1992) is the typical approach.
• Some mechanism for scale determination must be included (Menna et al, 2013).
• Scale can also be determined from known distances on the calibration fixture.
• Stability of the camera and optical path are critical.
Camera Calibration Algorithms • Absorption by the standard physical
parameter set (Fryer and Fraser, 1986; Harvey and Shortis, 1996; Schewe et al, 1996; Menna et al, 2013)
• Geometric correction (Li et al, 1997)
• Perspective centre shift (Telem and Filin, 2010)
• For stereo-cameras, the relative orientation is
computed using a constrained network solution or from post-processing of network output data.
Pros and Cons Technique Advantages Disadvantages
Correction Lenses Simple solution. Still requires calibration for high accuracy.
Absorption Standard physical parameters, no change to calibration algorithm used in air.
May still be some residual effects that are not modelled accurately.
Geometric Correction In theory can fully correct for refraction for a specific lens cover type.
Complex, two phase solution that is specific to the optical path geometry.
Perspective Shift In theory can fully correct for refraction for a specific lens cover type.
Solution is specific to the optical path geometry. Some physical terms are neglected, assumed to be absorbed by the standard calibration parameters.
Absorption • The principle effect of refraction is radial
distortion about the optical axis. • Small non-radial effects are absorbed by
decentring lens distortion and affinity terms.
Geometric Correction • Li et al 1997: specific
to concave lens covers. • Two phase calibration:
– In air calibration first, standard parameters.
– In water calibration second, 11 lens cover parameters and 2 refractive indices.
– Refractive indices of air and water can be pre-determined or estimated.
Perspective Centre Shift • Telem and Filin 2010: specific solution for a
planar housing/port. • Parameters include:
– standard physical parameters, – refractive indices of glass and water, – distance between the perspective centre and the
port, – tilt and direction of angle between optical axis and
normal to the housing port, and – housing interface thickness.
Perspective Centre Shift (2)
Some parameters are not included, and any remaining signal is absorbed by the standard calibration parameters.
Calibration Fixtures – Image Space • Size is a critical factor, can be partly
compensated by moving around within the FoV. • 3D is a critical factor, compensation as above.
www.vision.caltech.edu
Calibration Fixtures – 3D Space • Size is a critical factor, the fixture must suit the
expected distance range of measured objects. • The fixture should be moved to ‘fill’ the range.
Calibration Results Technique RMS Image
Error (pixels) RMS XYZ Error
(mm) Proportional Error
Shortis & Harvey, 1996…. absorption
0.1 - 0.3 0.1 - 0.5 1:3000 – 1:15000
Schewe et al 1996 absorption
0.3 40-200 1 : 500
Li et al 1997 geometric correction
(1.0) 10 1 : 210
Sedlazeck & Koch, 2009…. geometric correction
0.2 5 -
Telem & Filin 2010 perspective shift
0.3 2.0 1 : 1000
Menna et al 2013 absorption
0.2 – 0.25 1.9 1 : 32000
Accuracy Validation • RMS image residual not a good indicator. • Corrections to target XYZs or lengths are useful. • Better to use scale bar checks or silhouette
measurements throughout FoV.
Validation Results Technique Validation Percentage Error
Shortis & Harvey, 1996… absorption
Length measurement of silhouettes or rods throughout
the volume
0.2 – 0.7 %
Petrell et al, 1997 lens distortion grid
Calliper measurements of Chinook Salmon
1.5 %
Harvey et al, 2003 absorption
Calliper measurements of Southern Bluefin Tuna
0.2 %
Telem & Filin 2010 perspective shift
Flat reference plate and straight line re-construction
0.4 %
Menna et al 2013 absorption
Similarity transformation between above water and
below water networks
0.3 %
Wehkamp & Fischer 2014 Matlab and radial lens distortion
Distances on checkerboard 0.9 – 1.5 %
Calibration Stability • Estimated using
parameter changes between sequential calibrations.
• Lengths on the fixture ensures scale stability.
• Calibration applies to the entire optical path.
• Camera-port interface is a critical sensitivity.
0
5
10
15
20
0 2 4 6 8 10 12 14 16
Cha
nge
Sign
ifica
nce
Calibration Date (Days)
Open Water Right Camera Calibrations
PPx
PPy
PD
Rad Distn
Dec Distn
Affinity
Case Study • 3D cube using a self-calibrating
photogrammetric network compared with A3 and A4 checkerboards using the Bouget Matlab toolbox solution (Boutros et al, 2015).
• Stereo-video system of two Sony HDR-CX700 HD format camcorders with purpose-built housings designed to minimise movement.
• RO included as an integral part of the tests. • Also compared a range of three base
separations: 150mm, 400mm and 830mm.
Diver and AUV Systems Study Subject Range (m) Base (mm)
Seiler et al 2012 Marine fish < 2 78
Williams et al 2010 Benthic habitat < 2 78
Shortis et al 2000 Shellfish 0.5 250
Abdo et al 2006 Sponges 0.5 336
Ruff et al Captive fish 1 – 2 500
Langlois et al 2012 Reef fish 0.5 – 8 700
Santana-Garcon et al 2014 Pelagic fish 1 – 10 700
Harvey et al 2003 Southern Bluefin Tuna 2 – 8 800
Seager 2014 Whale sharks 10 – 12 900
Harvey & Shortis 1995 Reef fish and habitat 5 1400
Effect of Calibration on Biomass Estimates
Percentage error for a hypothetical 400mm snapper (Pagrus auratus)
Range (m) Cube A3 A4
2 0.00 0.91 2.12
3 0.49 0.09 0.54
4 1.07 0.92 0.02
5 1.49 2.67 10.20
6 1.88 6.03 3.57
7 2.45 7.98 16.01
8 2.03 12.18 42.90
Effect of Base on Biomass Estimates Range (m) 800mm 400mm 150mm
2 0.22 0.82 1.35
3 1.42 5.06 1.60
4 0.54 0.76 3.32
5 1.27 0.78 4.29
6 1.54 1.22 6.77
7 1.87 0.47 3.31
8 1.38 0.83 2.83
9 1.44 0.70 6.66
10 1.45 4.31 7.67
11 1.69 2.60 35.06
12 0.32 5.41 19.86
Percentage error for a hypothetical 400mm snapper (Pagrus auratus)
Analysis
• A 3D calibration object clearly produces improved accuracy, precision and reliability.
• Short baselines give surprisingly good results, but deteriorate quickly at longer ranges.
Future Developments • GoPro revolution is generated a need for rapid
calibration of different modes of operation. • Digital SLR cameras require (separate)
calibration in still and movie modes. • Different algorithms are likely to suit different
housings. • Simple, glass or acrylic plane surface
waterproof housings may suit absorption. • Curved acrylic housings may produce
improved results with geometric corrections.
Questions? Principal References: Boutros, N., Harvey, E. S., Oldham, C. and Shortis, M. R., 2015. Calibration and
configuration of underwater stereo-video systems for applications in marine ecology. In press for Limnology and Oceanography: Methods.
Harvey, E. S. and Shortis, M. R., 1996. A system for stereo-video measurement of sub-tidal organisms. Marine Technology Society Journal, 29(4): 10-22.
Ivanoff, A., and Cherney, P., 1960. Correcting lenses for underwater use. Journal of the Society of Motion Picture and Television Engineers, 69(4):264-266.
Li, R., Li, H., Zou, W, Smith, R. G. and Curran, T. A., 1997. Quantitative photogrammetric analysis of digital underwater video imagery. IEEE Journal of Oceanic Engineering, 22(2): 364-375.
Menna, F., Nocerino, E., Troisi, S. and Remondino, F., 2013. A photogrammetric approach to survey floating and semi-submerged objects. Videometrics, Range Imaging, and Applications XII, SPIE Vol. 8791: paper 87910H.
Schewe, H., Moncreiff, E. and Gruendig, L., 1996. Improvement of fish farm pen design using computational structural modelling and large-scale underwater photogrammetry. International Archives Photogrammetry and Remote Sensing, 31(B5) : 524-529.
Telem, G. and Filin, S., 2010. Photogrammetric modeling of underwater environments. ISPRS Journal of Photogrammetry and Remote Sensing, 65(5): 433-444.
Turner, J., 1992. The development of an operational digital photogrammetric system for the North Sea oil and gas industry. Videometrics II, SPIE Vol. 2067, pp 136-144.