Real-time geo-referencing of video images for forest fire using … · 2019. 5. 17. · 16/10/2013...
Transcript of Real-time geo-referencing of video images for forest fire using … · 2019. 5. 17. · 16/10/2013...
16/10/2013
1
16/10/2013
RPAS voor rampbeheer
Lewyckyj Nicolas ([email protected])
VITO – Remote Sensing Unit - Mol
Vlaamse Instelling voor Technologisch Onderzoek
Vision on Technology
16/10/2013 2 © 2010, VITO NV
VITO
sites Berchem » MIP2 » VBBV » VBAV
Greenbridge » FCA » West-Flanders
Mol » Head office » Transition, Environment, Remote sensing, Materials » SME’s Limburg, Antwerp, Flemish-Brabant
EnergyVille » Sustainable energy
Ghent » KMO » East-Flanders
> 700 employees
16/10/2013 3 © 2010, VITO NV
VITO : 8 Activity domains
Activity domains
Environmental modelling
Energy technology
Transition energy &
environment
Separation and
conversion technology
Materials technology
Remote sensing
Environmental analysis & technology
Environmental risk & health
16/10/2013 4 © 2010, VITO NV
SCK CEN Voluntaries - 1996
16/10/2013 5 © 2010, VITO NV
Structure of the presentation
» Remotely Piloted Aircraft System (RPAS)
» Payload
» Imaging sensors
» Non imaging sensors
» From Data to Information
» Legal aspects
» Trends
16/10/2013 6 © 2010, VITO NV
Structure of the presentation
» Remotely Piloted Aircraft System (RPAS)
» Payload
» Imaging sensors
» Non imaging sensors
» From Data to Information
» Legal aspects
» Trends
16/10/2013
2
16/10/2013 7 © 2010, VITO NV
Evolutie UAV concept
» UAV : Unmanned Aerial Vehicle
» UAS : Unmanned Aerial System
» UAS : Unmanned Aircraft System
» RPAS : Remote Piloted Aircraft System
16/10/2013 8 © 2010, VITO NV
San Franscisco - 1906
16/10/2013 9 © 2010, VITO NV
RPAS can be a very complex system
» Aircraft
» Payload (e.g. camera system)
» Ground Control Station
» Pilot(s)
» Communication systems
» Ground Infrastructure (catapult, airport,…)
16/10/2013 10 © 2010, VITO NV
16/10/2013 11 © 2010, VITO NV
Op het terrein
16/10/2013 12 © 2010, VITO NV
Central Data Processing Center
(CDPC) at VITO (Mol)
SPS
Web services (OGC compatible)
Interface
aircraft
planning
Interface
access to
catalogue
CWS
Production of the
requested images
WMS-WCS
Optical
fiber
Central Data Processing Center
(CDPC) at VITO (Mol)
SPSSPSSPS
Web services (OGC compatible)
Interface
aircraft
planning
Interface
access to
catalogue
Interface
access to
catalogue
CWSCWSCWS
Production of the
requested images
WMS-WCSWMS-WCS
Optical
fiber
InternetInternetInternet
Op VITO: beeldverwerking en webservices
16/10/2013
3
16/10/2013 13 © 2010, VITO NV
Satellites
Manned systems
Flying
Altitud
Global coverage
Low-Medium resolution
HAPs Regional coverage
High resolution
Loca/regional coverage
High resolution
New sensors (hyper)
“Small” RPAS Local coverage
Very high resolution
Remote sensing for civil applications
Flying
Altitude
MAV
16/10/2013 14 © 2010, VITO NV
Unmanned Aerial Vehicles (UAVs)
Micro Flying
Robot - Japan
Helios (Aerovironment/NASA, USA)
Sanswire (USA)
Delfly Micro
TUDelft
Global Hawk – US Army
16/10/2013 15 © 2010, VITO NV
Light weight RPAS : < 150 kg
» Gliders, fixed frame, rotary systems (up to 8 screws, 1 or 2 turbines),
paragliders, balloons (with or without tether)
» Altitude of 50 cm up to ~ 20 km
» Endurance : few minutes to weeks
» Payload capacity : few grams to > 40 kg
» Operated VLOS, LOS or BLOS
» Launched : hand, catapult, trolley, car, using wheels
» Recovered : on grass, concrete, water, net, hand, parachute, ...
» Energy: batteries, fuel, solar, laser
» Use : indoor or outdoor
16/10/2013 16 © 2010, VITO NV
16/10/2013 17 © 2010, VITO NV
Requirements for security applications
» Security (police, fire brigade, civil protection,…)
» Easy and rapid deployment
» Easily transportable
» Easy to pilot or automated system
» Limited in spatial coverage but not always
» Real-time often required
» Limited or no ground infrastructure available
» Must be operable under all meteorological conditions (wind, rain)
» Day and night operations
» “Low” cost
16/10/2013 18 © 2010, VITO NV
Focus of the presentation
» Total take-off of max. 35-65 kg for the RPAS
» Total payload mass up to 10-12 kg,
» but with more focus on “few kg” systems
16/10/2013
4
16/10/2013 19 © 2010, VITO NV
Multicopter systems
» MTOW : few kg
» Endurance : typically between 10-40 minutes (sometimes more)
» Payload : varying between 0,5 - 1,5 kg
» Wind resistance : up to 5 Beauforts (30-40 km/h)
» Most are not waterproof
» Operational : within few minutes
» Easy to fly manually (relatively stable)
» No ground infrastructure required
» Autopilot,
» Real time datalink
» Some examples : Microdrone, Microkopter, Altura, Airrobot,
Draganfly, Falcon8,….
16/10/2013 20 © 2010, VITO NV
» Multicopter systems : some examples (from Google)
ETC…
16/10/2013 21 © 2010, VITO NV
Fixed wing systems
» MTOW : few kg
» Endurance : typically between 30-60 minutes
» Payload : varying between 0,5-1,5 kg
» Wind resistance : up to maximum 6 beauforts
» Most are not waterproof
» Operational : within few minutes
» Less easy to fly manually = > mostly automated => autopilot
» Often bailey landing => no external camera.
» Limited number of system with real time datalink
» Some examples :Gatewing X100, Smartplane, Sumo, etc…
16/10/2013 22 © 2010, VITO NV
» Fixed wing systems (hand launch or small catapult)
ETC…
16/10/2013 23 © 2010, VITO NV
For regional monitoring
» Larger systems (MTOW ~ 30-60 kg)
» Higher endurance (hours)
» Larger payload capacity (~10 kg) => multiple sensors payload
» Require ground infrastructure (airport, larger catapult, …) for FW
» More complex Ground Control Station system
» Higher level of RPAS safety (legal requirements)
» Higher level of pilot experience/capacity/training
» Communication with ATC
» …
16/10/2013 24 © 2010, VITO NV
Colibri (DARPA project)
» http://www.avinc.com/nano
» vertical – horizontal flight capacity
» C&C, batteries
» Video camera + real-time data link
» Endurance up to 12 minutes
» Weight = 19 g
16/10/2013
5
16/10/2013 25 © 2010, VITO NV
Structure of the presentation
» Remotely Piloted Aircraft System (RPAS)
» Payload
» Imaging sensors
» Non imaging sensors
» From Data to Information
» Legal aspects
» Trends
16/10/2013 26 © 2010, VITO NV
The different parts of the payload are
» Sensors (imaging and non imaging)
» Instruments for the navigation (IMU, GPS)
» Data Handling Unit (computer, data storage, interfaces)
» Datalink (LOS or not)
» Power supply (generator, batteries,…)
» Stabilizating mount, gamble (turrets)
» Communication (hub, collaborative systems)
16/10/2013 27 © 2010, VITO NV
“Sensors” are numerous
» “Imaging”
» Fixed frame camera, line scan, video
» RGB, multi-hyperspectral, IR, Thermal,
» Synthetic Aperture Radar
» Lidar
» …
» Non imaging
» Atmospheric measurements, ash cloud, gazes
» Movement, light, sound, …
» Meteorological instruments (wind, temperature)
» Radiaoactivity
» Communications
» Sensors can be dropped from the UAS !!
16/10/2013 28 © 2010, VITO NV
Structure of the presentation
» Remotely Piloted Aircraft System (RPAS)
» Payload
» Imaging sensors
» Non imaging sensors
» From Data to Information
» Legal Aspects
» Trends
16/10/2013 29 © 2010, VITO NV
“Conventional” payloads
Wescam MX-15 (AAT)
16/10/2013 30 © 2010, VITO NV
Panasonic DMC LX3 ( ~ 350 €)
260 g – 10 Mpixels – 24 mm wide angle
Canon EOS 5D ( ~ 2.500 €)
> 800 g (only body) – 21 Mpixels – Full HD video
Examples of commercial RGB cameras
16/10/2013
6
16/10/2013 31 © 2010, VITO NV
Examples of very light weight video cameras
Panasonic KX-141 colour
~12 g, 480 lines Misumi 8x8mm color
CMOS video camera
Ueye USB UI-1228LE
16/10/2013 32 © 2010, VITO NV
Examples of light weight gimbals (VIS+IR)
MicroPilot (<900g)
TASE, (Cloud Cap technologies), <1Kg , ~ 60 k€
HE 60 C auto (HighEye)
Paylaod < 12 kg
OTUS-135 (DST),
1Kg, 25-50k€
16/10/2013 33 © 2010, VITO NV
Examples of uncooled light weight IR cameras
Miricle 110K, body 84 g Thermovision , ~120 g
Sony XCEI50CE B/W Analog
Near IR Camera, 60 g
HAWK by Raptor Photonics Ltd,
<150g, Analogue or digital
Quark (FLIR), 25mm , 28 g
XS-1.7-320 (Xenics), 225 g
16/10/2013 34 © 2010, VITO NV
Structure of the presentation
» Remotely Piloted Aircraft System (RPAS)
» Payload”
» Imaging sensor
» Non imaging sensors
» From data to Information
» Trends
16/10/2013 35 © 2010, VITO NV
Ionising radiation
» University of Reading
LND714 Geiger tube (sensitive to beta and
gamma radiation)
16/10/2013 36 © 2010, VITO NV
Automatic sampling system for use on UAV
Protootype:
Motor-driven rotating wheel
supports 7 substrate discs
Pump, flowmeter, HV and power
supply,
Accu, electronics
Dimensions: 25 x 20 x 20 cm
Weight: 2.6 kg + power
supply
Power: 12 V, 50 W
Goethe-University, Frankfurt/M., Germany
16/10/2013
7
16/10/2013 37 © 2010, VITO NV
Very light weight COTS systems tested at VITO
temp, pressure and light sensors
temp, movement and light
16/10/2013 38 © 2010, VITO NV
» Light weight
Aethalometer AE-51
Black Carbon
GPS antenna
Processing
GPRS antenna
VITO test with Aethalometer
16/10/2013 39 © 2010, VITO NV
Payload data link
» VLOS (Wifi, zigbee, GPRS, analogue, digital..)
» LOS (~ 200 Km) => tracking system
» Satcom, irridium,…
16/10/2013 40 © 2010, VITO NV
Some examples
HICAM MV 500
33 g, ~ 300 €, An.,
2 W, range ~ 900m
Edimaeg MMC-1105(S)
155 g, ~ 1500 €, Dig.,
1.5 W, range up to 15 km
Cobham High Definition Messenger Transmitter
380 g, ~ 85 k€, Dig. , 22 W, range ~ 1.5 km (omni)
64 Mbps
Active robots
< 50 g, ~ 70 €, Dig.,
1.5 W, range ~ 100m
Up to 115 kbps
16/10/2013 41 © 2010, VITO NV
Structure of the presentation
» Remotely Piloted Aircraft System (RPAS)
» Payload
» Imaging sensors
» Non imaging sensors
» From data to Information
» Legal aspects
» Trends
16/10/2013 42 © 2010, VITO NV
Added value of composite images
MercatorLow.mpg
16/10/2013
8
16/10/2013 43 © 2010, VITO NV
Video opgenomen met
een kleine video camera (~100 €)
Afstand > 1000 m
16/10/2013 44 © 2010, VITO NV
Composietbeeld van de video
16/10/2013 45 © 2010, VITO NV
Georeferenced images provided in real time from
video stream
16/10/2013 47 © 2010, VITO NV
Trucks
OSIRIS FP6 Project
16/10/2013 48 © 2010, VITO NV
16/10/2013
9
16/10/2013 49 © 2010, VITO NV
16/10/2013 50 © 2010, VITO NV
Composietbeeld in Google
La Palisse.KMZ
16/10/2013 51 © 2010, VITO NV
Ruwe beelden Brasschaat (2004) • 25 cm grondresolutie
• 86 Megapixel Camera (Ultracam D)
Beelden opgenomen door Aerodata
http://www.aerodata-surveys.com/
Kartering: GRB-
bijhouding [Grootschalig
Referentie Bestand Vlaanderen]
B) Automatische detectie van veranderingen
16/10/2013 52 © 2010, VITO NV
Vergelijking 2003-2004 op basis van UltracamD beelden.
16/10/2013 53 © 2010, VITO NV
Live demonstratie voor de vertegenwoordigers
van het nationaal crisiscentrum (Rampenplan)
» Fiets met fijn stofmeters
» UAV helikopter met temperatuur-, druk- en vochtigheidsmeters
» Stationaire meters (temperatuur, beweging en licht) in een gebouw
=> Interface met bv. Google voor de visualisatie van de data
16/10/2013 54 © 2010, VITO NV
Aerosols up to Ultra Fine
Particles,
video camera, gps, P-trak,
noise measurements and
PID-monitor
AëroFlex II: Biclycle equipped with an “utra fine particle”
detector
16/10/2013
10
16/10/2013 55 © 2010, VITO NV
Complexe combinatie van verschillende
databronnen
» Combinatie van instrumenten:
» Statisch/mobiel,
» Op de grond/in de lucht,
» Alles in real time
» Geïntegreerde aanpak
» Datainwinning
» Datatransmissie
» Dataverwerking
» Datadistrubutie voor visualisatie
16/10/2013 56 © 2010, VITO NV
16/10/2013 57 © 2010, VITO NV
Temperatuurmetingen door de RPAS
16/10/2013 58 © 2010, VITO NV
50 m
75 m
100 m
16/10/2013 59 © 2010, VITO NV
Traject van de fiets
Trajecyt Fiets VITO.KMZ
16/10/2013 60 © 2010, VITO NV
Realtime and historical data (Fixed station with multiple sensors)
Statische sensors
16/10/2013
11
16/10/2013 61 © 2010, VITO NV
Structure of the presentation
» Remotely Piloted Aircraft System (RPAS)
» Payload
» Imaging sensors
» Non imaging sensors
» From Data to Information
» Legal aspects
» Trends
16/10/2013 62 © 2010, VITO NV
Not yet harmonized RPAS legislation in EU
HOWEVER
» RPAS is an aircraft system => there are aviation rules !!!
» Legislation depend on RPAS category: above or below 150 Kg
» Need an authorization from the national CAA (DGLV),
» => everywhere out of RC model areas
» Depending on the airspace and altitude : ATC = Belgocontrol
» For military zones : ATC = COMOPSAIR in Semmerzake
16/10/2013 63 © 2010, VITO NV
Airspace structure is complex
» Different classes (G to A)
» Classes are function of altitude but not same everywhere
» Airspace classes or control can change depending on time
» Controlled versus non controlled, Segregated vs not segregated
» Temporary reserved airspace (TRA), temporary segregated airspace (TSA)
» In Belgium > 1,000,000 overflights /year
» In controlled airspace : NOTAMs but … not always read by gliders,…
16/10/2013 64 © 2010, VITO NV
Current situation in Belgium:
temporary Permit to Fly (PtF)
» Request PtF :
» technical dossier : description of the system
» safety case : place, what if (7 questions), pilot training, …
» CONOPS : how the fligth(s) will be performed
» Authorisation (maire, frequency use,…) + insurance
» Not for commercial purpose : only for tests, demos and research
» Difference VLOS, BVLOS, BLOS (radio)
» Takes several weeks, NOTAM is issued
» Permit only valid for the specified system, pilot, place (3D), period
16/10/2013 65 © 2010, VITO NV
Numerous civilian flights with PtF
have already been performed
Information from answer of Mr Schouppe to Parliamentary questions
from Mister Van den Bergh in the Flemish parliament (08/11/2011) :
Number of test and research flights performed with PtF in Belgium:
» 2007 : 1
» 2008 : 6
» 2009 : 31
» 2010 : 179 (1 up to FL240)
» 01-06/2011 : 353 (1 up to FL240)
Since then several hundreds flights more
16/10/2013 66 © 2010, VITO NV
Belgian legislation on RPAS below
150 kg expected by begin 2014
» Pragmatic approach (base on risk, no RPAS full certification)
» Probably no more difference VLOS - BVLOS as it is subjective
» Probably not related to MTOW of the RPAS (below the 150 kg !)
» Probably depending on :
» controlled versus non controlled airspace,
» flying altitude,
» distance between platform and Pilot In Control (PIC)
» Will probably require different pilot certification levels
» If flights in altitude or on longer range than probably required:
» => Transponder Mode-S
» => Navigation lights
» => Pilot Licence (PPL ? + IFR rating ?)
» => Communication ATC
16/10/2013
12
16/10/2013 67 © 2010, VITO NV
Be-UAS makes the link between RPAS
developers/users and authorities (CAA, ATC,…)
» Inform developers and users about legislation
» Act as representative of industry, research centers, universities,…
when discussing with national/international authorities, EC, …
» Provide support to authorities when developing legislation
» Membership have to be accepted (more than 27 members)
» Members have to follow the legislation when operating
» More info on http://www.beuas.be
16/10/2013 68 © 2010, VITO NV
Structure of the presentation
» Remotely Piloted Aircraft System (RPAS)
» Payload
» Imaging sensors
» Non imaging sensors
» Other instruments
» From Data to Information
» Legal aspects
» Trends
16/10/2013 69 © 2010, VITO NV
Some trends are
» Number of users
» Legislation is adapting and getting uniform at EU level
» Development of smart sensors
» Miniaturization and cost lowering
» Development of sensor networks/constellations
» Higher resolution => more data
» Data fusion (all type) will increase
» Processing will be more and more automated
» Standardization (e.g. OGC for geographic information)
» Data processing in limited specialized centres -> communication is very important
» RPAS will play a major role during the next decades for RS
16/10/2013 70 © 2010, VITO NV
Other trends
» Miniaturization (weight, power) for everybody (not only specialists)
» Use of COTS systems
» Multiple sensors payloads
» On board processing (e.g. contour flood) to avoid possible limitation
imposed by datalink
» Collaborative systems : RPAS-RPAS, RPAS - ground or RPAS-
(sub)marine systems
» Dropable low costs sensors used in network
16/10/2013 71 © 2010, VITO NV
Payload to be dropped from the UAS
» Vulcanology (movement, temperature)
» Trucks movements
» Fire detection
» Smart Dust
16/10/2013 72 © 2010, VITO NV
Tracking vehicles with a UAV-delivered
sensor network
» 29 Palms Fixed/Mobile Experiment
» UC Berkeley and MLB Co
» March 12-14, Marine Corps Air/Ground Combat Center (MCAGCC),Twentynine Palms, CA
Goals
1. The goals of the experiment were to Deploy a sensor network onto a road from an unmanned aerial vehicle (UAV).
2. Establish a time-synchronized multi-hop communication network among the nodes on the
ground.
3. Detect and track vehicles passing through the network.
4. Transfer vehicle track information from the ground network to the UAV.
5. Transfer vehicle track information from the UAV to an observer at the base camp.
http://robotics.eecs.berkeley.edu/~pister/29Palms0103/
16/10/2013
13
16/10/2013 73 © 2010, VITO NV
http://robotics.eecs.berkeley.edu/~pister/29Palms0103/
16/10/2013 74 © 2010, VITO NV
Sensor network dropped for fire detection
http://doc.utwente.nl/65246/1/MergePDFs.pdf
16/10/2013 75 © 2010, VITO NV
Scientists develop sensor that can be dropped into the heart of a
volcano to warn if it is going to erupt (by Daily Mail Reporter)
» The sensors can detect a range of gases in concentrations of tens of parts per million
at temperatures of 200-300 C. Silicon carbide can still work at temperatures of
around 600 degrees.
»
» Read more: http://www.dailymail.co.uk/sciencetech/article-1313594/Scientists-
develop-sensor-dropped-heart-volcano.html#ixzz1IksNo8Ai
16/10/2013 76 © 2010, VITO NV
Another example
» "Topflight engineers based in Newcastle have hit upon a radical plan for warning of
volcanic eruptions. They intend to build a heatproof sensor unit which can be dropped
into a volcano's caldera and wirelessly transmit data to monitoring stations despite
being possibly immersed in molten rock. 'At the moment we have no way of
accurately monitoring the situation inside a volcano and in fact most data collection
actually goes on post-eruption. With an estimated 500 million people living in the
shadow of a volcano this is clearly not ideal,' explains Dr. Alton Horsfall of Newcastle
Uni's Centre for Extreme Environment Technology. 'We still have some way to go but
using silicon carbide technology we hope to develop a wireless communication
system that could accurately collect and transmit chemical data from the very depths
of a volcano.'"
» http://news.slashdot.org/story/10/09/20/176241/Designing-Wireless-Sensors-To-Be-Dropped-Into-Volcanoes
16/10/2013 77 © 2010, VITO NV
Smart Dust
16/10/2013 78 © 2010, VITO NV
Thank you for your attention
Any questions ?