Integrated Sensor Systems for UAS - apps.dtic.mil · Integrated Sensor Systems for UAS 23rd Bristol...

Integrated Sensor Systems for UAS

23rd Bristol UAV Systems Conference – April 2008


Mark C. L. Patterson* and Anthony Brescia†

*Advanced Ceramics Research (ACR) 3292 East Hemisphere Loop, Tucson AZ 85706, USA

[email protected]

†Naval Air Systems Command (NAVAIR) Acoustic Systems Division, Code 4.5.14,Bldg. 2185, Suite 1158, (NAWCAD)

22347 Cedar Point Road, Unit 6, Patuxent River, MD 20670-1161 [email protected]

ABSTRACT The miniaturization of sensors has in recent years led to the ability to provide multiple sensor operations from a single Unmanned Aircraft System (UAS) platform. Multiple UAS platforms can be synchronized to link devices from separate UAS platforms thus proving a powerful capability for data collection, while opening up interesting opportunities in the way data is retrieved and used. A range of new sensors being investigated will be discussed with reference to selected case studies that have taken place. As we move into an increasingly growing, data rich environment, data management, quality and pedigree will become of increasing importance. Operations for both defence and non-defence applications will be discussed with reference to the present capability and what is required in future systems.

BIOGRAPHY Dr. Mark Patterson is the Director of Research and Technology for Advanced Ceramics Research of Tucson, Arizona, USA where he oversees research efforts on the development of unmanned air vehicles and their associated sensors. He obtaining his PhD from the University of Cambridge, England in 1986 in the area of Materials Science and was the technical secretariat for the US DoD MIL-17 Handbook on Ceramic Composite Materials from 1998 until it's first publication in 2003. He presently manages several US government programs with the Army, Navy and the Department of Homeland Security to integrate advanced sensor and embedded intelligence onto small unmanned air vehicles (UAVs). He also coordinates efforts with National Oceanic and Atmospheric Administration (NOAA), National Science Foundation (NSF), National Aeronautics and Space Administration (NASA) and other scientific communities to further the use of UAVs for monitoring climate change, pollution and the oceans with UAV. In 2006 he was part of a landmark international experiment that successfully monitored pollution in the Indian Ocean for 5 weeks using synchronized stacked UAVs that validated many of the climate change models. Mr. Anthony Brescia, is an advanced technology program manager and a systems engineer acquisition professional at Naval Air Systems Command (NAVAIR), Patuxent River, Maryland. He holds a Masters degree in National Security from the National War College and an Engineering Science Masters from Pennsylvania State University and B.S. in Ocean Engineering from Florida Institute of Technology. He currently serves as a program manager for the Silver Fox UAS, Coyote UAS, sensor payloads and alternate airframes, anti-submarine warfare underwater systems, and liaison with Special Warfare groups, foreign defense establishments, and other Government agencies. Mr. Brescia served in the U.S. Army Corps of Engineers officer corps in Europe and US. He served as the Navy's Director of Science and Technology Resources, N911D, from 1999 to 2003. .

Report Documentation Page Form ApprovedOMB No. 0704-0188

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.

1. REPORT DATE 01 APR 2008

2. REPORT TYPE N/A

3. DATES COVERED -

4. TITLE AND SUBTITLE Integrated Sensor Systems for UAS

5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) 5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Advanced Ceramics Research (ACR) 3292 East Hemisphere Loop,Tucson AZ 85706, USA

8. PERFORMING ORGANIZATIONREPORT NUMBER

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)

11. SPONSOR/MONITOR’S REPORT NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release, distribution unlimited

13. SUPPLEMENTARY NOTES The original document contains color images.

14. ABSTRACT

15. SUBJECT TERMS

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT

UU

18. NUMBEROF PAGES

13

19a. NAME OFRESPONSIBLE PERSON

a. REPORT unclassified

b. ABSTRACT unclassified

c. THIS PAGE unclassified

Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18



GLOSSARY ACR - Advanced Ceramics Research AGL - Above Ground Level BYU - Brigham Young University CCD - Charge Couple Device CCN - Cloud Condensation Nuclei Counter CIRES - Cooperative Institute for Research in Environmental Sciences CU - University of Colorado DARPA - Defence Advanced Research Project

Agency DHS - Department of Homeland Security DoD - Department of Defence (US) EM - Electro-Magnetic EMG - Electr-Magnetic Gradiometer EO - Electro-Optical GCS - Ground Control System HSI - Hyperspectral Imaging System IED - Improvised Explosive Device iGCS - Integrated Ground Control System IR - Infra Red JAUS - Joint Architecture for Unmanned Systems JMPS - Joint Mission Planning System LAASS - Low Altitude Airborne Sensor System MAC - Maldives Autonomous UAV Campaign MAIS - Manta Airborne Imaging System MWIR - Mid Wave Infra-Red NASA - National Aeronautics and Space

Administration NATO - North Atlantic Treaty Organization NAVAIR- Naval Air Systems Command NIR - Near Infra-Red NOAA - National Oceanic and Atmospheric

Administration NSCT - Naval Special Clearance Team NSF - Nation Science Foundation ONR - Office of Naval Research ORASIS – Object Real Time Adaptive Signature

Identification System RF - Radio Frequency RHIB - Rigid Hull Inflatable Boat SAM - Spectral Angle Method SAR - Synthetic Aperture Radar SBIR - Small Business Innovative Research SFOV - Synthetic Field of View SIO - Scripps Institute of Oceanography STANAG – STANdardization AGreement UAS - Unmanned Air System UAV - Unmanned Air Vehicle UUV - Unmanned Underwater Vehicle 3D - Three Dimensional


INTRODUCTION The purpose of most UAV flights is typically to collect data usually in the form of video or images of regions of interest. Climate and resource data have provided society with an improved prediction capability from weather to resource management. Governments rely on remote sensing for treaty verification, disaster management, weather forecasting, and resource planning. Businesses require it to improve the efficiency of their operations and consumers depend on it for everyday decisions - often without knowing the source. Over the next decade and beyond, the use of remote sensing as a primary observational tool for understanding the Earth will grow rapidly as emerging user needs push demand1. Airborne data collection is a large industry in itself, ranging from satellites that continuously monitor different aspects of our planet to small, single engine piloted planes or balloons used to obtain specific information about a localized region. Unmanned air vehicles are therefore a natural platform for many of these existing sensor capabilities. The “pilot less” nature of UAVs means that they can be designed to be small, as they do not need to be designed around the physical size of the pilot. There is an exciting vision that advanced sensor technologies will allow us to view the Earth in three dimensions at nested spatial scales, blurring the boundary between remote and in-situ information. Vast networks of sensors will bring the most remote corner of the world into our daily lives and internet geospatial portals and geographic search engines will put all of this information at our fingertips1. Many existing sensors that have been developed around these larger manned platforms however, are too large to fit within the small Tier I and II class of UAVs, and there is an emerging business in the miniaturization of sensors for these new autonomous vehicles. Within the commercial world there is already a significant effort to miniaturize electronics for more convenient personal use and these sensors (primarily higher resolution video, still imagery and solid state processors and memory) are benefitting small UAVs. The personal entertainment industry has driven the miniaturization of many components to the size that can readily be hand held, and very capable, high resolution imaging devices are now commercially available at a moderate price. The biomedical industry has spawned microscope attachments capable of collecting and analysing hyperspectral data in a very portable device. Similarly, the mining and exploration industries have refined several



sensors for greater portability by individuals and these devices are now available for use from small UAVs. Additionally for many defence and scientific missions the utilization of UAVs and the data required has been better defined and this has led to focused efforts to miniaturize sensors, or suites of sensors, for specific applications. Many of these devices generate a much greater volume of data than previous lower resolution devices and this has led to a need for on-board data storage or computers that can better control and manage the sensor utility. The present desire for additional data as well as data from different sensors for Tier I and II type UAVs is dependant upon, and intimately related to; 1) Miniaturization of existing sensors 2) Power management 3) Data management and communications 4) Unique attributes afforded by autonomous

machines. To facilitate the collection and rapid processing of data in future systems it will be important for the UAS to operate with an on-board processing capability. The processor should be generic in nature and interface with a range of autopilots and sensors, and should be capable of performing predetermined tasks based on the requirements that are desired. Ideally, the on-board processor should partition tasks/commands between the auto pilot, payload and wireless link, as well as perform on-board processing (decisions based on multiple sensor data inputs, real time video stabilization, synthetic aperture radar (SARs) and hyperspectral image processing etc.,). The computer will also provide on-board data storage capability. The interfaces and formats should be standardized so that they are compliant with North Atlantic Treaty Organization (NATO) STANdardization AGreement (STANAG) 4586 / 4609, Joint Architecture for Unmanned Systems (JAUS) and Joint Mission Planning System (JMPS) and others. This paper addresses some of the issues above that have been experienced by Advanced Ceramics Research (ACR) in the preparation and execution of specific missions. It covers a range of sensors and sensor suites and highlights some strengths and weaknesses that were encountered.

Spectral Imaging – By using multiple spectral bands the reflectance of objects can be better interrogated, thereby allowing discrimination of objects otherwise difficult to detect in normal reflected light. Multi spectral and hyperspectral imaging are extremely powerful tools that have been utilized extensively for a wide range of applications. There are literally hundreds of publications on the topic every year and many proprietary, secret and open source techniques that are available by which to sort the data. Recent papers have provided quantitative measures of algal distribution and composition in the Potomac River2, inland water quality3 and vegetative species and infestations4,5. Spectral analysis has been able to view through water and discriminate between channels and mud flats otherwise not observable to the human eye in rivers and shallow waters. Several commercial companies in the United States now offer multi-spectral analysis of crops to optimize productivity through the management of water and fertilizers as well as to identify “infestations” which might be harmful to the crops. They have also been used for mineral exploration and for the tracking of water based contaminants from weeping domestic septic systems or agricultural run-off6. These cover but a few of the capabilities of this technique but highlight the specificity with which the technique can be employed. Most conventional airborne multispectral imaging techniques are performed at relatively high altitudes including satellites. Although very high quality optics are employed the ultimate resolution that’s achieved can be relatively low (10s of meters for satellites and sub meter for lower altitude planes). Also the large viewing distance often results in the need for significant atmospheric corrections to be carried out that can be time consuming and expensive. Lower altitude aircraft would provide a higher resolution capability for many of these sensors and would avoid the need for atmospheric corrections. Infra-red (IR) imagery in the midwave IR (MWIR – typically 1.3μm to 2.5μm) is also sensitive to water absorption and has been used to map agricultural regions and provide information for differentiating vegetation, soil, and water and for identifying management practices such as irrigation7, 8, 9. Airborne imagery provides useful information particularly using filtered light such as observed in Figure 1 which shows a narrow band filtered image (1.635μm to 1.645μm) clearly identifying regions of irrigation fallow and treed lands10.



Figure 1. Airborne mid-infrared image of a portion of an

agricultural research farm: 1) citrus trees, 2) dry fallow land, 3) irrigated land, and 4) water body10.

Radio Frequency (RF) Sensing – The use of radio waves reflected from objects of interest, has been used since the second world war. These technologies are currently highly developed and are used for a wide range of applications ranging from accurately tracking fast moving objects to 3-dimensional (3D) imaging reconstruction. Synthetic Aperture Radar (SAR) compares images (reflections) taken from two different positions in space thereby allowing the reconstruction of the view from two vantage points providing details of the spatial scale. Through the comparison between images taken over a longer duration – usually days, weeks or years, the observer is able to discriminate between changes that have taken place between the data capture, known as “change detection”. This technique has been used successfully to observe buried mines using synthetic aperture radar with change detection11. In the area of remote sensing for exploration and mining, RF techniques are commonly used to partially penetrate the ground to provide information as to the rock type, sediment layers, density and conductivity. The electro-magnetic (EM) gradiometer measures the gradient of the EM field and is commonly used today by companies such as Fugro to explore regions of interest for mineral resources. Many of these techniques are now being transferred onto small UAVs.

As mentioned earlier in the introduction, many sensors are commercially available and operational today, but are not capable of being flown on small Tier I and Tier II size of UAVs due to weight and size limitations. This paper summarizes some of the sensors being investigated and flown on the Silver Fox and Manta UAVs and describes some of the applications for which they are being investigated. RECENT UAV DEVELOPMENTS Infrared – Several small microbolometer IR video cameras are available on the market today. These cameras observe the “thermal” range (typically 8μm to 12μm) and are able to discriminate between differences in temperatures. They can readily discriminate between a person’s or animal’s body heat and the background temperature and are often used to detect and track living things. When combined with imagery in the visible range (0.3μm to 0.75μm) they can be used very effectively to discriminate between target’s of interest. The system however needs to be selected for the desired task and the typical lower resolution (typically 240 x 320 pixels) can provide limited information on the “targets” being observed. Figure 2 shows a comparison between a low and higher resolution IR video camera flown simultaneously (240 x 320 versus 480 x 640 respectively) and their ability to discriminate between details around the target.

Figure 2. IR video frames collected simultaneously

at 1000 feet above ground level (AGL). These images clearly identify improved detail (including

tyre tracks) at the higher resolution.

This comparison compared a FLIR camera with a 50mm lens with a camera manufactured by I2Tech to observe roughly the same field of view. Both cameras were mounted into the Silver Fox payload as shown in Figure 3.

240 x 320 480 x 640



Figure 3. Camera mounts on the B4 Silver Fox UAV for the low and higher resolution IR video

cameras. High Resolution Visible Imagery – Over the past 2 years the resolution of both video and still cameras available to the public has increased significantly. The quality of these systems for the most part is extremely high and with the exception of certain additional features that would be beneficial, such as hard drive data storage or the need to trigger remotely, these systems are well priced and very capable. When triggered through the autopilot, the meta data can be collected for each of these images, allowing georectification and the generation of high resolution mosaic images such as that shown in Figure 4. Images up to approximately 12 M pixels in resolution are commonly available at very reasonable prices.

Figure 4. High resolution (4 Mpixel) image mosaic from Greenland taken from Silver Fox showing the transition off the ice shelf (left) to the sea (right).

Hyperspectral Imagery – Recently a small number of manufacturers (Galilio Avionica, BAE Systems, Bodkin Design and Engineering, Headwall Photonics, NovaSol and Resonon) have started fabricating and testing hyperspectral sensors for UAVs12. The last 4 of these manufacturers make systems primarily in the visible and NIR (0.3μm to 1.0μm) that are small enough to be mounted onto Tier I and II type UAVs. Resonon, Bozeman, USA produce the Manta Airborne Imaging System

(MAIS) that was developed for the Manta UAV and has successfully completed many data collection acquisitions over both land and water. The system weighs 5.5 lbs has 120 or 240 spectral channels and operates between 120 to 200Hz. The system is a push broom imager with nearly a 10 degree field of view and at 1000 feet AGL gives a pixel size of approximately 16cm (6”). During training with Navy Special Clearance Team One (NSCT1) at El Centro Proving Grounds CA a number of “targets” were laid out on the ground to simulate mines. The targets were plastic or metallic painted (camouflage) objects, and unexploded ordinance simulants, some buried and some obscured from view under bushes as shown in Figure 5. Several passes over the target area were made in the Manta UAV at 1000 feet AGL and the data was collected, georectified and post processed using the commercially available algorithm spectral angle mapper (SAM) to identify approximately 70% of the targets as shown in Figure 5.

Figure 5. Georectified MAIS hyperspectral data

collected in 2006 from the Manta UAV of “simulated mines” laid out in the desert in El Centro CA. Some of the objects were buried, some camouflaged and

some were partially hidden from view (insert). Parameters for SAM were based on parameters measured from some of the easily detected targets. Of the targets that were not detected however, some of them were very difficult to observe, buried under deep layers of brush or very small (small bombs



stuck vertically into the soil, much smaller than the system resolution. Of the targets that were "reasonable" everything was identified except some of the camouflage painted objects. The ORASIS program was also run by the US Navy on the same data set and did significantly better identifying the camouflage painted objects and many of the buried and covered targets although the actual data was not made available for this paper. The initial version of the “push broom” detector required manual settings for the gain and optical focus. Under a current program this spectrometer has been further miniaturized to be inserted and operated from the Silver Fox UAV. The spectrometer weighs 2.4 lbs and the on-board computer and data storage weighs approximately 2.5 lbs. Both components have been redesigned axially to fit within the 4.8” diameter of the UAV as shown in Figure 6. The current spectrometer characteristics are improved over that of the MAIS and the system is fully autonomous and is operated through the autopilot.

Figure 6. Hyperspectral imager produced by

Resonon for the Silver Fox UAV.. Other Imagery Devices – Under a current US Army program a low light video camera is being integrated into the Silver Fox UAV for night or low light conditions (dusk or dawn). The camera is made by a US manufacturer and is based on an electron multiplier (EM) CCD. Under a current NAVAIR program an imaging device called “synthetic field of view” (SFOV) device is being mounted onto the Silver Fox UAV to provide a persistent early warning capability for facilities under threat from ballistic projectiles such as mortars and rockets. The device identifies and tracks the projectile long enough for it to determine the flight path and time of impact. A NIR hyperspectral imaging system (viewing from 1.0μm to 1.7μm uncooled and 1.0μm to 2.5μm under cooled conditions) similar to the MAIS has been

developed by Resonon and will be flown in the Manta UAV later in 2008. It is expected that the system will have both military and non-military use in the assessment and management of crops. Electro-Magnetic Gradiometer (EMG) – In 2004 the US Navy through the Office of Naval Research (ONR) and Naval Air Systems Command (NAVAIR) funded the development of an airborne EMG for the detection of command wires used in the detonation of Improvised Explosive Devices (IEDs). Originally a hand held EMG produced by Stolar Research Corporation for the mining industry, the gradiometer was modified for use from the Silver Fox UAV. The technology is based on two components; a transmitted “primary” wave that stimulates all conductors being illuminated and a gradiometer antenna that receives the reradiated “secondary” field generated from the flow of current in the conductor that was generated by the initial “primary” field. Two configurations have been developed and tested. In the first configuration the transmitting antenna was mounted onto the Silver Fox UAV while the gradiometer (receiving antenna) was towed approximately 70 feet behind the UAV as shown in Figure 7. This configuration allows the UAV to operate remotely over a large distance. In order to allow this configuration to be towed the gradiometer was constructed using an air core in order to minimize the EMG weight.

Figure 7. Configuration One - Airborne gradiometer with the Silver Fox mounted transmitter and a towed,

air core gradiometer. Insert shows transmitter.

In the second configuration the gradiometer (receiving antenna) is mounted on the Silver Fox UAV as shown in Figure 8 and the transmitter is usually operated remotely from the ground. Both of these configurations have been tested successfully at altitudes up to approximately 250 feet AGL.



Figure 8. Configuration Two – Silver Fox mounted ferrite core gradiometer used in conjunction with a

ground based or separate transmitter.

The gradiometer technology has been used to identify the position of surface command wires as well as tunnels buried deep below the surface13. The EMG detects the “stimulated” field from the “target” conductor and measures the field gradient between the two antennae (positioned at either end of the gradiometer) shown in Figure 8. The two antenna are opposed in their electrical configuration giving rise to an increase in the cumulative current as the gradiometer comes close to a conductor, but dropping close to zero when directly overhead the conductor. This gives rise to a characteristic “M” shaped response for the cumulative field as illustrated from the white trace in Figure 9, when the sensor detects a conductor. The peak to peak distance in the gradient signature is also a characteristic of the tunnel depth. The signal phase (green) and level of synchronization (red) are also measured. Under an existing program being funded by the US Navy (NAVAIR) and the US Department of Homeland Security (DHS) this technology is being evaluated for the detection of buried tunnels along the border. Signal detection is being automated through the use of detection algorithms and linked to the user interface (iGCS).

Figure 9. Typical EMG signal response when the gradiometer passes directly over an IED detonation

wire or underground tunnel.

Magnetometer – In support of the US Defence Advanced Research Project Agency (DARPA) funded Low Altitude Airborne Sensor System

(LAASS) a Quasar magnetometer was integrated into the Manta UAV and used in the detection of underground facilities. The aircraft initially produced a high background noise but following modification and shielding, the noise was reduced considerably allowing the magnetometer to be used effectively. Synthetic Aperture Radar (SAR) – In May of 2005 Brigham Young University (David Young at BYU), Utah, built a lightweight microSAR unit under contract from the University of Colorado. The antenna, RF stack and data storage weighs less than 2 lbs and is shown laid out in Figure 10.

Figure 10. Antenna, RF stack and data storage device produced by BYU, operated by CU, flown by

ACR

Data is written directly to the CompactFlash Card at a rate of 0.67 MB per second, this gives about 25 minutes of collection time for a 1GB disk. Power requirements are 1.1 to 1.5A at 18 VDC. The radar has eight range/velocity settings that can be changed manually. These range from a slow velocity of 18m/s which at a height of approximately 344m would provide a swath 1024 m wide, to a maximum velocity of 385m/s which at an altitude of 16m would provide a swath of just 9m. The radar works at a frequency between 5520MHz and 5600MHz. The SARs has been integrated onto the Silver Fox UAV and flown both in the United States and in Greenland.

CASE STUDIES There have been a number of Silver Fox and Manta UAVs missions where specific data sets have been collected in addition to the standard low resolution electro-optical (EO) and infra-red (IR) video. While the EO and IR video has been transmitted in real time to the integrated ground control station (iGCS), these additional sensors have logged data on-board and processed post flight. The following summarises data that was collected during;



1) Littoral surveillance mission with the US Navy 2) Tunnel detection demonstration funded by

NAVAIR 3) Airborne pollution data collected in the Maldives

during an expedition funded by National Science Foundation (NSF), National Aeronautics and Space Administration (NASA), National Oceanic and Atmospheric Adminstration (NOAA), Scripps Institute of Oceanography (SIO) and the G.Unger Vetlesen Foundation.

4) Greenland expedition funded by Cooperative Institute for Research in Environmental Sciences (CIRES) and NOAA

Hyperspectral Littoral Surveillance. – Between the 8th and 10th May 2006 UAVs took part in a military exercise called Howler which was staged to demonstrate the collaborative use of underwater unmanned vehicles (UUVs) and UAVs from an offshore vessel for the purpose of littoral mine clearance. Standard mine clearance procedures for Naval Special Clearance Team One (NSCT1) had in the past used the combined efforts of divers and mammals to identify and clear mines, but this new direction was steered towards engaging autonomous vehicles that could carry out this task. During the exercise the Silver Fox and Manta UAVs worked together and shared data through a common operator interface with other autonomous vehicles. During the exercise the hyperspectral imaging device mounted in the Manta UAV was launched from the top deck of the US Navy’s “Stiletto” experimental vessel and flown over predetermined regions of the near shore littoral zone. For the purpose of this exercise the levels of gain and focus of the camera were set manually but these variables are now fully autonomous and can be set through and interface with the autopilot. The data was stored on the UAV and post processed using ORASIS. Figure 11 shows a number of georectified sweeps that were carried out over the region of interest (identified in the image by the acronyms LFOS, RFOS, RF and LF) superimposed over a Google land image. The surf zone is shown on the right of the image just out of view. The water was not clear and varied from approximately 5m to 20m in depth. Following analysis, the processed data successfully identified the position of submerged objects as shown in Figure 11. These objects were approximately 6 pixels in diameter (approximately 1m with a 16cm pixel size) and resembled spherical mines floating in the water column. The processed data also allowed the sea bottom to be seen in places and the position of kelp beds to be determined. The reflection of the sun off

the water was significant and appeared to saturate the image contrast but this effect was successfully removed during processing.

Figure 10. Georectified hyperspectral sensor sweeps

over a littoral zone showing processed data that allowed identification of spherical mine-like objects

EMG Tunnel Detection. – The first successful demonstration of the UAV mounted EMG was carried out during a short flight in December 2004 under a US Navy funded Small Business Innovative Research (SBIR) program. These first successful flights led to additional support through a Phase II SBIR from NAVAIR that allowed significant improvements to the technology and detection technique to be undertaken. Seven more development spirals were flown starting in March 2005 and system improvements continued through 2005 into 2006 that enabled a tunnel detection demonstration in August 2006. During the tunnel detection demonstration the gradiometer was mounted directly onto a UAV in a similar configuration to that shown in Figure 12 and a ground-based transmitter was used to stimulate the tunnel. Several flights over the area of interest showed that the detection technology could repeatedly locate the position of a known smuggler’s tunnel between the United States and Mexico in Douglas, AZ, setting another UAV industry milestone13.

Figure 12. Gradiometer configuration on the Silver

Fox UAV used for tunnel detection.



For the purpose of the demonstration the UAV was flown along the border at a height of approximately 100 feet AGL and approximately 40 knots. The tunnel was approximately 40 feet below the surface of the road. The technology has been further improved and continues to be funded by NAVAIR and DHS with the goal of developing a fielded capability for border surveillance and tunnel detection. Maldives Autonomous UAV Campaign (MAC). – As was mentioned previously at this conference14, in March 2006 a UAV campaign was launched from the Maldives by Dr V. Ramanathan to study how human beings are polluting the atmosphere and their impact on climate, including global warming15. During this extensive campaign data was collected which better characterized the particles in pollution, clouds while reflected solar radiation was simultaneously measured. The science mission was a great success16 logging over 120 flight hours that included 55 takeoffs and 18 science missions and collected data on pollution and dust transported from S. Asia, Arabian and SW Asian deserts and their impacts on global dimming at the sea surface, the energy absorbed in the atmosphere and cloud properties. The specific suite of sensors17, 18, 19, 20 that was selected for this mission were identified and configured by the SIO near San Diego CA. Some were commercially available sensors and some were developed and configured by SIO specifically for the Manta. The MAC campaign was unique in that it required the movement of three UAVs at different altitudes to be synchronized with respect to flying over the same ground position. The target footprint was typically within 60 feet width and within a 100 feet distance in the direction of travel. The aircraft at the different altitudes contained different combinations of sensors depending on the scientific requirement. A complete list of sensors is shown in Table I. The MAC campaign was the culmination of over 12 months of discussions, designs and testing and involved a combined approach to reduce weight and pair suitable sensors for each of the 3 aircraft. For instance, the cloud condensation nuclei counter (CCN) was a fundamental sensor for providing the link between cloud microphysics and the physical and chemical properties of the aerosol. The commercial instrument weighed 10 kg (5 kg for the chamber and 5 kg for the electronics) and was redesigned into a compact, automated instrument that weighed less than 2 kg and autonomously measure Cloud Condensation Nuclei Counter (CCN) concentrations at 1 Hz at a single supersaturation

between 0.13% and 2%. This evaluation was given to all the sensors under consideration. Table I. Payload description and specifications for aerosol, cloud and radiation experiments for MAC.

Instrument Weight (kg)

Power (W)

1. Condensation Particle counter 0.87 2.3 2. Optical particle counter 0.27 5.4 3. Pyranometer 0.17 <0.2 4. Temp. & relative humidity 0.05 <0.1 5. Data acquisition system 0.15 <0.2 6. Aerosol inlet 0.21 NA 7. Digital video camera 0.1 0.5 8. Cloud Cond. nucleus counter <2.0 25.0 9. Grating spectrometer 0.3 <1.0 10. Aethalometer 0.85 ~5.0 11. Cloud droplet probe 1.42 14.0 12. Narrowband radiometer 0.29 <0.2 There were a number of reasons why the Manta UAV was selected for these experiments. It has a large payload volume (0.45 ft3, (0.013m3)) that is readily accessible and can accommodate a number of sensors. It is also a “pusher” meaning that the sensors can sample “clean” air uncontaminated with exhaust gases. Figure 13 shows the type of consideration that was given to the sensor mounting, cabling and physical placement within the available payload volume and on the airframe.

Figure 13. The Manta payload volume showing

sensor instrument installation. The insert shows the Manta exterior with the cloud droplet probe and

pyranometer mounted on top. [Note: This was not the final configuration flown]



Selection of specific sensors for each of the 3 platforms was also considered with respect to the data, altitude and mission durations for each of the vehicles. A summary of these instrumented UAVs is shown in Table II. The above cloud UAV flew at between 10,000 and 12,000 feet, the in-cloud UAV approximately 3,500 feet and was directed manually to the specific region of cloud by the on-board video camera, and the below-cloud UAV at between 1,000 and 2,000 feet. Table II. Summary of the instruments flown on each of the 3 Manta UAVs with total weights and power. UAV Payload

(kg) (W) Instrumentation

Above- 2.7 Aethalometer cloud 14 Optical particle counter

Up and down pyranometers Condensation particle counter

In- 3.7 Cloud droplet probe cloud 27 Condensation particle counter

Digital video camera Optical particle counter

Below- 4.0 Aethalometer cloud 40 Optical particle counter

Up and down pyranometers Condensation particle counter Cloud cond. nuclei counter

Greenland Expedition August 2007. – Greenland has long been identified as having an important influence on global climate and as being one of the thermometers for climate change. It has recently been suggested that the ice cap at Swiss Camp is moving towards the sea at an astounding rate of 20 inches/day21. Other measurements have suggested that the melting of the Greenland Ice Sheet alone could raise sea level by 21 feet. The large amount of freshwater changes density currents regulating the Gulf Stream, disrupting the movement of the North Atlantic waters that regulate weather in Europe. With climate change, increased surface snow and ice melt provides additional melt water to lubricate the bottom of the ice sheet and increases the ice flow velocity toward the coast according to Konrad Steffen21 who has been studying the ice cap for over thirty years. Measuring the melt pools such as the one shown in Figure 14 and being able to better calculate the volume of melt water on the ice sheet would significantly aid the ability to model the potential impacts of the melting ice sheet. Three primary sensors were evaluated in Greenland – high resolution imagery, HSI and SAR. Initial

indications were that hyperspectral imagery could be used to calculate the depth of water within the melt pools and researchers at NOAA and CIRES proposed using the hyperspectral imager on a UAV to collect preliminary data over the ice sheet during August 2007.

Figure 14. A typical ice melt pool found on the Greenland Ice Cap, that was analysed by HSI.

High resolution imagery and SARs were selected as additional sensor payloads that could provide important information about the local environment and about the sea ice, which was another important indicator for climate change. High Resolution Imagery – A commercial high resolution camera with a resolution of approximately 4 Mpixel was mounted into the Silver Fox payload and triggered through the autopilot. Selected pictures (1660 by 4260 resolution) were taken and stored on a memory card associated with the camera. These pictures were post processed following the mission and the associated autopilot telemetry used to georectify the images. The images were also built into high resolution mosaic maps as shown in Figure 15 using commercially available software products22 showing a flight transition from the coast (left) up to the ice cap (right),

Figure 15. High resolution mosaic image from

Greenland using the Silver Fox UAV, built up from a series of high resolution frames.

Hyperspectral Imagery - The objective of the Greenland mission was to gather preliminary hyperspectral data to assist in the development of algorithms that would use the imager to remotely measure the depth of supraglacial melt pools. The



depth of the melt pools is important in quantifying the melt occurring the Greenland Ice Sheet in order to advance the models that are used for predicting the melting of the massive ice sheet as well as the global implications of the observed acceleration in melting. The miniaturized hyperspectral imager (HSI) collected 52 channels in the 400-800nm range with data rates at 12 bits and 135 fps. Though initial plans were to fly the HSI in the Manta UAS, a technical issue and closing window of opportunity forced the supraglacial melt pools to be remotely measured using from a manned aircraft. The flight on August 24th 2007 collected data at 500-1000 feet AGL and at 80-85 kts. Data was logged inside the HSI system and post-processed. Preliminary analysis of data from one of the melt pools is shown in Figure 16.

Ice

Shallow water

Deep water

Ice

Shallow water

Deep water

Figure 16. Data from the miniature hyperspectral

imager showing the transition from the ice shelf (top) into one of the melt pools (deeper at the bottom).

The spectral response for the vertical profile shows differential absorption between the green and blue

light with water depth..

This data indicates a good potential for the remote sensor to measure water depth and therefore melt-pool volume. Further analyses is being conducted by scientists at the University of Colorado and processing of the several sets of hyperspectral data is as yet incomplete. Figure 16 shows a hyperspectral scan from the edge of the ice sheet out into one of the melt pools where the increase in water depth can be seen from a deepening on the blue colour (from top to bottom). The insert shows a graph of the spectral response for three individual colours (red, green and blue) for the vertical profile show in the image as a white vertical line. The graph shows that there is an change in the absorption coefficients between green and blue light with water depth, from which the actual depth will be determined.

Synthetic Aperture Radar (SARs) - Synthetic aperture radar is often used to complement optical imaging capabilities as SAR can acquire imagery in inclement weather (through cloud cover) and at night. Working jointly with Dr James Maslanik at the University of Colorado, who was instrumental in the development of the MicroSAR, the system was mounted onto an “electric” Silver Fox as shown in Figure 17 during the Greenland mission in August 2007. In preparation for the Greenland project, the Micro-SAR stack containing the RF boards and data acquisition module (PC104 A/D and single board computer with two FlashDisks) was separated into two stacks to fit inside of the Electric Silver Fox fuselage. Mounts were constructed to hold the two RF flat panel antennas at a 60 degree angle on the side of the fuselage as shown in Figure 17. During these flights an additional digital camera was also flown, positioned under the wing between the two antennas.

Figure 17. The MicroSAR mounted onto the electric

Silver Fox UAV prior to launch in Greenland. A mission was conducted in the river valley to the northwest of Kangerlassuaq International Airport.



The mission consisted of three passes in the same valley and the flight plan of each pass was altered slightly to fully cover the main features of the valley. The launch and recovery site was on a sandy riverbank with the direction of the flight heading approximately southeast towards the airport. The river bottom consisted of a flowing river and glacial till with vegetated hillsides on either side creating the river valley. The temperature was approximately 50F, with wind < 5 kts and overcast skies. Data was recorded throughout the duration of the flight on the FlashDisk card and backed-up by downloading to a laptop computer upon landing. The data is currently being evaluated by scientist at the University of Colorado who plan to use the MicroSAR in the future for sea ice monitoring and research in the Arctic. CONCLUDING REMARKS Over the past few years we have seen a significant rise in both the miniaturization and capability of small sensors. This has been driven to some extent by the personal electronics industry which is moving to make small hand held devices such as high definition cameras and data storage devices, ever more powerful, and the desire to link “gadgets” together for “designer” devices that perform several tasks, but is also getting traction from the small UAV market which are projected to grow substantially over the next few years. With more capable sensors being developed for the UAV market and far greater data collection rates being projected there will be a need for an on-board processing capability, able to interact with the autopilot and sensors, and capable of performing predetermined tasks based on the requirements that are desired. In an effort to conserve weight and space while maintaining an aerodynamic exterior more thought will be given to sensor integration into the airframe. Radiating and receiving surfaces will become conformal where possible and the overall weight, power and space budget will be better managed. When a mature UAV is integrated with a mature sensor system, the problems that are encountered assuming suitable size, weight and environmental compatibility are generally small and can be readily overcome following a structured integration and test flight procedure to provide a combined platform capable of collecting data in a reliable manner. When the sensor is still “in flux”, which could refer to continued development of it’s capabilities or

reconfiguring to fit within the weight and size restriction, the integration and operation can be far from trivial and significantly increase the risk for reliable data collection. This situation is similar if the UAV is still under development or if large modifications are being adopted in order to carry the sensor. Typically this integration stage is not considered early enough or at worst, is overlooked till the data is actually required. Present configurations discussed in this paper have covered the integration and operation of spectral, radio frequency, and aerosol sensors which have been evaluated during various UAV exercises/missions. For spectral imagery there are a number of commercially available programs that can be used to process and manipulate the data. One of the most sophisticated spectral sensors available today in a small format is the hyperspectral sensor which relies heavily on both data capture and data interpretation for which many algorithms presently exist. Multispectral imaging, where 3 or 4 spectral bands are compared, also provides significantly better discrimination than is available from simple EO or IR imagery. Radio frequency sensors such as the EMG have finally been integrated into small UAV platforms and successfully used to determine the position of IED detonation wires and underground tunnels. Although still in development, the signal processing capability has progressed quickly and it is likely that this area will become a significant player for natural resource exploration and in future electronic warfare applications. Aerosol sensors have been miniaturized significantly and several particle size analysers, and associated pollution measuring sensors which might include the capture of chemical and biological agents, are presently available. Thorough planning and flight testing of these sensors prior to their fielded use goes a long way in reducing technical problems and ensuring scientifically valuable data can be collected. Above all, it is important to provide a successful flight for the data collection no matter how small or simple that data appears to be. This is still significantly more valuable than a failed flight.



REFERENCES _ 1 W.B. Gail, “Remote sensing in the coming decade: the vision and the reality”, Journal of Applied Remote Sensing, Vol. 1, 012505 (19 January 2007). 2 K.H. Szekielda, G. O. Marmorino, S. J. Maness, T.Donato, J. H. Bowle , W. D. Miller , and W. J. Rhea, “Airborne hyperspectral imaging of cyanobacteria accumulations in the Potomac River”, Journal of Applied Remote Sensing, Vol. 1, 013544 (26 October 2007). 3 R. Sugumaran, N. Green and M. Clayton, “Monitoring intra-annual water quality variations using airborne hyperspectral remote sensing data in Iowa lakes”, Journal of Applied Remote Sensing, Vol. 1, 013533 (7 September 2007). 4 C. Yang, J. H. Everitt, “Evaluating airborne hyperspectral imagery for mapping waterhyacinth infestations”, Journal of Applied Remote Sensing, Vol. 1, 013546 (15 November 2007) 5 Y-B Cheng, E. Tom and S. L. Ustin, “Mapping an invasive species, kudzu (Pueraria montana), using hyperspectral imagery in western Georgia”, Journal of Applied Remote Sensing, Vol. 1, 013514 (23 May 2007) 6 Alfonso Blanco (US Environmental Protection Agency) Private Communications May 2006. 7 E. L. Skidmore, J. D. Dickerson, and H. Shimmelpfennig, “Evaluating surface-soil water content by measuring reflectance,” Soil Sci. Soc. Am. Proc. 39, 238-242, (1975). 8 C. J. Tucker, “Remote sensing of leaf water content in the near infrared,” Rem. Sens. Environ. 10, 23-32 (1980) 9 D. H. Lobell and G. P. Asner, “Moisture effects on soil reflectance,” Soil Sci. Soc. Am. 66, 722-727 (2002). 10 R. S. Fletcher and D.W. Bartels “Mid-infrared digital electronic camera system for assessing natural resources “Journal of Applied Remote Sensing, Vol. 1, 013542 (15 October 2007) 11 N. Nasrabadi, M. Tates, and H. Kwon, “Change detection methods for identification of mines in synthetic aperture radar imagery”, Journal of Applied Remote Sensing, Vol. 1, 013525 (7 August 2007) 12 P.Donaldson, “Good with Colours”, Unmanned Vehicles, October 2007. pp 15 – 19. 13 “Eye in the Sky uses Radio Gradiometer to detect IEDs, Underground Tunnels” http://www.dcmilitary.com/stories/013108/tester_27998.shtml

_ 14 M.C.L. Patterson and A. Brescia, “Requirements Driven UAV Development” 22nd Bristol UAV Systems Conf. Proc. April 2007 15 V. Ramanathan, “Global Dimming and its Masking Effect on Global Warming”, Paper A23D-01, Pres. American Geophysical Union Fall Meeting, San Francisco, CA., 11th – 15th Dec. 2006. 16 V.Ramanathan, M.Ramana, G.Roberts, D.Kim, C.Corrigan, C.Chung and D.Winker, “Warming trends in Asia amplified by brown cloud solar absorption” Nature, Vol. 448, 2nd August 2007. pp 575 - 578 17 G.Roberts, V.Ramanathan, C.Corrigan, M.Ramana and N.Nguyen, “Simultaneous observations of aerosols, clouds, and radiometric fluxes using light weight autonomous UAVs”, Paper A33B-1007, Pres. American Geophysical Union Fall Meeting, San Francisco, CA., 11th – 15th Dec. 2006. 18 V.Ramanathan, G.Roberts, M.Ramana, C.Corrigan and N.Nguyen, “Simultaneous Measurements of direct, semi-direct and indirect aerosol forcing with Stacked Autonomous UAVs: A New Observing Platform” Paper A21G-08. Pres. American Geophysical Union Fall Meeting, San Francisco, CA., 11th – 15th Dec. 2006. 19 C.Corrigan, V.Ramanathan, M.Ramana, D.Kim and G.Roberts., “Chasing Black Carbon using Autonomous Unmanned Aerial Vehicles”, Paper A43A-0106, Pres. American Geophysical Union Fall Meeting, San Francisco, CA., 11th – 15th Dec. 2006. 20 M.Ramana, V.Ramanathan, C.Corrigan, D.Kim, G.Roberts and N.Nguyen. “Direct measurements of albedo and solar absorption over the Northern Indian Ocean with a new observing system of Stacked Multiple UAVs”. Paper A13B-0921, Pres. American Geophysical Union Fall Meeting, San Francisco, CA., 11th – 15th Dec. 2006. 21http://www.popsci.com/popsci/science/6661e3568cc83110vgnvcm1000004eecbccdrcrd.html 22 http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html

Integrated Sensor Systems for UAS - apps.dtic.mil · Integrated Sensor Systems for UAS 23rd Bristol...

Documents

Transcript of Integrated Sensor Systems for UAS - apps.dtic.mil · Integrated Sensor Systems for UAS 23rd Bristol...