IE-SUP-00042-001 Radar Alignment and Accuracy Tool Radar... · Radar Alignment and Accuracy Tool:...

Radar Alignment and Accuracy Tool RASS-R Radar Comparator Dual

Edition: 1.1 Date: 18-Feb-09 Status: Released

Radar Alignment and Accuracy Tool: RASS-R Radar Comparator Dual 2/60

DOCUMENT DESCRIPTION

Document Title Radar Alignment and Accuracy Tool: RASS-R Radar Comparator Dual

Edition 1.1

Edition date 18-FEB-09

Author Andrey Pchelintsev (Ph.D, R&D Scientist)

Editor Dirk De bal (System engineer) and Marcel Vanuytven (CEO)

Abstract

This document describes the general principles underlying the Radar Comparator Dual, demonstrates results of various evaluation campaigns, discuss the main outcomes of the technique for the accurate analysis of modern radar hardware. A special attention has been paid to using of ADS-B data for radar analysis, evaluation and continuous monitoring.

Keywords Multi-radar alignment accuracy Figure of Merit ADS-B radome 3D radar OBA lookup table

Contact person Andrey Pchelintsev Tel +32 14 231811

E-Mail address [email protected]

DOCUMENT STATUS

STATUS Working Draft � Draft � Proposed Issue � Released Issue �

ELECTRONIC BACKUP

INTERNAL REFERENCE NAME : IE-SUP-00042-001 Radar Alignment and Accuracy Tool

HOST SYSTEM SOFTWARE(S) Windows XP Pro Word 2003


DOCUMENT CHANGE RECORD The following table records the complete history of the successive editions of the present document.

EDITION DATE REASON FOR CHANGE SECTIONS

PAGES AFFECTED

APPROVED BY

1.0 24-NOV-08 New document All AP

1.1 18-FEB-09 New Layout All EV


TABLE OF CONTENTS

1. EXECUTIVE SUMMARY ...............................................................................................................9

2. INTRODUCTION..........................................................................................................................10

3. RADAR COMPARATOR NEW APPROACH ..............................................................................11 3.1 BASIC PRINCIPLES .................................................................................................................11 3.2 USED METHODS.....................................................................................................................12 3.3 SYSTEMATIC ERRORS MEASUREMENT ....................................................................................12

3.3.1 Two Radars – Absolute Measurement ............................................................................13 3.3.2 Two Radars – Relative Measurement .............................................................................13 3.3.3 Accuracy of the Measurement.........................................................................................13 3.3.4 Radar vs. ADS-B Analysis...............................................................................................14 3.3.5 Radar vs. ADS-B Analysis: Data Handling......................................................................15

3.4 TRAJECTORY RECONSTRUCTION AND ACCURACY MEASUREMENT ............................................19 4. RESULTS AND APPLICATIONS................................................................................................23

4.1 CASE A: REAL VALUE FOR RADAR EVALUATION AND CONTINUOUS PERFORMANCE MONITORING 23 4.2 BAROMETRIC HEIGHT ERROR AND ADS-B ..............................................................................26 4.3 CASE B: RADAR RADOME INFLUENCE EVALUATION AND MONOPULSE DISTORTION CORRECTION 29 4.4 CASE C: MEASUREMENT OF THE AZIMUTH ERRORS GENERATED BY LIGHTENING POLES...........33 4.5 CASE D: MEASUREMENT OF THE BEAM WIDENING OF AN LVA ANTENNA...................................34 4.6 CASE E: 3D RADAR ELEVATION OBA LOOKUP TABLES CORRECTION.......................................37 4.7 AC ADS-B QUALITY MONITORING ..........................................................................................39 4.8 RADAR BIAS MEASUREMENT INDEPENDENT OF AC TRANSPONDERS’ DELAYS ............................42

5. ANNEX 1: RADAR COMPARATOR DUAL USED METHODS ..................................................43 5.1 DATA CORRECTION METHODS ................................................................................................43

5.1.1 Timestamp Analysis ........................................................................................................43 5.1.2 ACP Eccentricity Correction ............................................................................................43 5.1.3 Barometric Height Correction using Atmospheric Soundings .........................................43 5.1.4 Atmospheric Refraction Error and Correction .................................................................44

5.2 GENERAL RADAR DATA PROCESSING METHODS......................................................................47 5.2.1 XYZ-t Filtering Method ....................................................................................................47 5.2.2 Linear Interpolation..........................................................................................................47 5.2.3 Correlation of the Data ....................................................................................................48 5.2.4 Height Reconstruction .....................................................................................................48

5.3 RADAR VS. RADAR ANALYSIS..................................................................................................48 5.3.1 Systematic Errors Measurement .....................................................................................48 5.3.2 Matrix Structure ...............................................................................................................49 5.3.3 Relative Measurement ....................................................................................................52 5.3.4 Accuracy of the Measurement.........................................................................................53

5.4 RADAR VS. ADS-B ANALYSIS .................................................................................................53 5.4.1 Matrix Structure ...............................................................................................................53 5.4.2 Measurement Figure of Merit (MFM)...............................................................................54 5.4.3 Dual Comparison: Accuracy of the Measurement of the Systematic Errors ...................55

5.5 TRAJECTORY RECONSTRUCTION AND RANDOM ERRORS..........................................................55 6. REFERENCES.............................................................................................................................60


TABLE OF FIGURES

Figure 1: Results of AC latency measurement for four radars (MODE-S rad1, rad2 and MSSR rad3, rad4) compared in 4 separate measurement campaigns to the same ADS-B data set All the demonstrated results represent the required corrections and not the errors........................................16

Figure 2: MODE-S radars compared to the same ADS-B data set. Comparison between the inferred latencies and transponder delays. The discrepancies correspond to the low MFM numbers..............17

Figure 3: Examples of the trajectories having different MFM: a) 9, b) 4, c) 0.......................................18

Figure 4: 400D8C measured latency vs. MFM number (manipulated by using a subset of the data, series 1 through 3 correspond respectively to shortening of the trajectory from the left, right, and both left and right) .........................................................................................................................................18

Figure 5: Along track ADS-B errors for 3 AC (S-addresses 3A1CD5, 3C6464, 400966) calculated comparing ADS-B with rad1 (white line) and then with rad2 (red line). ................................................20

Figure 6: Examples of the trajectory reconstruction: a), c) speed variation, b), d) heading variation (red and green lines are the raw measurement of the radars, black line is the reconstructed trajectory)..............................................................................................................................................................21

Figure 7: XY display: example of the trajectory reconstruction.............................................................22

Figure 8: Results of a 5-day continuous measurement including 2 MSSR, 3 MODES and ADS-B: azimuth bias dynamics for radar MODES#3 measured separately vs. different sources. ...................24

Figure 9: Results of a 5-day continuous measurement including 2 MSSR radars, 3 MODES radars and ADS-B: range bias dynamics for radar MODES#3 measured separately vs. different sources....25

Figure 10: Distribution of the barometric error for [5000, 5500] m measured 12:00pm to 12:00am on 24-01-2008............................................................................................................................................26





Figure 15: Picture of radar N.................................................................................................................29

Figure 16: Azimuth error vs. azimuth for radar N protected with a radome..........................................30

Figure 17: Gyro measurement for radar N............................................................................................30

Figure 18: Azimuth error vs. azimuth for radar N protected with a radome after ACP eccentricity correction...............................................................................................................................................31

Figure 19: Gyro measurement for radar N showing ACP glitch............................................................31

Figure 20: Azimuth error vs. azimuth and elevation angle for radar N protected with a radome (for the same azimuth band see the highlighted area the azimuth error depends on elevation). .....................32


Figure 21: Picture of radar K with the lightening poles located strictly at 0� and 180� azimuth (N and S)...........................................................................................................................................................33

Figure 22: Average azimuth error vs. azimuth for radar K (the azimuth distortion due to the two lightening poles located at 0degr. and 180degr. azimuth, are clearly detectable with the distortion areas of approximately 60degr.) ...........................................................................................................34

Figure 23: The azimuth error vs. the elevation angle measured for MSSR radar; (black dots: measurements, red line: azimuth bias). ADS-B vs. radar measurement..............................................35

Figure 24: The azimuth error vs. the elevation angle measured for MODE-S radar; (black dots: measurements, red line: azimuth bias). ADS-B vs. radar measurement..............................................36

Figure 25: The azimuth error vs. the elevation angle measured for MODE-S radar. Radar vs. radar measurement ........................................................................................................................................37

Figure 26: ADS-B elevation angle measurement noise vs. range........................................................38

Figure 27: Latency histogram acquired during 5 days continuous measurement using ADS-B vs. 1 radar comparison technique involving 6 radars ....................................................................................39

Figure 28: Examples of small latency AC measured for 4 aircraft using ADS-B vs. 1 radar measurement technique during 5 days continuous measurement involving 6 radars..........................40

Figure 29: Examples of significant latency AC measured for 4 aircraft using ADS-B vs. 1 radar measurement technique during 5 days continuous measurement involving 6 radars..........................41

Figure 30: Schematic setup of the absolute range bias measurement using comparison of ADS-B and 3 radars .................................................................................................................................................42

Figure 31: Barometric height measurement and correction method (6447_0 is being the number of the sounding station)...................................................................................................................................44

Figure 32: Calculation of the refraction .................................................................................................45

Figure 33: Family of the refraction correction curves computed according to the polynomial approximation (14) vs. the exact integral formula based on the computation of (11)-(12) ...................46

Figure 34: Two-plot setup for the minimization procedure....................................................................51

Figure 35:Principle of the trajectory reconstruction (search for the maximum likelihood function given the radar measurements and expected radar accuracy) ......................................................................58

TABLE OF TABLES

Table 1: Advantages of using ADS-B data for radar evaluation ...........................................................14 Table 2: Disadvantages and possible solutions in case of using ADS-B data for radar evaluation .....15


GLOSSARY OF TERMS ACP Azimuth Change Pulse ADS-B Automatic Dependent Surveillance, Broadcast Annex 10 Aeronautical Telecommunication, Annex 10 to the Convention on

International Civil Aviation, the principle international document defining SSR

ARP Azimuth Reference Pulse ATC Air Traffic Control COTS Commercial Off The Shelf CPU Computer Processing Unit CW Continuous wave dB Decibel Downlink The signal path from aircraft to ground FL Flight Level, unit of altitude (expressed in 100’s of feet) FRUIT False Replies Unsynchronized In Time, unwanted SSR replies

received by an interrogator which have been triggered by other interrogators

GPS Global Positioning System ICAO International Civil Aviation Organization ICD Interface Control Document IE Intersoft Electronics IF Intermediate Frequency I/O Input/Output IP Internet Protocol LAN Local Area Network LVA Large Vertical Aperture (antenna) Monopulse Radar-receiving processing technique used to provide a precise

bearing measurement MSSR Monopulse Secondary Surveillance Radar MTD Moving Target Detection MTI Moving Target Indicator Multipath Interference and distortion effects due to the presence of more than

one path between transmitter and receiver NM Nautical Mile, unit of distance OEM Original Equipment Manufacturer Plot extractor Signal-processing equipment which converts receiver video into

digital target reports suitable for transmission by land lines PPI Plan Position Indicator PRF Pulse Repetition Frequency PSR Primary Surveillance Radar Radar Radio Detection And Ranging Radome Radio-transparent window used to protect an antenna principally

against the effects of weather RASS-R Radar Analysis Support Systems – Real-time measurements RASS-S Radar Analysis Support Systems – Site measurements RCS Radar Cross Section RDP Radar Data Processing (system) RF Radio Frequency RTQC Real Time Quality Control RX Receiver SAC System Area Code SIC System Identification Code SLS Side Lobe Suppression, a technique to avoid eliciting transponder

replies in response to interrogations transmitted via antenna sidelobes

SLB Side Lobe Blanking SNR Signal-to-Noise ratio


Squitter Random reply by a transponder not triggered by an interrogation SSR Secondary Surveillance Radar STC Sensitivity Time Control TACAN Tactical Air Navigation TCP Transmission Control Protocol TIS-B Traffic Information Services, Broadcast Transponder Airborne unit of the SSR system, detects an interrogator’s

transmission and responds with a coded reply stating either the aircraft’s identity or its flight level

TX Transmitter Uplink Ground-to-air signal path UTC Coordinated Universal Time


1. Executive Summary This document describes the general principles underlying the Radar Comparator Dual, demonstrates the results of various evaluation techniques, discuss the main outcomes of the technique for the accurate analysis, measurement and improvement of modern radar hardware. Special attention has been paid to the evaluation techniques of radars vs. ADS-B data. A considerable part of ADS-B data represents a great value to radar evaluation, monitoring and correction. However not every ADS-B message can be used, less accurate data must be distinguished and carefully discarded from the analysis. Even the remaining “good” data subset can’t be used directly, the ADS-B being subject to the specific effects and parameters as the latency. Curiously the ±0.25µs and ±0.5µs delay deviations allowed for aircraft (AC) transponders can also be measured and handled to results in unprecedented accuracy and consistency compared with radar to radar measurements. Using the methods of multi-parameter optimization a model that can employ both radar and ADS-B data for radar analysis has been developed. The model inherited the methods used by MURATREC (SASS-C) tool for multi-radar measurement and adapted these to the case of two sources measurement (implemented in the Radar Comparator Dual). Limiting the problem to two sources has produced immediate benefit making the measurement more controllable and accurate. With the ADS-B data already available from 50-75% of modern AC the improvement in the Radar Comparator Dual monitoring capability and accuracy can be called spectacular. In this report we demonstrate the superior accuracy of the radar measurement using ADS-B data. A number of applications have been demonstrated, such as continuous monitoring of radar alignment, precision measurement of the azimuth error due to the mono-pulse distortions caused by a radome or/and lightening rods, and of the azimuth bias increase for a MODE-S radar due to the antenna beam widening, the dramatic potential of improvement of the elevation measurement accuracy by 3D radar. The side product of the radar vs. ADS-B evaluation is a valid method of the ADS-B evaluation and monitoring. Availability of more accurate measurement techniques has always been beneficial for the technological progress because it generally helps out to improve the existing technology. The discussed methods at the same time create a real possibility to propose valid correction techniques.


2. Introduction The idea of combining multi-radar data for radar alignment and accuracy measurement first appeared about 25 years ago as a result of the significant progress in radar technology and manufacturing and the public demand for the enhanced ATC safety in the conditions of continuously increasing AT density. As a response to this requirement stipulated by EUROCONTROL Agency 1982 in the public tender for a multi-radar measurement and evaluation software, SASS-C (with MURATREC 1 being the first measurement core) has been developed. MURATREC staying for MUlti-RAdar TRajectory REconstruction build a method to evaluate radars (the systematic and random errors) using multi-radar data and mathematical model allowing for a number of the systematic errors (biases) and random errors (accuracy). The tool became operational during 1987-1990. In order to improve poor quality of the trajectory reconstruction and therefore exaggerated error figures, the second version of the measurement core was developed by approximately 1994 (MURATREC 2), then TR3 (also called MURATREC 3) prototype development has been promoted for a while without significant breakthrough. Despite considerable amount of time and money spent for the development and maintenance of the tool, at present SASS-C still fail to play the role it initially was assigned to. The most upsetting is that despite all the efforts the system doesn’t meet requirements for being a valid measurement tool. After more that 20 years of development it is still a prototype demonstrating poor performance. A number of problems with SASS-C have been technical. However the general drawback is conceptual: the absolute lack of visibility of the data flow, so that it is impossible to find out, what actually causes a specific problem. Another drawback is the typically high count of the measured parameters, therefore high probability of the parameter cross-contamination, unknown accuracy of the parameter evaluation, and vague principles of the parameter selection. In other words there is no general vision what the method should be able to measure and what not because it is just impossible from the standpoint of physics. It has been assumed, that the more radars are used for the measurement, the better. However with noisy data multi-parameter optimization problem is often compromised by linear dependence of the parameters, inadequate models and/or non-Gaussian error distribution. It has been assumed as well that the data sets should be “balanced”, however no quantitative parameter that would indicate that the data set in balanced or unbalanced has ever been proposed. Unfortunately radar or ATC engineers have no control over balancing air traffic for the purpose of the measurement. “The more the better” doesn’t work out in SASS-C, for more data without rigorous selection and checking of all the possible sources of errors create more chance for cross-contamination of data and unstable results etc. “The more the better” doesn’t work for a non-“balanced” setup, for in a multi-parameter minimization problem there are dependent parameters that can’t be measured separately. Since no mean was introduced to test the dependence of the parameters, the accuracy of the measurement remains unknown, so that SASS-C can not be considered as a valid measurement tool.


3. Radar Comparator New Approach Using just the opposite approach “the less the better”, an alternative tool Radar Comparator Dual has been recently developed by Intersoft Electronics NV, which processes data originated from two sources at a time. The analysis starts with measurement of all the known errors (barometric height measurement and atmospheric refraction for example) and if required corrections can be applied, timestamp quality is routinely checked, ACP eccentricity corrected if required using RASS-S gyro measurement data. The Radar Comparator Dual greatly benefits of using RASS-S software tools to find trends in the processed data. This represents a very powerful technique to establish the true sources of radar problems. The Radar Comparator Dual was carefully redesigned to use ADS-B data for radar alignment and random errors measurement. The basic principles of the measurement are highlighted in the next section.

3.1 Basic Principles It is explicitly assumed the following:

1. Radars positions must be known to within 5m (absolute maximum positional error), so that the positional uncertainty doesn’t affect accuracy of the measurement of the systematic and random errors of the radar. The position of the radar is never used as a parameter.

2. Only suitable traffic must be used for the measurement, i.e. aircraft maintaining conservative

type of the motion. Non-conservative types of motion should be discarded for both systematic error and accuracy measurement.

3. There exist physical limitations for the alignment and accuracy measurement depending on a

particular setup or “constellation” of two radars (sources), the amount, profile and orientation of the traffic, and the individual accuracy of the radars. For example absolute measurement of alignment and errors of two radars is possible only if the radars are offset by a considerable distance (at least 20 NM); in this case systematic and random errors will be determined separately for both radars, except for time bias which is measured in a relative way. On the contrary for two co-located radars (or radar vs. ADS-B) only a relative measurement is possible, so that one must specify alignment errors for the reference radar. Random errors though can be measured in an absolute way, taking advantage of the trajectory reconstruction algorithm. Any constellation with an offset less than 20 nautical miles (NM) should be checked vs. another setup, if significant discrepancies vs. more regular setup (≥20NM) are observed, the overall trustworthiness of the absolute measurement is low. The quantitative estimator of constellation “quality” for radar measurement is currently under development.

4. For evaluation of the alignment errors using two radars 7 parameters are introduced:

1212121 ,,,,,, tθθKKRR ∆∆∆∆∆ comprising 2 range biases, 2 range gains, 2 azimuth biases and the relative time bias. Then a matrix system is built which consists of a quantity (vector) to minimize in its right hand side, and its partial derivatives against each of the 7 parameters, in its left side. A possible candidate for this quantity to minimize is the XY offset between the plots that belong to both radars, taken at the same time. Since the radar data are asynchronous, the plots must be re-sampled using a linear interpolation method to bring them to the same timestamp. The solution is found iteratively using matrix inversion techniques.

5. For the trajectory reconstruction the following method is used. The data are processed per

trajectory. For each trajectory the following quantities are minimized 2211 ,,,,, θRθRVV yx ∆∆∆∆∆∆ being the target speed noise and range and azimuth random

errors. For more accurate results maneuvering segments of the trajectories should be discarded.


6. Poor timestamp quality is a source of additional positional and measurement errors. Timestamp quality should be checked before using the data for the analysis. Timestamp might need to be corrected in order to avoid contamination of the accuracy figures by excessive timestamp jitter, jumps etc.

7. Barometric height measurement has been shown to induce large errors in cases when local

atmospheric conditions are very different from the standard (ICAO Standard Atmosphere 1964). The atmospheric balloon soundings for weather forecast routinely take place once or twice-a-day in many countries across the globe, are a good testimony of magnitude of the phenomenon. It has been shown that the height deviation can be as high as 1500m, however ±500m deviations are being more common. The deviations are strongly height dependent. For two-radar setup the height data typically affects accuracy of the predicted position of the target and radar alignment and accuracy. Radar data can be easily corrected using the balloon sounding height vs. pressure data, and it has been shown this correction is always beneficial. However this correction doesn’t take into account and therefore compensate all the differences because the pressure distribution can be non-uniform over the measurement area, and therefore represents only the first order correction.

3.2 Used Methods Prior to a multi-radar evaluation session the timestamp accuracy must be tested. Since the correct timing is as important for the accurate positional reconstruction as the range and azimuth, first timestamp data vs. azimuth is analyzed (see section 5.1.1). The data originating from the different sources must be processed in order to associate them with the same targets. An efficient and fast method to correlate data taken from two sources can be based on A-code or S-address. Sources of different nature generally represent more complexity for the correlation process, for example when analysing PSR vs. SSR (PSR vs. MODES, ADS-B vs. SSR etc.). In those cases the implemented method is to attribute the code (A or S) to the source having no code information based on the XY and time window processing and then use the above mentioned algorithm. For details please refer to section 5. In general the data taken by different sources are asynchronous. To bring the samples to the same time, the general interpolation methods are used (see section 5.2.2).

3.3 Systematic Errors Measurement The dual radar comparison is typically used in order to determine the systematic errors of the both sources (radars or ADS-B). The systematic errors for radars include range bias, range gain, azimuth bias and time bias. The systematic errors for ADS-B may include latencies and individual transponders’ delay variations. The measurement algorithm assumes that all the measured parameters are described by the Gaussian normal distribution law, with the first two statistical moments, the mean value and the standard deviation being of practical interest. Here below those two are named as biases (or systematic errors) and accuracy (or random errors) respectively. Depending on the constellation and nature of the data various minimization algorithms are used. In general 7 parameters are used for the measurement, except for radar vs. ADS-B analysis 4+2N, where N is the number of AC. However the tool contains an extra feature: a parameter locking mechanism (for example a relative measurement of a radar vs. reference is possible with only 3 to 4 parameters left) more flexibility for the analysis. The results are exportable in S4 format for further investigation using the RASS-S Inventory Tool which is extremely powerful in finding trends in the data. The tool produces the quality indicators (the average residual and speed noise), these are essential to detect significant systematic errors (erroneous positions, eccentricity etc.)


3.3.1 Two Radars – Absolute Measurement Seven parameters are introduced for two radars as the systematic errors: 2 range biases, 2 range gains, 2 azimuth biases and a relative time bias. For each pair of the plots taken at the same time t the following function is to be minimized ),,(),,,(),,,,,,( 2222111111212121 θKRXtθKRXtθθKKRRD ∆∆∆∆∆∆∆∆∆∆ −= , where D is the distance between the plots, tθθKKRR ∆∆∆∆∆ ,,,,,, 212121

are respectively range biases, gains, azimuth biases and the relative time bias of the radars,

21, XX are the coordinates of the plots. As a particular case only yx, components will be used. If the measurement noise is zero, the exact solution 0),,,,,,( 212121 =tθθKKRRD ∆∆∆∆∆ may be obtained taken sufficient number of equations to uniquely resolve all the parameters. In presence of the measurement noise the problem is solved approximately. The method typically reaches convergence within 5-15 iterations.

3.3.2 Two Radars – Relative Measurement In some cases the accuracy of the absolute measurement is low or the absolute measurement is even impossible. Consider for example two co-located radars. The corresponding parameters

212121 ,,,,, θθKKRR ∆∆∆∆ the range biases and gains, as well as the azimuth biases are linearly dependent. In this case the matrix minimization algorithm either generates errors or produces abnormal results (for example range bias of 2nm, and range gain of 2000ppm). For such configurations the measurement of relative misalignment is performed. Parameters of one of the radars are “locked” i.e. are taken as constants for example ppmKθR 293,0 111 −=== ∆∆ , and only 4 parameters left in the system tθKR ∆∆∆ ,,, 222

Often the user may want to additionally “lock” one of the remaining parameters (for example range gain

2K ). In such cases the size of the system will be further reduced. Note the range gain locking has a beneficial effect on the range measurement accuracy. Range gain can typically originate from two sources: clock inaccuracy and wrong speed of the light setting for converting time delay into the range. In modern radar systems typical clock accuracy is excellent and stable to within 1÷10ppm. The speed of light in vacuum is 299792458 km/s. In the air in the standard atmospheric conditions the speed of light is by −293ppm less on average. So that the theoretical range gain for modern radar is likely to be around −293ppm. Those two parameters (range and range gain) with noisy data might be not independent and influence each other, which is for example typical for low density air traffic. To prevent results with abnormally high range gain and significant errors on the range bias estimate, locking the range gain parameter setting to −293ppm could be recommended. However the user can easily re-compute and compare the results with the unlocked range gains (other parameters) and adopt either approach.

3.3.3 Accuracy of the Measurement Despite the fact that methods used in derivations are accurate and theoretically have accuracy of the measurement represented by a 77 × covariance matrix, 44 × for the relative measurement (see 3.3.2), in practice non-random residual errors often occur, such as for example barometric altitude measurement error, azimuth encoder eccentricity, transponders’ standard delay deviation, wrong radar positioning etc. As a result the exact evaluation of the accuracy of the measurement is difficult or nearly impossible. Performing multi-radar measurement combining many sources of data (SASS-C) as opposed to two-source comparison has the following trend: non-random errors on a number of parameters neglected by the model tend to smear out and contaminate the other parameters. Since in practice there always exist factors neglected by the model1, the covariance matrix may be untrustworthy as an estimate of the errors of the measurement. 1 As we have pointed out the barometric height errors typically influence the measurement accuracy, another parameter that is not taken into account by radar to radar evaluation is a particular distribution of the AC transponders. The transponders have standard 3µs delay ±0.5 µs for SSR replies and 128µs delay ±0.25 µs for Roll Call MODES replies. This may produce up to ±75m and ±38m errors respectively. Thus different sets of the transponders may produce different delay distributions which will influence the range bias (gain) predictions.


That is why the predictions of SASS-C are rarely stable and/or consistent. A common example is the range bias and range gain variation in time with the peak-to-peak variations on the level of 100m. Using Radar Comparator Dual real accuracy of the measurement can be evaluated as follows. A particular data source is sequentially compared with a number of independent sources (radars, ADS-B, etc.) As a result the average and the tolerance on the measurement are produced. Another advantage of comparing only two sources at a time is that in the case of a discrepancy possible causes of this are routinely found analyzing the processed results using RASS-S Inventory tool. In 100% percent of the cases the observed discrepancies lead to the specific problems of the radar. Once the origin of the problem is known it can be easily corrected and create more accurate and more controlled measurement.

3.3.4 Radar vs. ADS-B Analysis Today ADS-B data are readily available for approximately 50-75% of air traffic and the availability of this type of data is rapidly growing with its accuracy steadily improving. Quality of ADS-B measurement already provides a technically sound basis and method of radar evaluation and continuous monitoring as opposed to multi-radar evaluation SASS-C. Not all the ADS-B data can qualify for this purpose and a great deal of it must be discarded from the analysis. However the remaining part of the data represents a real value outperforming all the previously known radar evaluation methods. The ADS-B measurements used for radar evaluation have the following advantages and disadvantages listed respectively in Table 1 and Table 2. A typical approach of using the ADS-B data for radar evaluation is the following. The ADS-B data set is converted from ( )hLG ,, where G is the longitude, L is the latitude, h is the height above mean sea level (MSL), to the polar coordinate system ( )hθR ,, located at ( )000 ,, hLG which is the geodectic radar position. Then the ADS-B data are treated as if they were the virtual radar data. ADS-B source theoretically doesn’t have any bias i.e. ( )0=== θKR ∆∆ , so these parameters ( )θKR ∆∆ ,, are to be locked and the radar under test is measured vs. ADS-B in a relative way. Given the ADS-B is the absolute bias free reference, however imperfect, the systematic and random errors of the radar are evaluated in an absolute way. Properly selecting high accuracy ADS-B data for the analysis and using an adequate model provide means of the direct absolute measurement and monitoring of the radar alignment and accuracy.

Table 1: Advantages of using ADS-B data for radar evaluation Advantages Benefits position is measured globally and the measurement is theoretically bias free

a relative measurement vs. ADS-B produce the absolute measurement for Radar

the position accuracy is uniform all over the evaluation area, the data have the high update rate

uniform quality of the reference over the measurement area, the more accurate interpolation and trajectory reconstruction, superior accuracy for systematic and random error evaluation, less errors cross-contamination effects

the measurement typically contains the true height of the target

the most accurate reference for the range, the transponder delay deviations can be inferred individually for each AC

the measurement is easily available any place provide the proper antenna is used to reach the required coverage

any radar can be compared with the ADS-B reference locally


Table 2: Disadvantages and possible solutions in case of using ADS-B data for radar evaluation Disadvantages Problems Solutions limited number of GPS units may produce large errors on position

quality of ADS-B reference is corrupted

filtering based on positional mismatch between radar and ADS-B

significant average delay in transmission of the positional data: latency (moreover the latency was found to change in time), significant time jitter on delay of the transmission

data association algorithm is compromised and positional accuracy is corrupted

the latency (or average delay) can be efficiently handled by the model, the time jitter can be modeled as well by assuming the higher positional errors along the trajectory

ionospheric refraction effects might affect the positional accuracy locally

non-random spatially correlated XY positional errors

the effects are local and significantly reduced by WAAS system

3.3.5 Radar vs. ADS-B Analysis: Data Handling ADS-B messages don’t contain time of detection and are transmitted by the same AC transponder that is already intensively used for SSR and MODE-S surveillance communication. There exists a delay between the moment when the position data were actually measured by GPS receiver and their transmission. The average of this parameter is defined as the ADS-B latency. Typical latency measured for ADS-B capable AC with Figure Of Merit (FOM) 6≥ characterizing accuracy of the position during one week measurement campaign conducted in Belgian airspace in January 2008 was measured within 8.10 ÷ s as can be seen in figure 1. Some AC had latencies up to 2.5s, and on several rare occasions the instantaneous time errors up to 5-10s were encountered. These values are too large to be ignored for the purpose of accurate radar evaluation. For most of the civil targets visible for 15 min or longer the latency can be measured and compensated. Another parameter that needs to be taken into account is the transponder delay. According to ICAO Annex 10 (Ref [1]) transponders must have the following delays: SSR 3±0.5µs MODE-S 128±0.25µs, 128±0.5µs for roll calls and all calls respectively. The 3 µs delay is compensated by the radar, the ±0.25µs and ±0.5µs maximum deviations correspond to respectively ±37.5 and ±75m range errors. In fact the absolute character of the ADS-B reported position (bias free and independent on weather) provides means of differentiating the transponders according to their deviation from 3 µs. In figure 1 results of the measurement of the same set of 34 targets performed for 4 radars having common coverage with the ADS-B data set. The radars were compared to ADS-B data one by one independent of each other in separate measurement campaigns, and then the latency results for the matching aircraft have been compared. We can state a very good agreement between the latency values measured by different sources.


-2

-1.5

-1

-0.5

0

0.5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

AC ID

Latency, s

rad3

rad1

rad4

rad2

average

Figure 1: Results of AC latency measurement for four radars (MODE-S rad1, rad2 and MSSR

rad3, rad4) compared in 4 separate measurement campaigns to the same ADS-B data set All the demonstrated results represent the required corrections and not the errors.

Technically instead of using a system containing 4 parameters tθKR ∆∆∆ ,,, 222

we have used ×+ 24 count of AC, adding 2 extra parameters per each aircraft in particular the latency and AC

transponder delay deviation (further named as the transponder delay). Since the radar range bias and the transponders delays are dependent quantities one would need to lock the range bias as well. For a similar reason (i.e. given the radar time bias and the ADS-B latency are dependent quantities) the time bias must be locked as well. In the result we would obtain the following system having the following variables as the parameters: ( )NiRtθK ii ,0,,, 22 =∆∆∆ , where

ii Rt ∆∆ , stand respectively for ADS-B latency and transponder delay. Optionally the range gain parameter can be setup to −293ppm (accounting for difference of the speed of light in the air) and locked, leaving

N21 + parameters ( )NiRtθ ii ,0,,2 =∆∆∆ . The matrix solution is carried out in the standard way, please refer to section 5.4.1 for details. The discrepancies observed for a number of AC (AC ID # 2, 8, 19, 20, 23) can be explained as follows. In fact the two parameters

ii Rt ∆∆ , can be dependent and influence each other to some extent, which depends mostly on the orientation of the trajectory vs. the radar. For example a radial trajectory containing only inbound or only outbound section is a typical case where the ADS-B latency and the transponder delay can’t be determined separately. When a trajectory has both inbound and outbound section the separate calculation of these parameters becomes possible. Since the same trajectories have different orientation vs. different radars their latencies and transponder delays determined with the different accuracy. In order to build an estimator which allows to figuring out how a particular trajectory is suited for the separate measurement the Measurement Figure of Merit (MFM) is produced. For the details of the definition of this parameter please refer to section 5.4.2. In figure 2 another example of performance of the developed algorithm for the measurement of the latency and transponder delay for 62 AC is given. The ADS-B data have been compared to two MODE-S radars in two independent measurements. The resulting values for AC latency and transponder delay measured for 62 AC have been compared. Overall agreement of the measured quantities is excellent demonstrating some discrepancies. The observed discrepancies are typically present on both transponder delay and latency graphs (see AC ID #48, 49). The discrepancies


correspond to AC with low MFM parameter. The MFM parameter was defined as a number between 0 and 10 with larger values corresponding to the more accurate measurement. Examples of the trajectories with different MFM are given in figure 3. Taking a subset of a trajectory can effectively change the MFM. For example if we take figure 3 a) MFM=9 and select only inbound or outbound portion of the trajectory the MFM will drop to 0. In order to study how much error on the inferred parameter we can have with different MFM, we have used one particular aircraft 400D8C and manipulated its length in order to investigate how the MFM number will change, and what MFM value would be still usable for the practical purpose. As can be seen in figure 4 MFM numbers ≥5 from practical standpoint produce nearly the same result as MFM=9. MFM equal or less than 2 may have 100 % error in predicting the AC latency value. Note all the error figures cited in this paper have meaning of the corrections and not the errors.

-1.8

-1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61

AC ID

latency, s

rad1

rad2

-250

-200

-150

-100

-50

0

50

100

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61

AC ID

xponder delay, m

rad1

rad2

Figure 2: MODE-S radars compared to the same ADS-B data set. Comparison between the

inferred latencies and transponder delays. The discrepancies correspond to the low MFM numbers

a) b)


c)

Figure 3: Examples of the trajectories having different MFM: a) 9, b) 4, c) 0

0

1

2

3

4

5

6

7

8

9

10

-1.5-1.3-1.1-0.9-0.7-0.5-0.3-0.10.1

latency, s

MFM

Series1

Series2

Series3

Figure 4: 400D8C measured latency vs. MFM number (manipulated by using a subset of the data, series 1 through 3 correspond respectively to shortening of the trajectory from the left, right, and

both left and right) There are two possibilities about how to handle the trajectories having the low MFM values. In high density ATC areas the trajectories with MFM<5 can be filtered out and discarded from the analysis. More practical solution should be setting the transponder delays parameters for such trajectories to zero (optionally to the known range bias for the radar). The above method limits the latency prediction error for these AC. The agreement in the measurement of latency and transponder delay by using different sources is discussed here in order to provide evidence that the developed method accurately measures the actual physical quantities.


We need to mention that all the predicted latency values contain a bias which is the UTC time bias of the radar. Similarly the predicted transponder delay values are contaminated with the radar range bias. The range bias of the radar can be measured taking an average over a long data set using targets with MFM≥5. Once the range bias as the average transponder delay is measured the individual transponder delays can be measured. To check the radar timestamp quality, the best direct method is to compare it vs. the GPS timestamp as a reference. Since the transponder delay characteristic is different for SSR and MODE-S messages it is logical to expect the inferred transponder delays will only match for the radars of the same type, i.e. the transponder delays inferred comparing ADS-B data with a MODE-S radar will not match the quantities determined when comparing the same ADS-B set with a SSR radar. However the latency will (as we have demonstrated in figure 2), except for the cases when radar has serious timestamp problems.

3.4 Trajectory Reconstruction and Accuracy Measurement The main prerequisite for the trajectory reconstruction is absence of the systematic errors in the data. This is why trajectory reconstruction must be performed after measurement and compensation of all the systematic errors. When all the residual errors are described by zero-bias Gaussian distribution law the following minimization algorithm can take place. For each track (trajectory) the following quantities are minimized: the speed noise vector ( )yx VV , , azimuth and range errors for both sources ( )11,θR and ( )22,θR , in the case when ADS-B source is used the random ( )YX, errors are used instead of the range and azimuth errors. Since the above random errors might be substantially different in size, the proper statistical weighting is required. The solution is found iteratively for each trajectory. Due to the above mentioned timestamp inaccuracy of ADS-B the positional errors along the trajectory for a given aircraft appear to be much larger than those across the trajectory. Several examples of the along trajectory errors are given in figure 5. The larger errors should be filtered out for the analysis. The similarity of the error patterns established when comparing the ADS-B data set to different radars confirms once again the main idea that ADS-B data is not only usable for radar analysis but represent a great value. A typical result of the trajectory reconstruction can be expressed in using the speed and heading noise figures. These parameters can be defined as follows ( ) 1,0,1 −=−=

+NiVVstdevVσ ii

( ) 1,0,1 −=−=

+Niηηstdevση ii

When comparing the ADS-B data vs. radar the typical values for Vσ and ση are about 1÷1.5m/s and 0.2÷0.3° respectively.


Figure 5: Along track ADS-B errors for 3 AC (S-addresses 3A1CD5, 3C6464, 400966) calculated

comparing ADS-B with rad1 (white line) and then with rad2 (red line). As a typical result of the proposed method, the efficient smoothing and therefore low speed and heading noise is obtained, as given in figure 6. Another typical result is a neat geometrical shape of the reconstructed trajectories without arbitrary assumptions of a particular mode of flight MOF (see figure 7).


a) b)

c) d)

Figure 6: Examples of the trajectory reconstruction: a), c) speed variation, b), d) heading variation (red and green lines are the raw measurement of the radars, black line is the reconstructed

trajectory)


Figure 7: XY display: example of the trajectory reconstruction


4. Results and Applications

4.1 Case A: Real Value for Radar Evaluation and Continuous Performance Monitoring

Using the developed model, a setup including five radars (two MSSR and three MODE-S radars) and ADS-B has been continuously monitored for 5 days in January 2008. Using the Radar Comparator Dual, the different sources of the data have been compared between each other. Thus a given selected radar was separately compared to other radar sources and ADS-B. The purpose of the comparison was to show that ADS-B represents a valid source for radar evaluation providing similar or better accuracy. The results of the azimuth bias and range bias monitoring for the selected radar (MODES#3) are given in figure 8 and figure 9. All the sources produce very similar results on the azimuth bias, and very similar patterns on the range bias results except having significant DC offsets. The value of the azimuth bias monitored in time using ADS-B data was on average the closest reading to the average azimuth value computed using all the sources. The midnight sections have been discarded from the analysis because of low amount of air traffic present, and eventually the morning hours typically demonstrate more variability due to the same reason. The patterns of the range bias measurement look very similar for all the sources (see figure 9) however the range bias measured using ADS-B data has the lowest average compared to all the other sources, and for most cases is within 5-10m for modern radars. This is not surprising at all, because the range measurement is based on the time measurement and converting this into range using known speed of propagation in various media. Speed of light is very well measured and documented for all the common media and RF grade materials, so on contrary it must look very suspicious that most radars evaluated using the multi-radar software tools should have biases of 100m and higher, which is a typical result for SASS-C for more that a decade. Comparing radar vs. ADS-B data produces typically stable measurement result that: 1st has the lowest estimation for the range bias, and 2nd is stable for hours, days and weeks with typical variation of about 3-5m standard deviation. These results are very promising for continuous 24-hour radar monitoring.


0.04

0.05

0.06

0.07

0.08

0.09

0.1

0.11

0.12

0.13

0.14

1/24/2008 4:48

1/24/2008 7:12

1/24/2008 9:36

1/24/2008 12:00

1/24/2008 14:24

1/24/2008 16:48

1/24/2008 19:12

1/24/2008 21:36

1/25/2008 0:00

1/25/2008 2:24

1/25/2008 4:48

1/25/2008 7:12

1/25/2008 9:36

1/25/2008 12:00

1/25/2008 14:24

1/25/2008 16:48

1/25/2008 19:12

1/25/2008 21:36

1/26/2008 0:00

1/26/2008 2:24

1/26/2008 4:48

1/26/2008 7:12

1/26/2008 9:36

1/26/2008 12:00

1/26/2008 14:24

1/26/2008 16:48

1/26/2008 19:12

1/26/2008 21:36

1/27/2008 0:00

1/27/2008 2:24

1/27/2008 4:48

1/27/2008 7:12

1/27/2008 9:36

1/27/2008 12:00

1/27/2008 14:24

1/27/2008 16:48

1/27/2008 19:12

1/27/2008 21:36

1/28/2008 0:00

1/28/2008 2:24

1/28/2008 4:48

1/28/2008 7:12

1/28/2008 9:36

1/28/2008 12:00

1/28/2008 14:24

1/28/2008 16:48

1/28/2008 19:12

date&timedate&timedate&timedate&time

bias, deg

bias, deg

bias, deg

bias, deg

adsb

mssr1

modes1

modes2

mssr2

average

Figure 8: Results of a 5-day continuous measurement including 2 MSSR, 3 MODES and ADS-B: azimuth bias dynamics for radar MODES#3 measured

separately vs. different sources.


-100

-90

-80

-70

-60

-50

-40

-30

-20

-10

0

10

20

1/24/2008 4:48

1/24/2008 7:12

1/24/2008 9:36

1/24/2008 12:00

1/24/2008 14:24

1/24/2008 16:48

1/24/2008 19:12

1/24/2008 21:36

1/25/2008 0:00

1/25/2008 2:24

1/25/2008 4:48

1/25/2008 7:12

1/25/2008 9:36

1/25/2008 12:00

1/25/2008 14:24

1/25/2008 16:48

1/25/2008 19:12

1/25/2008 21:36

1/26/2008 0:00

1/26/2008 2:24

1/26/2008 4:48

1/26/2008 7:12

1/26/2008 9:36

1/26/2008 12:00

1/26/2008 14:24

1/26/2008 16:48

1/26/2008 19:12

1/26/2008 21:36

1/27/2008 0:00

1/27/2008 2:24

1/27/2008 4:48

1/27/2008 7:12

1/27/2008 9:36

1/27/2008 12:00

1/27/2008 14:24

1/27/2008 16:48

1/27/2008 19:12

1/27/2008 21:36

1/28/2008 0:00

1/28/2008 2:24

1/28/2008 4:48

1/28/2008 7:12

1/28/2008 9:36

1/28/2008 12:00

1/28/2008 14:24

1/28/2008 16:48

1/28/2008 19:12

date&timedate&timedate&timedate&time

bias, m

bias, m

bias, m

bias, m

adsb

mssr1

modes1

modes2

mssr2

Figure 9: Results of a 5-day continuous measurement including 2 MSSR radars, 3 MODES radars and ADS-B: range bias dynamics for radar MODES#3

measured separately vs. different sources.


4.2 Barometric Height Error and ADS-B The C-code info is the height obtained from the pressure measurement converted using the ICAO 1968 Standard Atmosphere curve. Actual atmospheric conditions can significantly differ from the Standard Atmosphere. This difference was proven to cause additional errors for multi-radar measurements (section 5.1.3). As opposed to C-code measured by AC, ADS-B message if configured may contain both true MSL height of the target and a C-code. This can be used to have correction lookup table map or volume as opposed to the uniform correction of the barometric height proposed earlier using data of the atmospheric soundings. In figure 10 through figure 14 there are a number of distributions for barometric height error for different height intervals and dates. The differences are considerable and in general are dependent on ( )thYX ,,, . So that the true height and C-code information available from ADS-B can be used to build 3D correction table for the barometric height measurement. This correction might be used to correct the C-code for dual radar, and multi-radar measurement tools. This must be a significant improvement in multi-radar evaluation techniques. On the other hand as it was already demonstrated above ADS-B data typically represent a way better source compared to any radar-to-radar measurement. ADS-B measurement represents an accurate external reference with the high update rate, uniform accuracy, available for up to 250nm distance and intrinsically bias-free, so the radar measurement becomes absolute. ADS-B typically contains the true height of the target above MSL which produces the most accurate external reference for the radar range reading. In the past the majority of radars users couldn’t take advantage of the multi-radar evaluation methods simply because they haven’t had access to the data from the other sources (radars). The simplicity of acquiring the ADS-B data is appealing, and any radar site equipped with proper ADS-B antenna can immediately have the quality reference data they have always dreamed of. In the simplest case the SLS channel can be used to have the ADS-B coverage up to 140-150nm. Today any radar site can benefit from the developed multi-source radar evaluation methods started within SASS-C and improved and extended to the ADS-B by Radar Comparator Dual.

Figure 10: Distribution of the barometric error for [5000, 5500] m measured 12:00pm to 12:00am on 24-

01-2008



01-2008


01-2008


4.3 Case B: Radar Radome Influence Evaluation and Monopulse Distortion Correction

Figure 15: Picture of radar N

The ADS-B provides a new powerful approach for radar measurement, monitoring, and error correction. Availability of ADS-B data and the high quality of a part of it can generate several interesting applications. Another example of how the ADS-B data can be used to enhance performance of a radar is the measurement of the mono-pulse distortion by a radome. The influence of a radome on the radar azimuth error is far from being of pure academic interest. Most often radome manufacturers will provide numbers specifying the frequency dependent RF phase distortion. Given radome is represented by the complex structures comprising fiberglass coated segments of varying thickness assembled with large and long metal bolts, the segments being of complex geometry and structure, till now the exact evaluation of the effect of a radome on the radar antenna diagram and thus azimuth measurement performance was not possible to measure. However on many occasions considerable degradation of the performance of radars protected with radomes has been observed compared with non-protected radars. ADS-B data changed this fundamentally bringing over the real possibility to accurately measure the beam distortion caused by a radome. Figure 16 illustrates the azimuth error distribution for radar N protected with a radome shown in Figure 15. The errors have been calculated in associating and comparing the radar data vs. the ADS-B data acquired at the same location using the SLS channel. The model (see section 3.3.4) computed the systematic errors, ADS-B latencies and transponder delays. The random azimuth error distribution given in Figure 16 is not completely random, and there are a number of factors discovered responsible for such a complex fingerprint. The azimuth errors range from approximately −0.2° to +0.2° resulting in 0.4° peak-to-peak (pp) band with both low (about 0.1Hz) and higher frequency (about 1Hz) contents present given the antenna revolution rate of 10s. Often the lower frequency content especially when it corresponds to the antenna revolution period (10s) is determined by the mechanical reasons, and as soon as this component was discovered the additional gyro measurement have been conducted. Two RASS-S gyro-inclinometer recordings of 20 scans each have been acquired in order to evaluate the mechanical behavior of the system. The RASS-S gyro-inclinometer can measure the rotational behavior and the inclination of the antenna. The gyro measurement has immediately detected the significant ACP encoder eccentricity of 0.15° pp as given in Figure 17. The eccentricity error has the typical sine shape, and by correcting the azimuth alignment it is only possible to correct for the azimuth DC offset. However in the Radar Comparator it is possible to correct for various error types including the eccentricity error. Using the measured eccentricity error the corresponding eccentricity correction has been used before processing the data with Radar Comparator Dual. The resulting azimuth errors of the eccentricity corrected data are shown in Figure 18. The 0.1Hz (1/10s) component was indeed compensated.


There is still the 1Hz component, (antenna revolution rate 10s) the saw tooth shaped error (0.15° pp) present and there is a less pronounced second saw tooth distribution below. The gyro measurement didn’t show any of the 1Hz azimuth error oscillation, and it must be produced by other, non-mechanical reasons. The synthetic targets injected using the RES directly into the receiver didn’t show any indication of the effect. The only element in the system between the antenna and the aircraft that can cause the observed phenomena is the radome. There is another evidence of this radome influence given in figure 20, where the azimuth error clearly depends both on azimuth and elevation. The radome panel structure and their count changes with the elevation angle which explains the observed differences in the azimuth error for the same azimuths.

Figure 16: Azimuth error vs. azimuth for radar N protected with a radome

Figure 17: Gyro measurement for radar N

The Gyro measurement of figure 17 shows 0.15° pp azimuth error, (green is the dt

θd g measured by the

gyroscope; blue is the corresponding dt

θd ACP reported by the ACP encoder, red is the difference

dtθd

dtθd

ACPg−

represents the azimuth error).


Figure 18: Azimuth error vs. azimuth for radar N protected with a radome after ACP eccentricity

correction.

Figure 19: Gyro measurement for radar N showing ACP glitch

The Gyro measurement of figure 19 shows an ACP glitch, (green is the dt

θd g measured by the gyroscope;

blue is the corresponding dt

θd ACP reported by the ACP encoder, red is the difference dt

θddt

θdACPg

−

represents the azimuth error).

The second saw tooth profile is also visible in both Figure 16 and Figure 18 which seems to be identical to the main saw tooth fingerprint, didn’t remain enigma much longer and has been shown to be caused by the occasional extra ACP (0.088°). This artefact was also visible on the gyro analysis diagram see Figure 19. This above artefact is caused by an extra ACP due to a glitch in the system, and it occurred in about one out of five revolutions. It may occur at any azimuth during rotation. This results in 4097 ACPs instead of 4096 ACPs.


Availability of more accurate measurements is always beneficial because it helps out to improve the existing technology. The outlined method of the measurement of the radome mono-pulse distortion immediately creates a real possibility to propose a valid correction method. Once the mono-pulse distortion has been measured for different elevations a 3D lookup azimuth correction table can be implemented. The potential azimuth accuracy improvement for this radar is breathtaking. If all the errors: the eccentricity, extra ACP glitch and the radome distortion are corrected the expected azimuth accuracy of this radar must be within 0.020° (standard deviation).

Figure 20: Azimuth error vs. azimuth and elevation angle for radar N protected with a radome (for the

same azimuth band see the highlighted area the azimuth error depends on elevation).


4.4 Case C: Measurement of the Azimuth Errors Generated by Lightening Poles

The next example of the application is rather close to the previous case. Many radar antennas and towers are protected from the lightening strikes by lightening poles commonly placed at the extremities of the upper platform of the radar tower where up to 4 lightening rods can be encountered. A picture of radar K equipped with 2 lightening conductors is given in figure 21.The poles positioned at 0° and 180° azimuth. Processing of the radar data and comparing them with ADS-B reveals two systematic peaks of approximately 0.04° pp in the distribution of the azimuth error vs. the azimuth (see figure 22). The angular location of the peaks is 0° and 180° azimuth agrees with the layout of the lightening rods on the satellite image. This influence however seemingly rather small compared with the required 0.07° standard deviation for the random azimuth error is not that small for the future ATC challenges with high load air traffic and decreased separation of aircraft. The 0.04° pp systematic difference at 250nm produces more than 300m position error. Compared with the previously measured azimuth error due to the radome distortion, the distortion amplitude due to the lightening pole is typically 3 times less. In fact in many cases non random errors erroneously have been considered as being random mostly due to the lack of accurate measurement methods (see section 4.3). Sometimes these were apparent errors resulting from the cross-contamination produced by other sources. As soon as an error is accurately measured and its nature is correctly understood, it can be corrected resulting in immediate performance benefits of the radar system.

Figure 21: Picture of radar K with the lightening poles located strictly at 0degr. and 180degr. azimuth (N

and S)


Figure 22: Average azimuth error vs. azimuth for radar K (the azimuth distortion due to the two lightening

poles located at 0degr. and 180degr. azimuth, are clearly detectable with the distortion areas of approximately 60degr.)

4.5 Case D: Measurement of the Beam Widening of an LVA Antenna A large vertical aperture (LVA) antenna beam widening is known to increase the azimuth error with elevation. Typically the radars use a single OBA table for the measurement. MSSR radars do not suffer much from the effect because of the multiple interrogations and the azimuth averaging. However MODE-S radars are more affected by the phenomena because they determine position typically using a single reply, and a target is interrogated typically early in the beam, when the target is at the right side of the beam. This typically increases the azimuth bias at higher elevation angles. The problem has been addressed by EUROCONTROL standard documents (see Ref [2]) where it is stated as follows: “The azimuth bias shall not increase at elevation angles more than 10° by an amount attributable to the antenna (e.g. beam widening effects -normally the inverse cosine of the elevation angle). The system azimuth bias elevation changes attributable to the antenna beam widening at large elevation angles shall be stated by the Tenderer in his proposal. This value shall be verified by tests of the antenna as part of the overall system test.” So far the measurement of the antenna beam widening and associated azimuth bias increase with elevation using the traffic of opportunity data using radar vs. radar comparison methods was much less obvious. ADS-B data allowed measuring this parameter more accurate than ever before compared with a multi-radar measurement. In figure 23 and figure 24 two examples of the azimuth error distribution vs. the elevation angle respectively for a MSSR and MODE-S radar are given. The MSSR radar doesn’t have any azimuth bias increase vs. the elevation, the MODE-S radar has demonstrated the azimuth bias increase of about 0.45° for elevation angles greater than 10°. Compare the results in figure 24 with the same results obtained with radar to radar comparison (see figure 25). The magnitude of the bias increase is much less pronounced that might be due to the lower update rate and non-uniform accuracy of the measurement, and possible cross-contamination of the errors.


Figure 23: The azimuth error vs. the elevation angle measured for MSSR radar; (black dots:

measurements, red line: azimuth bias). ADS-B vs. radar measurement


Figure 24: The azimuth error vs. the elevation angle measured for MODE-S radar; (black dots:

measurements, red line: azimuth bias). ADS-B vs. radar measurement


Figure 25: The azimuth error vs. the elevation angle measured for MODE-S radar. Radar vs. radar

measurement

4.6 Case E: 3D Radar Elevation OBA Lookup Tables Correction The last but not the least is the example of enhanced performance of radar taking advantage of ADS-B data is the target elevation correction algorithm for 3D radar. 3D radar height measurement has relatively poor accuracy especially for low elevation targets. The elevation error converted to height can easily attain a few hundred meter levels. The elevation error of 3D radar has the systematic and random components. The systematic component is determined by insufficient accuracy of the vertical mono-pulse OBA lookup tables. The error may depend on weather, anomalous propagation phenomena etc. The random component is determined by limited accuracy of the system and measurement noise. However if ADS-B data having the common coverage with the 3D radar is available the following processing can be performed. The radar data can be correlated with ADS-B data using the above discussed techniques implemented in the Radar Comparator Dual and then the true height measurement can be used to correct for the systematic component of the 3D height (or elevation) measurement. In order to investigate possible improvement the elevation measurement noise using ADS-B data has been determined. The ADS-B data has been sorted according to S-address and for each trajectory the internal true height reference has been built. Comparing the true height data of ADS-B with this reference the height noise has been measured for each trajectory. The height noise has been converted to the elevation noise, and the result of such a measurement is given in figure 26. Typically the random elevation angle error as measured using ADS-B data doesn’t exceed 0.1°, and for ranges greater than 20nm is typically within 0.05°. The availability of ADS-B makes possible dynamic correction of the vertical mono-pulse OBA tables. Based on the ADS-B 3D radar comparison a 3D lookup correction table for OBA can be built and regularly updated if required. Typical accuracy of the elevation measurement by 3D radar is about 0.3° (standard deviation), so that the benefit of such a correction is obvious.


Figure 26: ADS-B elevation angle measurement noise vs. range


4.7 AC ADS-B Quality Monitoring The side product of the radar evaluation techniques vs. ADS-B is the ADS-B squitters monitoring and evaluation. As shown above the ADS-B positional and time accuracy is consistent between two independent radars observing the same aircraft (see Sections 3.3.5 and 3.4). In fact this agreement produces an excellent basis for continuous ADS-B quality monitoring. Using the above described technique of the comparison when ADS-B was compared vs. single radar at a time, 5 days of data involving 6 radars have been processed. The results have been stored in a database for aircraft having FOM≥6. Overall 22491 records have been produced and stored in the database. Typical accuracy of the latency measurement was about 30ms (standard deviation). The histogram of the measurement is given in figure 27. The histogram has a central peak at 200ms, with some reports delayed as faraway as 2.5÷3s, there also exists a limited count of advanced reports, typically produced up to 0.5÷0.8s in advance. The latency was found to vary significantly on a significant part of AC, with the larger variability observed on the AC having latency equal to or greater than 400÷500ms. Several distinct examples of aircraft having small latency and large varying latency are given respectively in figure 28 and figure 29.

Figure 27: Latency histogram acquired during 5 days continuous measurement using ADS-B vs. 1 radar

comparison technique involving 6 radars

FL

advanced reports

delayed reports


Figure 28: Examples of small latency AC measured for 4 aircraft using ADS-B vs. 1 radar measurement

technique during 5 days continuous measurement involving 6 radars

300168

-1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

23-Jan-

08

24-Jan-

08

25-Jan-

08

26-Jan-

08

27-Jan-

08

28-Jan-

08Date

Latency, s

341498

-1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

23-Jan-

08

24-Jan-

08

25-Jan-

08

26-Jan-

08

27-Jan-

08

28-Jan-

08Date

Latency, s

342419

-1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

23-Jan-08 24-Jan-08 25-Jan-08 26-Jan-08Date

Latency, s

34241A

-1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

23-Jan-

08

24-Jan-

08

25-Jan-

08

26-Jan-

08

27-Jan-

08

28-Jan-

08

29-Jan-

08Date

Latency, s


Figure 29: Examples of significant latency AC measured for 4 aircraft using ADS-B vs. 1 radar

measurement technique during 5 days continuous measurement involving 6 radars

341582

-1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

23-Jan-08 24-Jan-08 25-Jan-08 26-Jan-08 27-Jan-08Date

Latency, s

3C5EE1

-1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

23-Jan-

08

24-Jan-

08

25-Jan-

08

26-Jan-

08

27-Jan-

08

28-Jan-

08

29-Jan-

08Date

Latency, s

34138C

-1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

23-Jan-08 24-Jan-08 25-Jan-08 26-Jan-08Date

Latency, s

3422DE

-1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

24-Jan-08 25-Jan-08 26-Jan-08 27-Jan-08 28-Jan-08Date

Latency, s

different hours


4.8 Radar Bias Measurement Independent of AC transponders’ delays Another advantage of database integration of the ADS-B evaluation results produced by comparing separately each radar to ADS-B, is a real possibility of inferring the radars’ range biases completely independent of the individual AC transponder delay collection. There must be at least three radars observing the same air traffic in order to predict the true range biases of these. Theoretically the radars might even be located in the opposite hemispheres, given the same transponders are used (see figure 30). Of course the same aircraft are not likely to fly worldwide, so that the more local comparison is more practical. When comparing ADS-B to a radar the measured transponder delays it∆ are affected by the range bias

jR∆ of the corresponding radar i.e. ji Rt ∆∆ + is actually measured. Given every radar worldwide can

be compared with ADS-B ( )ji Rt ∆∆ + for m radars and k aircraft can be measured, then when the

sufficient count of the measurements is acquired it∆ and jR∆ can be determined separately. This can be

easily shown taking 3 radars producing 1Rti ∆∆ + , 2Rti ∆∆ + , 3Rti ∆∆ + i=1,100

� 321 ,, RRR ∆∆∆ and it∆ i=1,100. However typically civil AC is equipped with two transponders generally having different delays, therefore more local comparison is required to make sure the same instantaneous latency and transponder delay is affecting radar bias measurements. This requirement can be easily fulfilled in many parts of the world having dense radar coverage, for example the Western and Central Europe, North America etc. Once the radar biases are known from such measurement the transponder delays accuracy can be refined as well. The transponders with non-standard delays can be therefore detected.

Figure 30: Schematic setup of the absolute range bias measurement using comparison of ADS-B and 3

radars

rad1+ads-b

rad2 +ads-b

ads-b DB


5. Annex 1: Radar Comparator Dual Used Methods

5.1 Data Correction Methods

5.1.1 Timestamp Analysis Before any data fusion process combining data originating from sensors of different type and/or nature the timestamp accuracy must be inspected. The correct timing is as important as position for the accurate positional reconstruction and the correct object association. First the integrity of timestamps for plot (track) data, the sector messages and the north messages are tested. Number of scans and the average time per scan is determined for all three sources. If the data are consistent sector messages timing is used to produce the best linear fit using a number of the antenna rotations (default setting). Using the obtained linear fit the timestamps of the plot data are evaluated and corrected if required. If the time of detection is available the correction of the timestamp is not required. On the older radars when only the time of recording is available, the correction may have some benefits, especially when using multi-source comparison techniques. With high load air traffic situations the time of recording may have up to 1-1.5 s error vs. the correct time of detection. However the user must be aware of the other unwanted effects this correction might cause. For example if the radar has significant eccentricity, correcting for the timestamp error might actually increase the effect of the eccentricity.

5.1.2 ACP Eccentricity Correction ACP encoder eccentricity was shown to have significant negative impact on the results of a multi-radar or dual radar evaluation. The same radar with eccentric ACP encoder may typically have 0.05-0.10° azimuth bias difference when measured to radars that happen to have the common coverage at the opposite maxima of the eccentricity sine wave. The phenomenon becomes more complex when more than one radar from the evaluation setup has this problem. Moreover the azimuth biases of the other radars compared to the radar with ACP eccentricity may appear different compared to the other pairs. The eccentricity is the systematic error so it must be corrected before the measurement.

5.1.3 Barometric Height Correction using Atmospheric Soundings The C-code info is the height obtained from the pressure measurement converted using the ICAO 1968 Standard Atmosphere curve. It is known that actual atmospheric conditions can significantly differ from the Standard Atmosphere. This difference was proven to cause additional errors for multi-radar measurements. Atmospheric balloon soundings routinely taken for the weather forecast services twice a day in many countries throughout the world are the best testimony of the phenomenon. It has been shown that the ultimate height deviation can be as high as 1500m however ±500m deviations are being more common. The data are routinely corrected for possible deviations of the local atmospheric conditions using web access to a public meteorological database. As given in figure 31 the C-code (height) is converted to the actual pressure according to ICAO 1964 Standard Atmosphere, and then the actual height is obtained using the local pressure vs. height curve provided from the atmospheric sounding measurement. Unfortunately the assumption of the uniformity of the barometric pressure distribution (both in space and time) is not correct but until only recently the distributed barometric measurement was not available. ADS-B data typically contain true height of the target above MSL and C-code. This can be used to have correction table map or volume as opposed to the uniform correction of the barometric height. Barometric soundings data bases are available from various sources. One is available from the University of Wyoming, Department of Atmospheric Science (http://weather.uwyo.edu/upperair/sounding.html). An alternative is the website of National Oceanic and Atmospheric Administration (http://www.ncdc.noaa.gov/oa/climate/igra/index.php). The formats of the data and access methods are different for both sources. The Radar Comparator tools have implemented the barometric correction using the database from the University of Wyoming.


0

200

400

600

800

1000

0 5000 10000 15000

Height, m

Pressure, hPa

6447_0

ICAO 1964

Figure 31: Barometric height measurement and correction method (6447_0 is being the number of the sounding station)

5.1.4 Atmospheric Refraction Error and Correction The gradient of the air density in the atmosphere produces the gradient of speed of light and is responsible for the atmospheric refraction. The main effect is that the radio-waves travel along slightly curved downwards trajectories. This produces an error in elevation angle and range. The effect is negligible for all but low elevation angles and long ranges. Using a stratified model for the atmosphere recurrent formulae allow evaluating the range error as follows. The refractive index of the troposphere is approximated by

−=7

exp)(h

NhN S

(1) where 313≈SN .

1

2

12 sinsin θ

n

nθ =

(2) where n is given by the following formula

( ) 6101 −

×+= hNn

(3) Iterative process based on CRPL model with number of layers 16=γ covering the height range from 0÷70 km (limiting height of the atmosphere), thickness of each layer can be determined as follows

( )2ln70000lnexp γα −=

(4)

FL true height

measured pressure


Thickness of each next layer is doubled compared to the previous one.

α1

α2 θ0

θ1

β0

β1

re+α0 re+α0+α1

re+α0+α1+α2

Earth

Figure 32: Calculation of the refraction

The schematic layout of the problem is given in figure 32. Using the Snell’s law

0

11 sinsin β

n

nβ

k

k=

+

(5) Using the law of sines

0

0

10

0 sin2sin

αr

β

ααr

πθ

ee +=

++

+

(6)

++

+

+

=10

00

0

)(2

sin

arcsinααr

αrπ

θ

βe

e

(7) Combining the previous two relations

)(

sincos

0

11

10

0

αrn

nβ

ααr

θ

ek

k

e +

=

++

+

(8)

)(

coscos

0

11

10

0

αrn

nθ

ααr

θ

ek

k

e +

=

++

+

(9)

++

+=

+

0

101

01 cos

)(

)(arccos θ

ααrn

αrnθ

ek

ek

(10)


Converting the local elevation angle θ1 to the reference system one gets

−−−= 0001

2θβ

πθθ

(11)

∑=

−−−=

k

iiik θβ

πθθ

00

2

(12) Computing the refraction angles according to (39)-(40) in a wide interval of elevation angles and ranges the correction formula can be expressed as follows

)()0167.11( 44.0 RPER −

+=∆

(13)

dR =(1+1.0167E)^-0.4(-0.182743x10-9R 4

+0.380884x10-6R 3-6.200607x10-6R 2

+0.160306x10-3R)

0

5

10

15

20

25

30

0 100 200 300 400 500

range, km

range correction, m

0

0.5

1

2

3.5

7

15

Figure 33: Family of the refraction correction curves computed according to the polynomial approximation

(14) vs. the exact integral formula based on the computation of (11)-(12) where ∆R is the range correction in meters, E is the elevation angle in degrees, R is the range in km,

dRcRbRaRRP +++=2344 )( is the polynomial of 4-th order with the following coefficients

910182743.0 −

⋅−=a

610380884.0 −

⋅=b (14)

610200607.6 −

⋅−=c 310160306.0 −

⋅=d


CRPL model is limited to the average atmospheric conditions and doesn’t take into account real pressure, humidity and temperature distributions. These parameters are able to affect the refraction of RF in the atmosphere.

5.2 General Radar Data Processing Methods

5.2.1 XYZ-t Filtering Method The filtered position of the plot ( )22' tX given measurements ( ) 3,1,' =itX ii

taken, is given by the following formula

( )( )( )

( ) 212

13

13122

3

1

3

2' Xtt

tt

XXXtX +

−

−

−+=

(15)

( )( ) ( )( )

( ) 2

13

121313122

3

1

3

2' X

tt

ttXXttXtX +

−

−−+−=

(16)

( )( ) ( )

( ) 2

13

23112322

3

1

3

2' X

tt

ttXttXtX +

−

−+−=

(17) where 321 , X, XX represent vectors. The above smoothing method is efficient for linear motion and generates significant positional errors with manoeuvring aircraft (with the error magnitude strongly depending on the data update rate and the angular velocity).

5.2.2 Linear Interpolation To re-sample the radar data to bring the plots to the same time, the following formula is used

( )( )

( )13

12

1213 tt

tt

XXXtX −

−

−+=

,

(18) where

231 ttt ≤≤ .


5.2.3 Correlation of the Data An efficient and fast method to correlate data taken from two sources is based on the A-code or S-address correlation. The data produced by each source are sorted first using their codes and then their timestamps. The data sets having the same code originated from both radars are processed by a tracking correlation algorithm. As a result a number of tracks which belong to both sources are established. Sources of different nature generally represent more complexity for the correlation process, for example analysing PSR vs. SSR (PSR vs. MODES, ADS-B vs. SSR etc.). In those cases the implemented method is to attribute the code (A or S) to the source having no code information based on the XY and time window processing and then use the above mentioned algorithm. A more general alternative is using the general principle Object Correlator (OC). Given possible misalignment errors (including timestamp errors, especially for ADS-B) that can be significant, this algorithm is subject to various difficulties, with large errors tending to deteriorate quality of the correlation result.

5.2.4 Height Reconstruction For established tracks simultaneously monitored by two sources the height reconstruction is performed. The purpose is to reconstruct the correct height of the aircraft in presence of garbled or absence of C-code info. Incorrect height information may produce significant positional errors when converting the coordinates from the local coordinate systems to the reference system. The following method is applied to correct the height errors. The combined flight data (coming from both radars) are first sorted in time filtered using the median filter of rank 2. If the difference between the raw data and filtered data is larger than the specified threshold, the raw plot height information (C-code or 3D height) is changed by the filtered one and the plot is flagged as having incorrect C-code.

5.3 Radar vs. Radar Analysis

5.3.1 Systematic Errors Measurement The dual radar comparison is typically used in order to determine the systematic errors of the both sources (radars or ADS-B). The systematic errors for radars include the range bias, range gain, azimuth bias and time bias. The systematic errors for ADS-B may include latencies and individual transponder delays variations (further transponder delays). The measurement algorithm assumes that all the measured parameters are described by the Gaussian normal distribution law. The first two moments, the mean value and the standard deviation are of practical interest. Here below those two are named as biases (or systematic errors) and accuracy (or random errors) respectively. Depending on the constellation and nature of the data various minimization algorithms are to be used.


5.3.2 Matrix Structure Seven parameters are introduced for two radars as the systematic errors

1212121 ,,,,,, tθθKKRR ∆∆∆∆∆ that include 2 range biases, 2 range gains, 2 azimuth biases and a relative time bias. For each pair of the plots taken at the same time t the following function is to be minimized

),,(),,,(),,,,,,( 2222111111212121 θKRXtθKRXtθθKKRRD ∆∆∆∆∆∆∆∆∆∆ −=

(19) where D is the distance between the plots,

1212121 ,,,,,, tθθKKRR ∆∆∆∆∆ are respectively range biases, gains, azimuth biases and the relative time bias of the radars, 21,XX are the coordinates of the plots. As a particular case only YX, components will be used. If the measurement noise is zero, the exact solution

0),,,,,,( 1212121 =tθθKKRRD ∆∆∆∆∆ may be obtained with the sufficient number of equations to uniquely resolve all the parameters. In presence of the measurement noise the problem is solved approximately using the following method. Function (19) is represented in the following shape

...

),,,,,,(),,,,,,(

1

1

2

2

1

1

2

2

1

1

2

2

1

1

102010201020101212121

+∂

∂+

∂

∂+

∂

∂+

∂

∂+

∂

∂+

∂

∂+

∂

∂+

+=

tt

Dθ

θ

Dθ

θ

DK

K

DK

K

DR

R

DR

R

D

tθθKKRRDtθθKKRRD

∆∆∆∆∆∆∆

∆∆∆∆∆∆∆∆∆∆

(20) where

10201020102010 ,,,,,, tθθKKRR ∆∆∆∆∆ are some reasonable initial approximations for the values. In our case we take the initial value for ),,,,,,( 10201020102010 tθθKKRRD ∆∆∆∆∆ the actual measured distance between the plots as dtθθKKRRD =),,,,,,( 10201020102010 ∆∆∆∆∆ . To minimize (19) we must require

0),,,,,,( 1212121 =+ dtθθKKRRD ∆∆∆∆∆

(21) Relation (20) can be linearized as follows

dtt

Dθ

θ

Dθ

θ

DK

K

DK

K

DR

R

DR

R

D−=

∂

∂+

∂

∂+

∂

∂+

∂

∂+

∂

∂+

∂

∂+

∂

∂

1

1

2

2

1

1

2

2

1

1

2

2

1

1

∆∆∆∆∆∆∆

(22)


Equations of this type written down for each pair of the plots will result in the following system

−=

∆

∆

∆

∆

∆

∆

∆

∂

∂

∂

∂∂

∂∂

∂

∂

∂

∂

∂∂

∂∂

∂

∂

∂

∂

∂∂

∂∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂∂

∂

∂

∂

∂

∂

∂

∂∂

∂

∂

∂

∂

∂

∂

∂

n

nnnnnnn

d

d

d

d

d

d

t

K

K

R

R

t

D

t

Dt

Dt

D

D

D

D

D

D

D

D

D

K

D

K

D

R

D

R

D

K

D

K

D

R

D

R

DK

D

K

D

R

D

R

DK

D

K

D

R

D

R

D

5

4

3

2

1

1

2

1

2

1

2

1

1

1

3

1

2

1

1

2

2

3

2

2

2

1

1

1

3

1

2

1

1

2121

2

3

1

3

2

3

1

3

2

2

1

2

2

2

1

2

2

1

1

1

2

1

1

1

...

...

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

(23) or in matrix notations

DAX = (24)

where the matrix [A] has n×7 size, the unknown vector X has 7×1 size, the right hand term D is n×1 vector. System (23) has n×7 size (n>>7) and is called over-determined. The solution is sought using the following method

DAAXA TT=

(25)

BAXAT = (26)

where DAB T

= is 7×1 vector, and AAT is a 7×7 matrix which can be decomposed in upper-triangular and lower-triangular matrices as follows (Cholesky decomposition)

CCAA TT=

(27) Then the solution is found as follows separately solving two systems

BYCT=

(28) and then

YCX = (29)

Solving equations (27) and (28) the vector-update [ ]1212121 tKKRR ∆∆∆∆∆∆∆ θθ is obtained, the initial values of the biases are updated and the process is cycled iteratively till the required minimum size of


the update is reached. If the right hand side vector is a random vector, then the approximate solution has the following covariance matrix

12 )()( −

= CCx Tσ

(30) The partial derivatives composing matrix A are calculated as follows.

13

1

22

2

11

1

22

2

11

1

22

2

11

1

)cos(

)cos(

)cos(

)cos(

)cos(

)cos(

)cos(

Vt

D

RD

RD

RK

D

RK

D

KR

D

KR

D

i

i

i

i

i

i

i

α

αθ

αθ

ϕ

ϕ

ϕ

ϕ

=∂

∂

=∂

∂

=∂

∂

=∂

∂

=∂

∂

=∂

∂

=∂

∂

(31) where the corresponding quantities 32121 ,,,, αααϕϕ are shown in figure 34.

R1 R2

X1,Y1,t X2,Y2,t

V1

ϕ2

α2 ϕ1

α1

α3

Figure 34: Two-plot setup for the minimization procedure


5.3.3 Relative Measurement In some cases the absolute measurement is impossible or its accuracy is very low. An example of such a problem is two co-located radars. The corresponding parameters range biases, and azimuth biases

2121 ,,, θθRR ∆∆∆∆ become linearly dependent on each other and the matrix minimization algorithm either generates error or produces abnormal results (for example range bias of 2nm, and range gain of 2000ppm). In these cases relative measurement is performed, i.e. parameters of one of the radars are “locked” taken as constants in this case the system (23) is reduced to the following shape

−=

∆

∆

∆

∆

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂∂

∂

∂

∂

∂

∂

∂

∂∂

∂

∂

∂

∂

∂

∂

∂

n

nnnn

d

d

d

d

d

d

t

K

R

t

DD

K

D

R

D

t

DD

K

D

R

Dt

DD

K

D

R

Dt

DD

K

D

R

D

5

4

3

2

1

1

2

2

2

1222

1

3

2

3

2

3

2

3

1

2

2

2

2

2

2

2

1

1

2

1

2

1

2

1

...

...θ

θ

θ

θ

θ

(32) With the corresponding partial derivatives defined as follows.

13

1

22

2

22

2

22

2

)cos(

)cos(

)cos(

)cos(

Vt

D

RD

RK

D

KR

D

i

i

i

i

α

αθ

ϕ

ϕ

=∂

∂

=∂

∂

=∂

∂

=∂

∂

(33) where the corresponding quantities 322 ,, ααϕ are given in figure 34. In some cases user might want to additionally “lock” one of the remaining parameters (for example the range gain). In such cases the system will be further reduced. Note the range gain locking has a beneficial effect on the range measurement accuracy. Range gain can typically originate from two sources: clock inaccuracy and wrong speed of the light setting during the conversion of the time into range. In modern radar systems typical clock accuracy is excellent not exceeding 1÷10ppm. Typical speed of light setting is typically 299792458 km/s. In the air given the standard atmospheric conditions the speed of light is on average −293ppm less. So that the theoretical range gain for modern radar is likely to be around −293ppm. Those two parameters (range and range gain) might be not independent and influence each other, which is typical for low density air traffic for example. To prevent results with abnormally high range gain and significant errors on the range bias estimate, locking range gains parameters setting the values to −293ppm could be recommended. However the user can easily re-compute and compare the results with the unlocked range gains (other parameters) and adopt either approach.


5.3.4 Accuracy of the Measurement Despite the fact that systems (23) and (32) use accurate derivations and theoretically have accuracy of the measurement represented by a 7×7 covariance matrix, 4×4 for system (32), in practice non-random residual errors always exist, such as for example barometric altitude measurement error, azimuth encoder eccentricity, transponders’ standard delay deviation, wrong positioning etc., so that the exact evaluation of the accuracy of the measurement is difficult or nearly impossible. Performing multi-radar measurement combining many sources of data (as opposed to two-source comparison adopted by Radar Comparator Dual) has the following trend: non-Gaussian errors on a number of parameters neglected by the model tend to spread out and contaminate the other parameters. Since in practice there are always a number of factors neglected by the model2, the covariance matrix can not be trusted as an estimate of the errors of the measurement. The measurement accuracy can be evaluated as follows. A particular data source can be sequentially compared with a number of independent sources (radars, ADS-B, etc.) As the result it is not only possible to produce the average and standard deviation for the measurement but also detect and fix possible errors of all the sources first just comparing them between each other. That is the typical usage guideline for the Radar Comparator. Compare results produced by the more than one source when evaluating radar and you will find something very specific about these, you will fix it and then produce the more accurate measurement at the end.

5.4 Radar vs. ADS-B Analysis

5.4.1 Matrix Structure A typical approach of using the ADS-B data for radar evaluation is the following. The ADS-B data set is converted from (WGS84 LONGITUDE, LATITUDE, 3D Height) to the radar polar coordinate system then the ADS-B data are treated as if they were virtual radar data. ADS-B source theoretically shouldn’t have any bias (range, range gain, azimuth) so these parameters are locked and the radar under test is measured vs. ADS-B in a relative way. Properly selecting high accuracy ADS-B data for the analysis must provide means of the direct absolute measurement and monitoring of the radar alignment and accuracy. ADS-B messages don’t contain time and are transmitted by the same AC SSR/MODES transponder used for radar surveillance communication. There exist a delay between the moment when the position data were actually measured by GPS receiver and their transmission. The average of this parameter is defined as the latency. The latency can be of 1-2 s i.e. too large to be ignored for the purpose of accurate radar evaluation. So instead of using system (32) with 4 parameters tθKR ∆∆∆ ,,, 222

we should use N×+ 24 or N×+ 23 where N is the number of AC, adding 2 extra parameters per AC in particular the latency and

transponder delay deviation from 3µs (further the transponder delay) leaving ( )NiRtθK ii ,0,,, 22 =∆∆∆ . Since the radar range bias and the transponders delays are dependent quantities one would need to lock the range bias as well. Due to the similar reason the time bias must be locked as well leaving

( )NiRtθ ii ,0,,2 =∆∆∆ . In the result we would obtain the following system

2 As we have pointed out the barometric height errors typically influence the measurement accuracy, another parameter that is not taken into account by radar to radar evaluation is a particular distribution of the AC transponders. The transponders have standard 3µs delay ±0.5 µs for SSR replies and 3µs delay ±0.25 µs for MODES replies. This may produce up to ±75m and ±38m errors respectively. Thus different sets of the transponders may produce different delay distributions which will influence the range bias (gain) predictions.


−=

∆

∆

∆∆

∆

∆

∆

∆

∆

∆

∆

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂∂

∂

∂

∂∂

∂

∂

∂

∂

∂

∂

∂

∂

∂∂

∂∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂

∂∂

∂

∂

∂

∂

∂∂

∂

∂

∂

∂

∂

−−

++

++

++

−

+

+

+

nm

m

k

k

m

n

m

n

m

n

m

n

k

i

k

i

k

i

k

i

k

i

k

i

n

n

i

i

i

iii

d

d

d

d

d

d

t

r

tr

t

r

t

r

t

r

t

D

t

D

r

D

r

D

t

D

r

Dt

D

r

Dt

D

r

D

D

D

D

D

D

t

D

r

DD

t

D

r

DDt

D

r

DDt

D

r

DD

...

...

...

...

...

0

...

0...

...

00

...

...

...

...

...

...

5

4

3

2

1

3

3

2

2

1

1

2

11

33

22

11

2

2

1

2

3

2

2

2

1

112

1

3

1

3

2

3

1

2

1

2

2

2

1

1

1

1

2

1 θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

(34) With the corresponding partial derivatives defined as follows.

13

22

22

2

)cos(

)cos(

)cos(

Vαt

D

Kr

D

Rαθ

D

k

i

k

i

i

=∂

∂

=∂

∂

=∂

∂

(35) with MkandNi ,1,1 == Once the latencies and transponder delays are determined the transponder delays can be used to predict the radar range bias. Note transponder delays measured contain radar range bias component in them that is why they can not be compared if measured using ADS-B set with this or that radar. However the radar range bias can be inferred more accurate after a sufficiently long measurement, and then the transponder delays might be measured nearly independent from the radar bias. The same consideration is valid for the latency and the timestamp accuracy (time bias) of the radar. Restrictions apply for the analyzed trajectories. For some trajectories separate measurement of the transponder delay and latency is impossible, for example too short segment or nearly radial trajectory for inbound or outbound AC. The software evaluate quality of the measurement for each trajectory setting Measurement Figure of Merit (MFM) numbers between 0÷10, see section 5.4.2. For most accurate measurements trajectories with low MFM should be filtered out.

5.4.2 Measurement Figure of Merit (MFM) As opposed to FOM contained in an ADS-B message, Measurement Figure of Merit (MFM) characterizes the accuracy of the measurement of AC latency and transponder delay by the method. We have defined this quantity for each trajectory as follows,

NiVRN

VRN

FMFN

i

ii

N

i

ii ,011

1000

=

×−×= ∑∑

==

,

(36) where iR is the range vector in a point of the trajectory, iV is the speed vector of the target in the same point. Formula (36) represents a measure of the trajectory balance. Other definitions of this parameter are also possible. Thus for some trajectories separate measurement of the transponder delay and latency is


impossible, for example too short segment or nearly radial trajectory for inbound or outbound AC. The software evaluates quality of the measurement for each trajectory computing MFM numbers between 0 (lowest) and 10 (highest).

5.4.3 Dual Comparison: Accuracy of the Measurement of the Systematic Errors The measurement approach and accuracy depends on a particular radar setup as well as the amount and spatial distribution of the air traffic. As we have already mentioned for two co-located radars the absolute measurement is impossible, however the measurement of the random errors can be still performed. For radars separated by a distance an absolute measurement of the alignment errors becomes possible. For radars separated by a very large distance, accuracy of the absolute measurement of the alignment errors decreases. Similarly with ADS-B latency analysis quantitative evaluation the FOM of the particular measurement setup is required. This estimator is currently under development.

5.5 Trajectory Reconstruction and Random Errors The main prerequisite for the trajectory reconstruction is absence of the systematic errors in the data. This is why trajectory reconstruction is to be performed after the measurement and compensation of all the systematic errors. When all the residual errors are described by zero-bias Gaussian distribution law the following minimization algorithm can take place. For each track (trajectory) the following quantities are minimized: speed noise vector ( )yx VV , , azimuth and range errors for both sources ( )11,θR and ( )22,θR , in the case when ADS-B source is used the random ( )YX, errors instead of the range and azimuth errors are used. Since the above random errors might be substantially different in size, the proper statistical weighting is required. The solution is found iteratively in minimizing the following system

ii

i

ii

i

i

ii

i

ii

i

i

ii

i

ii

i

i

ii

i

ii

i

i

yii

i

yi

xii

i

xi

θyy

θx

x

θ

Ryy

Rx

x

R

θyy

θx

x

θ

Ryy

Rx

x

R

Vyy

V

Vxx

V

222

222

111

111

−=∂

∂+

∂

∂

−=∂

∂+

∂

∂

−=∂

∂+

∂

∂

−=∂

∂+

∂

∂

−=∂

∂

−=∂

∂

∆∆

∆∆

∆∆

∆∆

∆

∆

(37) Equations of this type written down for each point Ni ,1= of the trajectory will result in the following system


−=

∂

∂

∂

∂∂

∂

∂

∂∂

∂

∂

∂∂

∂

∂

∂

∂

∂∂

∂

∂

∂

∂

∂∂

∂

∂

∂∂

∂

∂

∂∂

∂

∂

∂

∂

∂∂

∂∂

∂

∂

∂∂

∂

∂

∂

∂

∂

∂

∂∂

∂

∂

∂∂

∂∂

∂

n

n

n

n

xn

xn

x

x

x

x

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

yn

n

xn

y

x

y

x

θ

Rθ

RV

V

θ

Rθ

RV

V

θ

Rθ

RV

V

y

x

yx

y

x

y

θ

x

θy

R

x

Ry

θ

x

θy

R

x

R

y

Vx

V

y

θ

x

θy

R

x

Ry

θ

x

θy

R

x

R

y

Vx

Vy

θ

x

θy

R

x

R

y

θ

x

θy

R

x

Ry

Vx

V

2

2

1

1

21

21

11

11

2

1

21

21

11

11

2

1

2

2

1

1

22

22

1

12

1

12

11

2

22

2

22

2

22

2

22

2

12

2

12

2

12

2

12

2

2

2

2

1

21

1

21

1

21

1

21

1

11

1

11

1

11

1

11

1

1

1

1

...

...

...

...

0

0

......

......

0

0

0

0

∆∆

∆∆∆∆

(38) or in matrix notations

DAX = (39)

where the matrix A has n×6n size, the unknown vector X has n×1 size, the right hand term D is 6n×1 vector, where n is the number of plots (tracks) in the trajectory. Solving the system (38) is time consuming given the size of the matrix. More practical alternative is sequential solving subsets of system (38). In order to decrease statistical weights of the equations related to the larger distances between the plots each equation

of (38) is multiplied by the corresponding iw

1 weight function. The solution is sought using the matrix

processing methods described above. The corresponding partial derivatives encountered in system (38) are given as follows for simplicity indexes i are being omitted,

( ) ( )

( )

( )

( )

( )

1

5.02

2

2

2222

5.02

2

2

2222

5.02

1

2

1111

5.02

1

2

1111

1

23

1

12

=∂

∂=

∂

∂

+=∂

∂=

∂

∂

+=∂

∂=

∂

∂

+=∂

∂=

∂

∂

+=∂

∂=

∂

∂

−+−=∂

∂=

∂

∂

−

−

−

−

−−

y

Y

x

X

yxyx

θ

y

R

yxxy

θ

x

R

yxyx

θ

y

R

yxxy

θ

x

R

tttty

V

x

V yx

(40)


Note the system produces the correct solution if the initial estimate for the reconstructed trajectory is close to the true position. To generate this first estimate of the true trajectory the following algorithm is used. The data coming from the two sources are re-sampled to bring them to the same timestamp. Of course re-sampling data using interpolation procedure induces some extra errors, but most of the time they have the same order of magnitude as the random errors of the radar in question. So that re-sampled plots presumably have nearly the same positional accuracy as the raw plots of the radar. Given that the plot has range and azimuth errors we can build for radar 1 the following likelihood function

∆−

∆−=

2

1

1

2

1

111 exp

2

1),(

σθ

θ

σπθ

R

RRP ,

(41) where '111 RRR −=∆ ,

111 'θθθ −=∆ ,11,σθRσ are standard deviations for range and azimuth

respectively. If there are two plots taken at the same time by the two radars (one is typically being the real plot and the other interpolated the likelihood function of the true position will be defined as the product of two likelihood functions of type (40). For radar 2 the following likelihood function is

∆−

∆−=

2

2

2

2

2

222 exp

2

1),(

σθ

θ

σπθ

R

RRP ,

(42) where '111 RRR −=∆ ,

111 'θθθ −=∆ ,11,σθRσ are standard deviations for range and azimuth

respectively. The joint likelihood function is

−−

−−

−−

−−=

2

2

22

2

2

22

2

1

11

2

1

1122211

''''exp

4

1),,,(

σθ

θθ

σσθ

θθ

σπθθ

R

RR

R

RRRRP ,

(43) the ',' iiR θ are coordinates of the corresponding detections and

iiR θ, coordinates of the real position of the target as detected by the i-th radar. Maximizing function (20) equals to maximizing the following function

( )( ) ( ) ( ) ( )

2

1

2

112

1

2

112

2

2

222

2

2

222121

'''',,,

σθ

θθ

Rσ

RR

σθ

θθ

Rσ

RRθθRRG

−

−

−

−

−

−

−

−=

(44) Let us assume that the density of the probability parabolas constitute with respect to the X-axis of the reference system angles α, and β are respectively for radar 1 and radar 2 (see figure 35). Then the local coordinates x, y and x’, y’ are related to the reference system coordinates X, Y as follows ( ) ( )11 Y-ysin-X-x-cosx αα= ( ) ( )1l Y-ycosX-x-siny αα += ( ) ( )22 Y-ysin-X-x-cos' ββx =

( ) ( )22 Y-cosX--sin'y yβxβ +=

(45)


where 11 Y,X and

22 Y,X are respectively plot 1 and plot 2 coordinates in the reference system. Then function (44) in the reference system is given

( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( )2

2

22

2

22

22

2

1

11

2

11

11

R

sin X--cos Y--

sin Y-cos X--

R

sin X--cos Y--

sin Y-cos X--,

+

+=

σ

βxβy

σθR

βyβx

σ

αxαy

σθR

αyαxyxG

(46) which represents the sum of two parabolas, whose maximum is determined analytically as follows

( )

( )0

YX,

0YX,

=∂

∂

=∂

∂

Y

GX

G

(47) System (27) yields 2 linear equations for X and Y, analytically solved as follows.

R2, θ2R1, θ1

σR2

σθ2 σθ1

σR1

Reconstructed position

X

Y

α β

X1,Y1 X2,Y2

y y’

x

x’

Figure 35:Principle of the trajectory reconstruction (search for the maximum likelihood function given the

radar measurements and expected radar accuracy)

( ) ( ) ( ) ( )

+

+++

+=

22

2

2222

2

2221

2

1121

2

11 xas-yacyasxacxs-ycysxc-Y) F(X,

RσσθRσσθ

(48) where 11,cs are the sine and cosine of the range vectors in the reference system. After some manipulations we obtain

)/ABA-)/(BC-/AA(CY

)CY(B-1/AX

11222121

111

=

+=

where


)aab(bY-)bab(aX-)aab(bY-)bab(a-XC

aabbaabbB babababaA

)bab(aY-)bba(aX-)bab(aY-)bba(a-XC

babababaB bbaabbaaA

443324433222111221112

443322112443322112

443324433222111221111

442233111443322111

++++=

+++=+++=

++++=

+++=+++=

and

2

14

2

14

2

13

2

13

1

12

1

12

1

11

1

11

sb,

ca,

sb,

ca,

sb,

ca,

sb,

ca

RσRσσθσθRσRσσθσθ−====−====


6. References [1] ICAO International Standards and Recommended Practices Aeronautical Telecommunications, ANNEX 10, (Surveillance Radar and Collision Avoidance Systems), Second Edition July 1998 [2] SUR/MODES/EMS/SPE-01 (form. SUR.ET2.ST03.3114-SPC-01-00) European Mode-S Station Functional Specification, p 24 Released Issue, Edition: 3.11

IE-SUP-00042-001 Radar Alignment and Accuracy Tool Radar... · Radar Alignment and Accuracy Tool:...

Documents

Transcript of IE-SUP-00042-001 Radar Alignment and Accuracy Tool Radar... · Radar Alignment and Accuracy Tool:...