EUROCONTROL€¦ · SAPPHIRE RAIM ALGORITHMS VALIDATION EEC Note No.16/01 Project GNS-Z-E2 Issued:...

EUROPEAN ORGANISATIONFOR THE SAFETY OF AIR NAVIGATION

EUROCONTROL EXPERIMENTAL CENTRE

RAIM STUDY ANDSAPPHIRE RAIM ALGORITHMS VALIDATION

EEC Note No.16/01

Project GNS-Z-E2

Issued: June 2001

The information contained in this document is the property of the EUROCONTROL Agency andno part should be reproduced in any form without the Agency’s permission.

The views expressed herein do not necessarily reflect the official views or policy of the Agency.

EUROCONTROL

REPORT DOCUMENTATION PAGE

Reference:EEC Note No.16/01

Security Classification:Unclassified

Originator:EEC BretignyGNSS Programme

Originator (Corporate Author) Name/Location:EUROCONTROL Experimental CentreCentre de Bois des BordesBP1591222 Brétigny-sur-Orge CEDEXFRANCETelephone : +33 (0)1 69 88 75 00

Sponsor: Sponsor (Contract Authority) Name/Location:

TITLE:

RAIM Study and- SAPPHIRE RAIM ALGORITHMS VALIDATION -

AuthorsA. LHERMITE

Date06/2001

PagesX+26

Figures23

Tables3

Appendix1

References15

TaskSpecification

-

ProjectGNS-Z-E2

Task No. Sponsor

-

Period10/2000 – 02/2001

Distribution Statement:(a) Controlled by: EEC - Head of the EATMP GNSS Programme(b) Special Limitations: None

Descriptors (keywords):

SAPPHIRE, DUAU, RAIM, Algorithms, Validation, Satellite Navigation

Abstract:

This report presents the results of the analysis of the seven different RAIM Algorithmsimplemented in the SAPPHIRE DUAU system. The study concludes with the validation of six ofthe algorithms by comparing the theoretical behaviour described in literature and the resultsobtained from two algorithms already independently validated. The study uses real flights onwhich some errors could be simulated.

SAPPHIRE – RAIM Algorithms ValidationEUROCONTROL

v

FOREWORD

This note presents the result of the analysis of the seven different RAIM Algorithms implemented intoSAPPHIRE DUAU system.

This analysis was not only the occasion to validate the different algorithms in comparison with theirexpected behaviour, but also the opportunity to evaluate their performance.

Six of the seven algorithms were validated through a comparison process with the results providedby two of them already independently validated. A further note is intended to report on the validationof the remaining algorithm.

Real flight data were used for the study, among which some were modified with the addition ofsimulated errors. The analysis was carried out on a sample by sample basis.

This document is part of a series of regular publications reporting on the results derived within theSAPPHIRE Project.

We would like to thank the EATMP GNSS Programme and our colleagues from the University ofBraunschweig for their valuable inputs and their support to this study.

Aline LHERMITE

Bernd TIEMEYER

EUROCONTROL Experimental CentreEATMP GNSS Programme


vi


vii

TABLE OF CONTENTS

1. INTRODUCTION.............................................................................................................................. 11.1 General ...................................................................................................................................... 11.2 Overview.................................................................................................................................... 11.3 Glossary..................................................................................................................................... 1

2. VALIDATION APPROACH .............................................................................................................. 22.1 RAIM Algorithms implemented in THE SAPPHIRE DUAU System........................................... 22.2 Input Data for RAIM Algorithm tests .......................................................................................... 22.3 Validation Process ..................................................................................................................... 2

3. RAIM THEORY ................................................................................................................................ 33.1 General ...................................................................................................................................... 33.2 RAIM Algorithm of Sturza/Brown (Algorithm_Id 3 and 4) .......................................................... 43.3 RAIM Algorithm of Brenner (Algorithm_Id 5) ............................................................................. 43.4 RAIM Algorithm of MOPS (Algorithm_Id 6) ............................................................................... 43.5 RAIM Algorithm of RG. Brown (Algorithm_Id 7, 8 and 9) .......................................................... 4

3.5.1 Identification process for FDI Algorithm (Algorithm_Id 7)................................................... 53.5.2 Identification process for FDE Algorithm (Algorithm_Id 8) ................................................. 53.5.3 Identification process for PI Algorithm (Algorithm_Id 9) ..................................................... 5

4. RAIM RESULTS ON AN IDEAL FLIGHT WITH SIMULATED ERRORS ....................................... 64.1 Description OF ERROR MODELLING ...................................................................................... 64.2 Availability Results Analysis....................................................................................................... 74.3 Detection Results Analysis ........................................................................................................ 7

4.3.1 Early detection.................................................................................................................... 74.3.2 Algorithm behaviour in the presence of the different types of simulated errors ................. 94.3.3 Verification of theory........................................................................................................... 9

4.4 Identification Results Analysis ................................................................................................. 114.4.1 Verification of Theory........................................................................................................ 114.4.2 Satellite identified ............................................................................................................. 13

5. RAIM RESULTS ON REAL FLIGHTS........................................................................................... 195.1 Availability Results Analysis..................................................................................................... 195.2 Detection Results Analysis ...................................................................................................... 195.3 Identification Results Analysis ................................................................................................. 20

5.3.1 Verification of theory......................................................................................................... 205.3.2 Multipath problems ........................................................................................................... 21

6. CONCLUSIONS............................................................................................................................. 22

7. ACKNOWLEDGEMENTS.............................................................................................................. 23

8. REFERENCES............................................................................................................................... 24

APPENDIX 1 : SIMULATED A340 ERROR FLIGHT - ERROR LIST .................................................. 25


viii

LIST OF FIGURES

Figure 1: PR Peak error ......................................................................................................................... 6Figure 2: PR ramp error......................................................................................................................... 6Figure 3: PR step error .......................................................................................................................... 6Figure 4: PR Gaussian Markovian error .............................................................................................. 6Figure 5: Alarm limit and position error comparison for detection - FK 100090 ............................. 8Figure 6: Detection Occurrence (algorithm 10 shows the ideal behaviour) .................................... 8Figure 7: Detection problem of algorithm 8 - FK 100090 ................................................................. 10Figure 8: Automatic identification when detection occurs for algorithms 3, 4 and 9 - FK

100090 ................................................................................................................................. 11Figure 9: identification interval for algorithm 7 and 8 - FK 100090................................................. 12Figure 10: Pseudorange residuals behaviour in a single ramp error case - FK 100090 ............... 12Figure 11: Interrelation between Pseudorange residuals ................................................................ 13Figure 12: Wrong identification for algorithm 8 - FK 100090........................................................... 14Figure 13: Pseudorange residual behaviour in the case of a ramp error on a single

pseudorange- FK 100090................................................................................................... 14Figure 14: two similar ramp errors - FK 100090................................................................................ 15Figure 15: Identification problems in the case of two same ramp errors - FK 100090 ................. 15Figure 16: Pseudorange residual behaviour in the case of two similar ramp errors - FK

100090 ................................................................................................................................. 16Figure 17: two different ramp errors - FK 100090 ............................................................................. 17Figure 18: Identification problems in the case of two different ramp errors - FK 100090 ............ 17Figure 19: Pseudorange residual behaviour in the case of two different ramp errors ................. 18Figure 20: RAIM Availability – FK 210015 .......................................................................................... 19Figure 21: Detection and Identification occurrence - FK 210019 .................................................... 20Figure 22: Pseudorange residuals behaviour - FK 210019 .............................................................. 20Figure 23: CNO's - FK 210019 ............................................................................................................. 21

LIST OF TABLES

Table 1: Algorithms principles.............................................................................................................. 3Table 2: Alarm Limit and Phases of Flight........................................................................................... 7Table 3: Time to Alarm and Averaging interval for RAIM MOPS algorithm.................................... 10


ix

SUMMARY

The SAPPHIRE Database Update and Access Unit (DUAU) system consists of two parts: a databasecontaining real and simulated measurements and a data evaluation part dedicated to provide statisticalanalysis on the performance of Satellite Navigation.

This report summarises the result of the analysis of seven different Receiver Autonomous IntegrityMonitoring (RAIM) Algorithms implemented in the SAPPHIRE DUAU system. The study is based on asample by sample comparison of RAIM results and behaviours between the seven algorithms, two ofwhich had been independently validated in an earlier study [15].

This study was performed between October 2000 and March 2001 using real flight data, with andwithout additional simulated errors.

Validation of six of the seven RAIM algorithms implemented into SAPPHIRE DUAU system wassuccessfully concluded. Further work needs to be carried out on the validation of the remainingalgorithm.


x


1

1. INTRODUCTION

1.1 GENERAL

The aim of this study was to validate whether the seven RAIM Algorithms implemented in theSAPPHIRE DUAU system function correctly. The behaviour of the algorithms was tested on bothsimulated flight data with modelled errors and real flight data (9 Boeing B747 and one simulatedAirbus A340 flight). The objective was to verify that the behaviour of the seven algorithmscorresponds with the theory described in the literature. In the comparison process, two algorithms ofthe seven, which were already independently validated, were used as reference.

1.2 OVERVIEW

This document is divided into the following chapters:

• Chapter 2, Validation Approach. In this chapter the general approach to the comparison ofRAIM Algorithms implemented into SAPPHIRE DUAU system is described.

• Chapter 3, RAIM Theory. This chapter summarises the theoretical background for the differentRAIM Algorithms.

• Chapter 4, RAIM Results on flights containing simulated errors. This chapter describes theperformance of each algorithm during simulated error scenarios.

• Chapter 5, RAIM Results - Real Flights. This chapter contains the results describing theperformance of the different algorithms applied to real flight data.

• Chapter 6, Conclusions. This chapter presents the conclusions of the report.

1.3 GLOSSARY

AL Alarm LimitARP Approximated Radial-Error ProtectedCFAR Constant False Alarm RateCPOD Constant Probability Of missed DetectionDR Delta RangeDUAU Database Update and Access UnitEEC EUROCONTROL Experimental CentreEPE Estimated Position ErrorFD Fault DetectionFDI Fault Detection and IdentificationFK Flight KeyGNSS Global Navigation Satellite SystemGPS Global Positioning SystemID IdentifierPI Partial IdentificationPFA Probability of False AlarmPMD Probability of Missed DetectionPRN Pseudo Random NoisePRRes X PseudoRange Residual on satellite XRAIM Receiver Autonomous Integrity MonitoringRTCA Radio Technical Commission for AeronauticsSAPPHIRE Satellite and Aircraft Database Project for System Integrity ResearchTPE Total Position ErrorXPL For HPL and VPL, respectively Horizontal and Vertical Protection Level


2

2. VALIDATION APPROACH

2.1 RAIM ALGORITHMS IMPLEMENTED IN THE SAPPHIRE DUAU SYSTEM

The aim of this report is to gather conclusions on the comparison of seven RAIM algorithmsimplemented in the SAPPHIRE DUAU system.

These algorithms have been implemented according to the following sources:- Constant False Alarm Rate (CFAR) by Sturza/Brown (ALGORITHM_ID 3)- Constant Probability of Missed Detection (CPOD) by Sturza/Brown (ALGORITHM_ID 4)See reference [1] to [5] for CFAR and CPOD Fault Detection and Isolation (FDI) RAIM Algorithmsby Sturza and Brown.- Constant False Alarm Rate (CFAR) by Brenner (ALGORITHM_ID 5), see [6].- MOPS Baseline (ALGORITHM_ID 6), see [7] and [8].- Fault Detection and Identification (FDI) by R.G.Brown (ALGORITHM_ID 7)- Fault Detection and Exclusion (FDE) by R.G.Brown (ALGORITHM_ID 8)- Partial Identification (PI) by R.G.Brown (ALGORITHM_ID 9)See [9] to [13] for the description of Algorithms proposed by R.G. Brown.

2.2 INPUT DATA FOR RAIM ALGORITHM TESTS

Different inputs were required to support the validation:- One real A340 flight (ideal one) with additional simulated errors which covers a high number oferror scenarios, in order to test the behaviour of each algorithm with respect to known errors.- 9 real B747 flights (identified in the DUAU Database with the labels FKs 210013, 210015,210018, 210019, 210034, 210035, 210036, 210037, 210042), to test algorithm behaviour on realflight data

2.3 VALIDATION PROCESS

The validation process was performed step by step as follows:- First it was necessary to extract from the literature the principles and the expected behaviour ofeach algorithm.- The algorithms were then tested with simulated erroneous and real flight data.- Finally, the comparison of results between the different algorithms and with the theory leads toconclusions on the performance and the validation of each one.


3

3. RAIM THEORY

The aim of this chapter is to summarise the description of the RAIM algorithm and the basic principlesof each algorithm implemented into the SAPPHIRE DUAU Software.

3.1 GENERAL

The aim of the RAIM Algorithms implemented in SAPPHIRE is to detect errors in the rangemeasurements to individual satellites used in the position calculation and, if possible, to identify thefaulty satellite. It should be underlined that all the algorithms (except for one) are designed to identifysingle satellite failures only.

Whenever a failure is detected and identification succeeds, the position computation should beperformed again excluding the faulty satellite(s). But this second step is not automated in theSAPPHIRE DUAU System, and the user has to manually exclude the erroneous satellite to obtain the‘true‘ position.

The results depend on the satellite constellation (number of satellites, geometry) and the requirementsfor different phases of flight.

The Failure Detection principle generally relies on the use of redundant information. Therefore, aminimum of 5 satellites is necessary. During the detection process a decision variable, D derived fromthe current measurement data is compared to a threshold, T. T depends on requirements for thereliability and precision of the position solution, and the visible constellation. It is derived from themodel of the probability density function of the decision variable, which is different for every algorithm.If D is bigger than T, then a failure is detected and an identification process follows. If not, everything iswithin specification.

If either the available satellite geometry does not enable the algorithm to detect errors within therequirement constraints or the minimum constellation is not available (less than 5 satellites), RAIM isunreliable (not available) and no further computation is made (detection is not tested).

After failure detection, the algorithms behave differently. While some try to identify the faulty satellite byworking on various subsets of satellites, others exclude the one or two satellites most likely to fail. Forthose of the first type additional redundancy is needed and a minimum of 6 satellites is required in theinitial constellation. For the second type, the optimisation process does not require additional satellites,5 satellites are sufficient (6 in the case of a 2-satellite-exclusion).

The detection processes are similar for all algorithms, relying on computation using the pseudorangeresiduals and their probability distribution, they differ in their “identification” or “exclusion” processes.

The following Table 1 summarises the different principles:

Algorithm(id) Detection utilising test statistic:decision variable distribution

Identification by …

Sturza/Brown(3,4) Chi 2 distribution Maximum likelihood methodBrown (7,8,9) Chi 2 distribution Binary search on subset utilising statistic testMOPS (6) Chi 2 distribution Binary search on subset utilising statistic testBrenner (5) Normal distribution Binary search on subset utilising statistic test

( + optimisation for algorithm 9 )

Table 1: Algorithms principles


4

3.2 RAIM ALGORITHM OF STURZA/BROWN (ALGORITHM_ID 3 AND 4)

The RAIM algorithms of Sturza/Brown are based on the calculation of a decision variable D definingthe length of the pseudorange residual vector (see [1] to [5]).

Two different methods can be used. The first one, implemented in algorithm 3, relies on integritycomputation based on a Constant False Alarm Rate (CFAR), whereas algorithm 4 uses a ConstantProbability Of missed Detection (CPOD).

For both algorithms, identification is based on the maximum likelihood method. Each time a failuredetection occurs the satellite most likely to cause the failure is identified as the error source.

The first of the two algorithms was independently validated [15], and is considered as the reference inthe validation process.

3.3 RAIM ALGORITHM OF BRENNER (ALGORITHM_ID 5)

Following the Brenner RAIM methodology, n decision variables D are computed (n being the number ofsatellites tracked by the user receiver) using orthogonal projections of the pseudorange residuals. Thisway, satellites are examined individually, and the failure detection process is optimised (see [6]).Detection occurs if any decision variable exceeds the computed threshold.

Identification is processed using the same methodology: n-1 decision variables are computed for each‘subset k’ of n-1 satellites (where satellite k is excluded). Satellite k is identified if the decision variablefor ‘subset k’ is the only one below the threshold.

The CFAR method is implemented and was also independently validated [15]. This implementationserves as a reference in the validation process.

3.4 RAIM ALGORITHM OF MOPS (ALGORITHM_ID 6)

This MOPS Baseline algorithm is generally described as a simplified version (as far as the detectionprocess is concerned) of the CFAR algorithm of Sturza/Brown. The only slight difference is that thefinal decision variable (length of pseudorange residual) is the averaged one over an interval of time.This is the only algorithm implemented in SAPPHIRE DUAU system that takes into account part of thehistory of the signal. An inconvenience of this method is that the smoothing effect induced by the use ofaveraging could decrease the number of failures detected. In order to reduce this effect two decisionvariables are computed within the time-to-alarm limit (which depends on the flight phase) (see [7] and[8]).

Concerning Identification, the same method is used on the n-1 subsets of n-1 satellites (subset k,where satellite k is excluded). Satellite k is identified if the decision variable for subset k is the only onebelow the threshold. Otherwise identification is not performed.

3.5 RAIM ALGORITHM OF RG. BROWN (ALGORITHM_ID 7, 8 AND 9)

The FDI, FDE and PI algorithms from RG. Brown differ from each other only from the identificationpoint of view. Concerning detection, they were all implemented in the same way. For the FDE algorithm(algorithm 8) the process functions on only 6 measurements regardless of the number of satellites thatare visible (see [9] to [13]).

The detection process is similar to the CFAR approach of Sturza/Brown where the decision variable isderived from the length of the pseudorange residuals. The difference is the test for availability of RAIM.For all algorithms described above, reliability of RAIM is ensured based on Probability of False Alarm(PFA) and Probability of Missed Detection (PMD) with respect to flight phase requirements. Here thetest is made on a different parameter called Approximated Radial-Error Protected (ARP). ARP can be


5

considered as the forerunner for the Protection Levels XPL’s (Horizontal and Vertical Protection Levelsused in today’s SBAS and GBAS systems). In that case, RAIM algorithms are considered to beavailable if the ARP stays within specifications that would be derived from the one existing on VPL andHPL.

3.5.1 Identification process for FDI Algorithm (Algorithm_Id 7)

The FDI identification process is similar to the one used in the MOPS algorithm. When detectionoccurs, identification is based on subsets of n-1 satellites. For each subset k, where satellite k isexcluded, a decision variable is computed. Then, satellite k is identified if decision variable for subset kis the only one below the threshold.

3.5.2 Identification process for FDE Algorithm (Algorithm_Id 8)

In theory, the identification process is carried out at the same time as the detection process. Thealgorithm chooses the subset of 6 satellites that does not produce a detection alarm, excluding at thesame time satellites with potential errors.In reality, the detection process is run on the n satellites tracked by the user receiver. If detectionoccurs the identification process is based on the comparison of the decision variables for subsets of n-1 satellites. The faulty satellite identified is the first one which, when excluded from the constellation,gives a subset that no longer produces a RAIM alarm.

3.5.3 Identification process for PI Algorithm (Algorithm_Id 9)

This identification process is comparable to the one used in the algorithms 3 and 4. Every timedetection occurs, two satellites are excluded from the constellation. The algorithm chooses the twosatellites which, when excluded from the constellation, provide a subset (n-2 satellites) with the lowestdecision variable.

If the number of visible satellites equals 6, the FDI identification process of algorithm 7 is used.


6

4. RAIM RESULTS ON AN IDEAL FLIGHT WITH SIMULATED ERRORS

4.1 DESCRIPTION OF ERROR MODELLING

A range of error scenarios was used to validate the correct performance of the different algorithms. Itwas decided to use a real and ideal Airbus A340 flight, where no GPS constellation limitations could beidentified, in which different simulated errors were included.

These errors model some typical and real problems:

- Case 1: Clock error producing PseudoRange (PR) peaks.

Figure 1: PR Peak error

- Case 2: Clock Drift inducing Ramp error onPR.

Figure 2: PR ramp error

- Case 3: Power interruption, maintenanceconsequences or change of ephemeris datainducing PR steps.

Figure 3: PR step error

- Case 4: Signal/Transmission fluctuations(SA) producing Continuous variation in PRmodelled by a Gaussian Markovian model.

Figure 4: PR Gaussian Markovian error


7

These four cases were tested in the following two scenarios:- Single satellite failure.- Two satellite failures (similar type of errors starting at different times).

These scenarios were simulated during periods of time when no change in satellite constellation orphase of flight took place.

The complete list of the errors introduced into the simulated flight can be found in Appendix 1.

4.2 AVAILABILITY RESULTS ANALYSIS

The reason for using data from real flights with simulated errors was to test the performance of thedifferent algorithms in the presence of well known errors. In order to avoid availability problems ofRAIM and concentrate on that particular goal, the real flights were chosen to guarantee a minimalconstellation in terms of number of visible satellites and constellation geometry.Therefore only test conclusions for detection and identification are available in this part of the report.RAIM Unavailability is not considered here.

4.3 DETECTION RESULTS ANALYSIS

4.3.1 Early detection

The objective of RAIM is to produce an alarm whenever the position error reaches the Alarm Limit(AL). This AL is part of the requirements and depends on the phase of flight. The different values areindicated in the following table:

Phase of Flight (Phase_Id) Alarm Limit (m)Departure (2) 555En Route (3) 1850Terminal (4) 1850

Initial (5) 555NPA (6) 555

Table 2: Alarm Limit and Phases of Flight

Detection quality was evaluated by the comparison of the Estimated Position Error (EPE) and theAlarm Limit (AL). The Estimated Position Error was obtained by the difference between the twopositions, computed before and after the introduction of the modelled errors in the flight data. Becauseatmospheric and clock errors are similar for both positions, the EPE represents only the contribution ofthe pseudorange errors into the Total Position Error (TPE). The EPE is therefore an estimate of TPEwith a precision of approximately 10 meters.


8

In Figure 5, Horizontal Position Error and Horizontal Alarm Limit are represented.

0

500

1000

1500

2000

2500

3000

3500

4000

0 5000 10000 15000 20000 25000Samples [s]

Hor. Pos. errorAL (HPL)

Figure 5: Alarm limit and position error comparison for detection - FK 100090

According to Figure 5, detection must occur at three different intervals. Detection results were laterobtained for each algorithm and it appears that detection occurs more often than required (Figure 6).

2

3

4

5

6

7

8

9

10

11

0 5000 10000 15000 20000 25000sample id

Alg

o id

Figure 6: Detection Occurrence (algorithm 10 shows the ideal behaviour)


9

Different conclusions can be drawn from Figures 5 and 6. Generally it can be concluded that all theimplemented algorithms detect failure problems earlier than expected. This behaviour is referred to asearly detection. Because early detection occurs in the cases of continuous increasing ramp errors, theassumption can be made that the algorithms take into account the near past of the Position Error, andits tendency to raise the RAIM alarm. Thus Detection seems not to be only a function of the currentcalculated Position Error.

It can also be seen that detection stops as soon as failures disappear. In that case the behaviour of thePosition Error changes suddenly, no prediction can be formulate regarding its tendency, and thedecision whether to raise a RAIM alarm or not is guided by the instantaneous value of the PositionError.

Because of time constraints, the assumptions on the capabilities of the algorithms to take into accountthe past and to extrapolate the evolution of the Error values were not be verified in the source code.This could be investigated in a further note.

Further conclusions derived from these two figures will be discussed in the following paragraphs.

4.3.2 Algorithm behaviour in the presence of the different types of simulated errors

4.3.2.1 Same behaviour

Figure 6 shows that all the implemented algorithms detect a failure during the same time intervals. Thelikelihood of every algorithm detection process (calculation and origin of threshold detection) deducedfrom the theory, is confirmed here. But a more detailed analysis has to be carried out to issue precisecomments and draw conclusions on the different algorithms. The following paragraphs aim to do this.

Except Peak and SA errors, every scenario of error (single or double failure source) and all caseswhere position accuracy would exceed requirements have been detected.

4.3.3 Verification of theory

4.3.3.1 Delay for algorithm 4 detection compared to algorithm 3

Comparing the different times for detection of the individual algorithms, it can be noticed that the onefor algorithm 4 is delayed compared to the others. This can be explained going into the theory of thedetection process for algorithm 3 and 4. Everything relies upon the calculation of the threshold in thefailure detection process. If this computation is based on a Constant False Alarm Rate assumption(CFAR algorithm) this leads to a smaller threshold than if it is based on a Constant Probability OfMissed Detection (CPOD algorithm). Indeed, requirements for False Alarm Rate (PFA = 0.002 perhour) are more stringent than those for Missed Detection (PMD = 0.001).

This explains why algorithm 4 detects a failure later than algorithm 3. For further information see [14].

4.3.3.2 Algorithm 6

Algorithm 6 has a peculiarity compared to the other algorithms described in this report. It is the onlyone that computes RAIM results over an averaging interval of time. This interval is a function of theTime-to-Alarm required for each phase of flight. The following table contains the different values of theTime-to-Alarm, and the duration of the corresponding averaging interval.


10

Phase of Flight (Phase_Id) Time to Alarm [s] Averaging interval [s]Departure (2) 15 7En Route (3) 15 7Terminal (4) 15 7

Initial (5) 10 5NPA (6) 10 5

Table 3: Time to Alarm and Averaging interval for RAIM MOPS algorithm

Unfortunately, the observed values for the averaging intervals did not correspond to the valuesindicated in the previous table. A look into the source code enabled the error to be located andsoftware update will be done in the future to remedy this problem.

4.3.3.3 Algorithm 7, 8 and 9

Following the concepts detailed in Chapter 3.5, the detection process for Algorithms 7, 8 and 9 shouldall commence at the same time.

This can be observed for algorithms 7 and 9. However, the detection of algorithm 8 is delayed. Figure 7displays this particular event:

Figure 7: Detection problem of algorithm 8 - FK 100090

In this particular case, detection occurs one sample later for algorithm 8 than for algorithms 7 and 9. Adetailed look into the source code may be required to identify the reason for this behaviour.


11

4.4 IDENTIFICATION RESULTS ANALYSIS

4.4.1 Verification of Theory

Following the theory described in Chapter 3, two issues have to be verified:1. For Algorithms 3, 4 and 9, the identification process has to be carried out every time a detection

alarm occurs. In fact, because it is based on optimisation methods, the identification process doesnot require more satellites than the detection process does.

2. Identification samples for Algorithm 7 should be a subset of identification samples for Algorithm 8.In fact, algorithm 8 excludes the first satellite which, when removed from the constellation, resultsin a subset of satellites with no failure detection. Algorithm 7 excludes it, provided it is the only onein this case.

These two points could be observed on both simulated and real flights.

Figures 8 and 9 show an example for each case. Each coloured diamond indicates that detectionoccured. A blue diamond means that identification occurred, and a red one that identification didn’toccur.

1. Figure 8 shows that, for the whole period where detection occurs using algorithms 3, 4, and 9,identification also occurs.

2

3

4

5

6

7

8

9

10

8200 8700 9200 9700 10200 10700 11200 11700 12200

sample id

Alg

o id

Det Occ +Id Occ

Det Occ +Id Not Occ

Figure 8: Automatic identification when detection occurs for algorithms 3, 4 and 9 - FK 100090


12

2. Figure 9 gives an example of the inclusion of the interval of identification for algorithm 7 into theone for 8. This could be verified in every occurrence of identification for the two algorithms.

2

3

4

5

6

7

8

9

10

8200 8220 8240 8260 8280 8300 8320

sample id

Alg

o id

Det Occ +Id Occ

Det Occ +Id Not Occ

Figure 9: identification interval for algorithm 7 and 8 - FK 100090

The analysis of Figure 9 also raises some questions about the behaviour of the identification processfor algorithms 5, 6, 7 and 8. Indeed, for each of these algorithms, identification generally occurs severalsamples after detection and lasts only for a short period of time. The reason for this can be found bylooking at the pseudorange residuals from which the decision variable is derived for both the detectionand identification processes. These are shown in Figure 10.

Figure 10: Pseudorange residuals behaviour in a single ramp error case - FK 100090


13

Figure 10 shows that a single ramp error on the pseudorange measurement has a consequence on allthe PseudoRange RESiduals (PRRes). This can be explained by a simple drawing as shown in Figure11.

Figure 11: Interrelation between Pseudorange residuals

Pseudorange Residuals are functions of the estimated position. As a consequence a single error inone Pseudorange measurement, that will modify the estimated position, will have a consequence onevery Pseudorange Residual.

In this particular case (Figure 10), the continuous variation of the pseudorange measurement error isreflected on every residual. Because of the increase of its pseudorange residual, SVN 2 is identifiedmore or less rapidly depending on the algorithm. But because this pseudorange residual is not the onlyone to increase, the computed decision variables for identification (on subset) are bound to exceed theallowed threshold. In that case, identification is no longer possible for algorithms 5, 6, 7 and 8.

4.4.2 Satellite identified

4.4.2.1 Definitions

• Wrong identification (Id Occ but Not OK): Identification is considered as wrong when the satellite(or the 2 satellites) on which an error had been introduced is not mentioned in the algorithmsoutputs.That means:

- Single failed satellite should be one of the two identified by algorithm 9- In case of a double failure, all the algorithms identifying only one satellite (every

implemented algorithm except algorithm 9) should choose the most erroneous one in theideal case, or at least choose one of the failure sources.

• Partial Identification (Id Occ but PI): This concerns only with algorithm 9 in the case of doublefailures. Partial identification occurs when only one of the erroneous satellites is part of theidentified satellites.

PRRes 1

PRRes 3

Estimated position(no error)

Wrong Estimated position(one single error)

PRRes 2

PRRes 2PRRes 3

PRRes 1

SVN 1SVN 2

SVN 3


14

4.4.2.2 Single erroneous satellite

In this particular case, every time identification occurs, the erroneous satellite is the one that isidentified. Algorithm 8 was the only exception, for unknown reasons. This will be investigated in thefuture.

Single error source on SVN 26 (see Appendix 1), between sample 1000 and 2000 is the only casewhere algorithm 8 makes a wrong identification. This happens at the beginning of the detection andidentification interval. In this case the algorithm number 8 identifies SVN 4 instead of SVN 26.

Figure 12: Wrong identification for algorithm 8 - FK 100090

Looking at the pseudorange residuals (Figure 13), no explanation can be seen for this strangebehaviour. Additional work on that subject may be conducted in the future.

-200

-150

-100

-50

0

50

100

150

200

250

300

350

1065 1070 1075 1080 1085 1090 1095 1100 1105 1110 1115 1120

sample_id

resi

dual

[m]

SVN 4SVN 6SVN 10SVN 17SVN 19SVN 24SVN 26SVN 27

Figure 13: Pseudorange residual behaviour in the case of a ramp error on a singlepseudorange- FK 100090


15

4.4.2.3 Two erroneous satellites

Algorithm 9 is an exception in the set of algorithms implemented in the DUAU system as it is designedto exclude 2 satellites whenever detection occurs. But reference documents insist on the fact that all ofthese algorithms were designed to cover only unique error source scenarios.This may explain why double identification does not always happen correctly.

4.4.2.3.1 Scenario One: two similar ramp errors, one moved forward by 50 seconds

0

20 0

40 0

60 0

80 0

100 0

120 0

3 90 0 4 100 43 00 450 0 47 00 490 0 5 10 0s am p le id

S V N 2 6S V N 1 0

Figure 14: two similar ramp errors - FK 100090

In this particular case, algorithms 3, 4 and 9 encountered some problems. In the following figure everydetection sample is drawn. Except for red diamonds, identification occurred in the expected way (blue),in a partial way (green) or in a wrong way (yellow).

2

3

4

5

6

7

8

9

10

3900 4100 4300 4500 4700 4900 5100

sample id

Alg

o id

Id O cc and O KId Not O ccId O cc but NO KId O cc,but PI

Figure 15: Identification problems in the case of two same ramp errors - FK 100090


16

Following the results of Figure 15 and additional information on wrong and partial identification:

− Algorithms 3 and 4: identify SVN 17 instead of SVN 26 (or 10) from 4227 to 4685identify SVN 27 instead of SVN 26 (or 10) from 4686 to 5000

− Algorithm 9: identifies SVN 26 and another wrong satellite from 4057 to 4209 (PI)identifies SVN 17 and 27 instead of SVN 26 and 10 from 4210 to 5000

The Algorithm 8 problem is not seen here, even if three wrong identifications (SVN 6, 10 and 10 forsamples 4027 to 4029) occur at the beginning of the detection interval.

The pseudorange residuals to be used in the computation of the decision variables for detection andidentification need to be analysed for a better understanding.

-6000

-4000

-2000

0

2000

4000

6000

8000

3900 4100 4300 4500 4700 4900 5100

sam p le_id

resi

dual

[m]

SVN 6SVN 10SVN 17SVN 19SVN 23SVN 24SVN 26SVN 27

Figure 16: Pseudorange residual behaviour in the case of two similar ramp errors - FK 100090

Here it can be verified that the identification process follows the evolution of pseudorange residuals.Until sample 4220 SVN 26 has the biggest pseudorange residual, and the other ones are very similarto each other. This explains the behaviour of algorithm 9 in this first interval, and the difficulty it has toidentify a second satellite. For the second interval, SVN 17 and 27 clearly have the biggest residuals(absolute values), which explains the identification choice for all 3 algorithms.

Concerning the other algorithms, it can be verified that a constant increase of pseudorange residualsleads to the identification threshold being exceeded for subsets of satellites that are not necessarily atthe origin of the failure (see paragraph 4.4.1). This explains why algorithms 5, 6, 7 and 8 identificationprocesses do not succeed in this scenario.


17

4.4.2.3.2 Scenario Two: two ramp errors, different rates, start offset by 50 seconds

0

1 0

2 0

3 0

4 0

5 0

6 0

7 0

8 0

2 43 00 24 320 2 43 40 24 36 0 2 43 80 24 40 0 24 420 2 44 40 24 46 0 2 44 80 24 50 0sam p le id

S V N 4S V N 3 0

Figure 17: two different ramp errors - FK 100090

For single identification algorithms (3, 4, 5, 6, 7 and 8) and in the ideal case, SVN 4 should be identifieduntil sample 24370, then SVN 30 until the error disappears on sample 24410, and finally SVN 4.

In this case, unlike the first scenario, identification is done correctly (with the exception of algorithm 8,see detected problem in chapter 4.4.2.2), as can be seen in Figure 18.

2

3

4

5

6

7

8

9

10

24300 24320 24340 24360 24380 24400 24420 24440 24460 24480 24500

sample id

Alg

o id Id Occ and OK

Id Not OccId Occ but NOK

Figure 18: Identification problems in the case of two different ramp errors - FK 100090

The justification for such a good behaviour can be verified by examining pseudorange residual valuesfor this interval of time (Figure 19)


18

Figure 19: Pseudorange residual behaviour in the case of two different ramp errors - FK 100090

SVNs 4, 30 and 4 can be successively identified because of their clearly biggest residuals.


19

5. RAIM RESULTS ON REAL FLIGHTS

5.1 AVAILABILITY RESULTS ANALYSIS

RAIM unavailability was detected only once when using real flight data. This was detected on the FK210015 by all algorithms except algorithms 6 and 8.This unreliability must be due to the fact that on this error sample the number of satellite tracked by thereceiver is equal to 6. Every receiver failure interval had been excluded from calculation except thiscase.

In the following figure, the availability of RAIM detection is shown for each RAIM algorithm. For eachdiamond RAIM results are given. Detection is said to be unreliable (or unavailable) in the red case, andreliable in the blue one.

Figure 20: RAIM Availability – FK 210015

The smoothing effect can explain why algorithm 6 does not detect the outage. Concerning algorithm 8,which is supposed to behave identically to 7 and 9, no explanation can be given.

5.2 DETECTION RESULTS ANALYSIS

Detection occurred for three different flights (FK 210018, 210019, and 210035) when the aircraft is onthe ground. Multipath problems were suspected and further investigations confirmed this to be thecase. This will be detailed in the following paragraph 5.3.

The list of confirmed points already considered for the simulated flight is as follows:

- Detection occurs during the same time intervals for all algorithms.- Sometimes, where the intervals are short, algorithms 6 and 4 do not have time to react to the

failure.- Algorithms 7, 8 and 9 detect failures at the same time as expected from the literature.


20

5.3 IDENTIFICATION RESULTS ANALYSIS

5.3.1 Verification of theory

This real flight illustrates, like the one with additional simulated errors, several principles expected fromthe theory:- Identification is made for each detection instant by algorithms 3 and 9 (algorithm 4 never detected

a failure on the real flights studied)- There is a discontinuity in identification for algorithms 5, 7 and 8. (Detection occurs for 6 at only

one sample and cannot be added to the list)

To illustrate these two points, FK 210019 can be used as an example. For this flight, algorithms detecta failure between samples 281 and 292 (Figure 21). Figure 22 presents the variations of thepseudorange residuals.

Figure 21: Detection and Identification occurrence - FK 210019

Figure 22: Pseudorange residuals behaviour - FK 210019


21

The residual variations explain the behaviour of the identification process for algorithms 5, 7 and 8, andconfirm the results of paragraph 4.4.1 on the unavailability of RAIM identification in the case of toomany large pseudorange residuals.Identification of SVN 31 can be explained with regard to the residual’s behaviour.

5.3.2 Multipath problems

Figure 22 shows the irregular behaviour of the pseudorange residuals, characteristic of multipathproblems, which could be the cause of the failure.This assumption can be verified by looking at the signal-to-noise (CNO) ratio shown in Figure 23.

29

31

33

35

37

39

41

43

45

47

49

51

53

55

270 275 280 285 290 295 300sample_id

CN

O

SVN 31SVN 7SVN 27SCN 4SVN 20SVN 8SVN 16SVN 2SVN 11

Figure 23: CNO's - FK 210019

Figure 23 shows that the quality signal for SVN 31 is much worse and irregular than the ones for theother satellites. This is characteristic of multipath problems.

Every failure detection occurrence for the 3 flights (FK 210018, 210019 and 210035) took place whenthe aircraft was on the ground, and shows the same properties observed as on FK 210019. Multipath,which is more frequent on the ground due to the presence of building and the earth, is most probablythe common reason for all of these events.


22

6. CONCLUSIONS

This report presents the results of the RAIM Validation study of the 7 algorithms implemented in theSAPPHIRE system. This study results in the following conclusions:

− Algorithms 3, 4, 5, 7 and 9 follow the principles described in the original references. (Late detectionfor algorithm 4 compared to 3, synchronised failure detection for 7 and 9, identification on eachsample when algorithms 3, 4 and 9 detect a failure, etc…)

− Algorithm 6 had some problems concerning the frequency of output data (depending on phase offlight). A look into the source code identified a need for updating the software.

− Several problems were raised for Algorithm 8. Some more investigations are needed into thesource code to understand why detection is not perfectly synchronised with that of algorithms 7 and9, and why identification is sometimes wrong. This point is still open.

Generally speaking, all the algorithms behaved in a similar way and detected every case whereposition error exceeded accuracy requirements. Early detection is a common property to everyalgorithm, which raises the question of whether or not to consider such early detection as false alarms.

Identification tests show that they are reliable in case of unique error sources, according to the wayalgorithms were designed.

From these results it can be concluded that the RAIM algorithms implemented in the SAPPHIREDUAU System are valid, except for algorithm 6 whose implementation has to be corrected andalgorithm 8 where problems will be investigated in future work.


23

7. ACKNOWLEDGEMENTS

On behalf of EEC, we would like to thank the University of Braunsweig, and more preciselyCarsten Butzmuehlen, for his support during this study.


24

8. REFERENCES

[1] M. Sturza, A. Brown: ‘Integrated GPS/GLONASS for Reliable Receiver Autonomous IntegrityMonitoring (RAIM)’, ION Annual Meeting, June 1990

[2] M. Sturza, A. Brown: ‘The Effect of Geometry on Integrity Monitoring Performance’, ION AnnualMeeting, June 1990

[3] M. Sturza, A. Brown: ‘Comparison of Fixed and Variable Threshold RAIM Algorithms’, ION GPS-90, September 1990

[4] M. Sturza: ‘Navigation System Integrity Monitoring Using Redundant Measurements’, Journal ofthe Institute of Navigation, Vol. 53, No 4, Winter 1988-89

[5] M. Sturza: ‘Fault Detecion and Isolation (FDI) Techniques for Guidance and Control Systems’,AGARDOGRAPH No 314, AGARD Advisory Group, NATO

[6] M. Brenner: ‘Implementation of a RAIM Monitor in a GPS Receiver and an Integrated GPS/IRS’,ION GPS 90, September 1990

[7] ‘Minimum Operational Performance Standards (MOPS) for Airbone Supplemental NavigationUsing the Global Positioning System (GPS)’, RTCA DO-208, July 1991, including Change I andChange II

[8] B.W. Parkinson, P. Axelrad: ‘Autonomous GPS Integrity Monitoring Using the PseudorangeResidual’, NAVIGATION, Journal of the Institute of Navigation, Vol 35, No 2, Summer 1988

[9] R.G. Brown, G.Y. Chin, J.H. Kraemer: ‘GPS-RAIM – Screening out bad Geometries under worst-case Bias Conditions’, ION Annual Meeting 1992, June 1992

[10] R.G. Brown, G.Y. Chin, J.H. Kraemer: ‘Update on GPS Integrity Requirements of the RTCAMOPS’, ION National Technical Meeting 1996, January 1996

[11] R.G. Brown, G.Y. Chin, J.H. Kraemer: ‘A partial Identification RAIM Algorithm for GPS SoleMeans Navigation’, ION GPS 1994, September 1994

[12] R.G. Brown, G.Y. Chin, J.H. Kraemer: ‘Comparison of FED and FDI RAIM Algorithms for GPS’,ION National Technical Meeting 1994, January 1994

[13] R.G. Brown, G.Y. Chin, J.H. Kraemer: ‘RAIM Availability for CAT IIIB Operations’, ION AnnualMeeting 1995, June 1995

[14] Dr W. Lechner, S. Vieweg: ‘Integrity Monitoring Strategies for GNSS Applications’, ION AnnualMeeting 1994, January 1994

[15] EEC Technical Note “SAPPHIRE RAIM Validation”, March 2001.


25

APPENDIX 1 : SIMULATED A340 ERROR FLIGHT - ERROR LIST

Error type Start End Value Freq. Chn/SVN RunPeak 500 510 5m 2 2/24 1Peak 520 530 10m 2 2/24 2Peak 540 550 20m 2 2/24 3 Peak 560 570 30m 2 2/24 4Peak 580 590 40m 2 2/24 5Peak 600 610 50m 2 2/24 6Peak 620 630 75m 2 2/24 7Peak 640 650 100m 2 2/24 8Peak 660 670 150m 2 2/24 9Peak 680 690 200m 2 2/24 10Peak 700 710 500m 2 2/24 11

Peak 750 760 10m 2 0/26 1Peak 770 780 25m 2 0/26 2 Peak 790 800 50m 2 0/26 3Peak 810 820 100m 2 0/26 4Peak 830 840 200m 2 0/26 5Peak 750 760 10m 2 1/6 1Peak 770 780 25m 2 1/6 2 Peak 790 800 50m 2 1/6 3Peak 810 820 100m 2 1/6 4Peak 830 840 200m 2 1/6 5

Ramp 1000 2000 5m/s 0/26 6 Ramp 2500 3000 20m/s 3/10 1 Ramp 4000 5000 10m/s 0/26 7Ramp 4050 5050 10m/s 3/10 2Ramp 8100 12100 2m/s 1/2 6

Step 15500 15530 10m 3/7 3Step 16000 16030 20m 3/7 4Step 16500 16530 30m 3/7 5Step 17000 17030 50m 3/7 6Step 17500 17530 75m 3/7 7Step 18000 18030 100m 3/7 8Step 18500 18530 150m 3/7 9Step 19000 19030 200m 3/7 10Step 19500 19530 500m 3/7 11

Gauss Markov 20000 20985 100m/0.75m/s 150s 4/1 1Gauss Markov 21428 23500 110m/0.75m/s 150s 4/1 2

Ramp 23650 24050 5m/s 4/1 3

Step 24225 24230 10m 1/30 7Step 24235 24240 20m 1/30 8Step 24245 24250 30m 1/30 9Step 24255 24260 50m 1/30 10Step 24265 24270 100m 0/4 8Step 24275 24280 200m 0/4 9Step 24285 24290 500m 0/4 10

Ramp 24330 24480 5m/s 0/4 11Ramp 24350 24410 10m/s 1/30 11Error type Start End Value Freq. Chn/SVN Run

EUROCONTROL€¦ · SAPPHIRE RAIM ALGORITHMS VALIDATION EEC Note No.16/01 Project GNS-Z-E2 Issued:...

Documents

Transcript of EUROCONTROL€¦ · SAPPHIRE RAIM ALGORITHMS VALIDATION EEC Note No.16/01 Project GNS-Z-E2 Issued:...