PERFORMANCE EVALUATION OF SMARTPHONE GNSS …...antenna, and (2) active external helical and (2)...

PresentedatRoyalInstituteofNavigationInternationalNavigationConference,Brighton,27-30November2017 1

PERFORMANCE EVALUATION OF SMARTPHONE GNSS MEASUREMENTS WITH DIFFERENT ANTENNA CONFIGURATIONS

Ranjeeth Siddakatte, Ali Broumandan and Gérard Lachapelle

PLAN Group, Department of Geomatics Engineering, Schulich School of Engineering, University of Calgary, 2500 University Drive N. W. Calgary, AB T2N 1N4 Canada

E-Mail: [email protected] (A.B.)

ABSTRACT With the release of the Android N operating system in 2016, logging raw GNSS measurements became possible with modern smartphones. In this research, a recent Android smartphone, the Huawei Mate9, is used to log GNSS raw measurements with different internal/external antenna configurations under various conditions. A method to use and evaluate external antenna performance without smartphone hardware modification is introduced. The code noise is measured using a series of hardware simulator test. Performance of raw measurements is evaluated using the above antenna configurations. Multipath performance, carrier-to-noise-ratio (C/N0) quality, satellite availability and position data quality are evaluated for the case of a dense urban environment. Results show that performance can be significantly enhanced by incorporating external single/dual helical antenna connection without having to modify the smartphone hardware.

KEYWORDS: Smartphone, GNSS, raw measurements, Android, performance, measurement quality, Huawei Mate9, iPhone

INTRODUCTION With the advent of smartphone technologies and enhancements in radionavigation sensor capabilities, there is an increasing availability of phone location and measurement information is available. Performance evaluation of smartphone GNSS measurements is gaining attention. For instance, Pesyna et al (2015) used a smartphone antenna to feed GNSS signals into a software-defined receiver. Their analysis revealed that extremely poor multipath suppression of phone antennas is a primary impediment to obtain high level accuracy. Kirkko-Jaakkola et al (2015) conducted a raw measurement analysis of a Nokia smartphone using custom firmware. The authors identified pseudorange measurement noise of tens of metres and carrier-phase observations corrupted by multiple outliers.

In 2016, Google announced the availability of raw GNSS measurements in phones operating on Android N. This is beneficial as pseudorange, Doppler and carrier-phase measurements can be used to derive more accurate positions or at least to carry out more in-depth performance analyses. Banville & Van Diggelen (2016) analysed measurements from several smartphones using a custom-built Android OS. They


observed that noisy pseudorange observations could only provide metre-level accuracies.

Antenna quality is of major importance to enhance signal reception and reduce measurement degradation. Antenna diversity in smartphones is one possible solution to enhance signal quality through diversity combining.

This research investigates the performance of smartphone measurement and location data under various scenarios using internal and different external antenna configurations with the Huawei Mate9 phone, which is equipped with a Broadcom BCM4774 GNSS chipset. More specifically, the goal is to analyse GPS L1 data under different environments without having to physically connect external antennas to phone hardware, thereby increasing ease of applicability. The phone is subjected to hardware simulation tests, static and vehicular kinematics tests in open-sky and urban canyon environments. Different antenna configurations are tested, namely (1) the internal antenna, and (2) active external helical and (2) pinwheel antennas. In addition, performance evaluation with external antenna spatial diversity is reported. In the case of the external antenna use, the radio signals received by that antenna are beamed to the phone placed inside a metallic box as described in the sequel to ensure that no strong signals are received through the internal antenna. The following metrics are evaluated for each of the antenna configurations, namely (a) Carrier-to-Noise density ratio (C/N0) statistics and the gain obtained using antenna diversity, (b) measurement availability and continuity, (c) multipath performance, (d) accuracy of position and velocity estimation, and (e) location data from the internal navigation engine of the phone. The smartphone was not connected to the mobile network for any of the tests reported in the sequel.

DATA COLLECTION I: SIMULATOR TEST

A first data set was collected to evaluate code noise characteristics as a function of input C/N0. GNSS signals were fed to the phone using a Spirent hardware simulator and a variable attenuator as shown in Figure 1. A static simulation with no multipath or atmospheric errors was performed for 15 minutes. Controlled attenuation was introduced at regular intervals as shown in Figure 2. The range of input signal strength for all PRNs was 45 to 23 dB-Hz. The phone was made to receive the external signals through its own internal antenna through the following configuration. First, the phone was placed inside a metallic box so as to isolate it from external radiation. Next, the external signals generated by the simulator were connected through a co-axial connector fixed to one of the sides of the metallic box. An exposed cable acting as a monopole antenna was attached to a connector feeding inside the box to propagate the signals inside the box as shown in Figure 1. These propagated signals were then received wirelesslyby the internal antenna of the phone inside the box; hence no phone


hardware modification was needed. The metal box wall thickness is slightly below 1 mm); 3-cm standard foam covered its internal walls. As a consequence, the influence of external radiation, including that from attenuated GNSS signals, was reduced by up to 20 dB. In addition, the simulation tests were conducted in an indoor lab environment where external GNSS signals hardly penetrate. A major advantage of placing the phone inside a metallic box was also to protect nearby GNSS receivers from an inadvertent spoofing attack during the tests.

Figure 1: Simulation test set-up to verify code noise of Huawei Mate9 smartphone

Figure 2: C/N0 as measured by the smartphone - step attenuation observed


The phone measurements were logged using the publically available android application called “GNSS Logger” developed by Google Inc (Banville & Van Diggelen 2016). It logs various measurements including C/N0, pseudoranges and accumulated delta-ranges. The app logs the measurements in a text file, read using the open source MATLAB library kit called “GPS Application Toolkit”, also developed by Google Inc. The measured pseudoranges were used to estimate the code noise as follows: (a) fix the 3D position states using epoch-by-epoch least-squares (LS), (b) estimate time series of the 4th state in the LS equations including clock bias and code noise, (c) smooth the 4th state to obtain the clock bias variations, (d) remove clock and true ranges from all pseudoranges to obtain time series of code noise for each PRN, and lastly, (e) obtain the standard deviation of code noise for each PRN for different values of C/N0. The approximate code noise values as a function of C/N0 for different PRNs are shown in Figure 3. The phone is observed to have about 2-3 m code noise at 45-40 dB-Hz, and about 15-20 m at 23 dB-Hz. Loss of tracking occurred below 20 dB-Hz. Interestingly, the noise is still below 5 m at 30 dB-Hz.

Figure 3: C/N0 versus code noise for Huawei Mate9 smartphone

DATA COLLECTION II: OPEN-SKY TEST WITH INTERNAL ANTENNA

In order to investigate positioning performance, open-sky tests were conducted at a static location. The phone was placed on a surveyed pillar. GPS L1 signals available in the measurement log. Figure 4 shows the horizontal scatter plot and Figure 5, the velocity states. The Position and velocity states were computed by the LS estimator. The weights assigned to the measurements were inversely proportional to their standard deviations as provided in the measurement data files.The East, North and Up (ENU) position RMSE values were 6, 14 and 33 m, respectively. Likewise, the ENU


velocity RMSE values were 0.08, 0.14 and 0.33 m/s, respectively. There was a loss of a satellite in the north direction that led to slightly poorer geometry, the East and North DOP being 0.8 And 1.3 respectively; this, combined with multipathd caused higher position spread in northing as seen in Figure 4.

Figure 4: Static position results - horizontal scatter plot

Figure 5: Static position results - 3D velocities


DATA COLLECTION III: OPEN SKY TEST WITH INTERNAL AND TWO EXTERNAL ANTENNAS

The third data set was collected in an open-sky location over a period of 12 minutes with the antenna configurations described earlier. In the case of re-propagating signals with an external antenna, the smartphone was placed in the metallic box. The monopole antenna that beams signals to the phone inside the box was successively connected to two types of external antennas, namely the helical and the pinwheel antenna. The three test cases are shown in Figure 6 as follows: (a) phone placed on top of the box and used its own antenna, (b) phone placed inside the box and received signals from the external helical antenna, (c) phone was placed inside the box and received signals from the external NovAtel pinwheel antenna. In Case 2 and 3, signals from the external antennas were used; the phone internal antenna could still theoretically receive signals however due to the higher attenuation of signals directly propagated from outside the box, this was inconsequential as the stronger signals propagated through the monopole antenna(s) inside the box would have been used by the phone.

Figure 6: Open-sky tests with different antenna configurations: internal, antenna and pinwheel antennas


Figure 7: C/N0 values as measured by the smartphone with different antenna configurations

C/N0 measurements from the phone for different antenna configurations are shown in Figure 7. Mean C/N0 values for the pinwheel antenna case are between 44 to 48 dB-Hz for most PRNs. For the helical antenna case, the corresponding mean is 42 to 46 dB-Hz and for the internal antenna case, it is 27 to 37 dB-Hz. The pseudorange measurements logged in each case were used to estimate LS position solutions with the position accuracy results shown in Table 1. The horizontal position estimates obtained by connecting an external pinwheel antenna are somewhat better than the ones obtained using the phone internal antenna.


Table 1 LSQ Positioning performance (m)

Pinwheel Error Metric E N U

RMS 4.2 3.6 19.3 Mean 0.8 1.7 -16.5

Helical RMS 5.7 5.1 19.1 Mean -2.0 1.1 -14.1

Smartphone RMS 6.7 4.5 16.5 Mean 1.1 -0.2 -12.3

DATA COLLECTION IV: URBAN CANYON TESTS

The fourth data set was collected in an urban canyon environment, namely downtown Calgalry, to study multipath performance, satellite visibility, measurement quality and positioning performance. The tests were conducted using a car and the phone measurements were collected successively using the following configurations:

(A) internal antenna and phone on dashboard (B) internal antenna only with phone placed inside the metallic box near the dashboard (C) internal antenna and phone on car roof (D) external single helical antenna connected to phone (E) external dual helical antennas connected to phone.

Configuration B was selected to measure the signal attenuation by placing the smartphone inside the metallic box and the effect on its capability to estimate positions. In configuration D and E, the external antennas were placed on the roof vertically as shown in Figure 8 (top). The co-axial cables from these antennas were connected to the two monopole antennas placed inside the box that beamed electric signals to the phone internal antenna, as shown in Figure 8 (bottom). Further, for configuration D, only one of the helical antennas was powered, and configuration E was a case of spatial diversity reception where the two helical antennas were separated by a distance of about 1 m. The diverse signals received by the two antennas combined spatially at the RF level in the metallic box, and the combined signal was received by the phone internal antenna, allowing assessment of diversity combining. Further, the phone internal navigation engine performance (position and velocity states estimated by the phone internal software) were investigated for different antenna configurations. For this purpose, internal location data was logged using the GNSSLogger app.


Only Android phones are currently able to provide raw GNSS measurements. Nonetheless, it is worthwhile to compare location data with another phone, namely an iPhone 5SE in this case; this phone was placed at same locations as those of the Huawei Mate9 (configurations A to E) and location data outputs were recorded using the iPhone SensorLog Application. As in the case of the Huawei phone, the iPhone 5S was not connected to the mobile network during the tests. Figure 14 shows the test location and routing in downtown Calgary. The true routing was estimated with an accuracy of 1 m using a tightly coupled GNSS/IMU integration of measurements from a NovAtel SPAN-LCITM system. Several high-raise buildings of 50 to 250 m are located along the trajectory. Five loops of 10 minutes eachwere made, one for each configurations. Figure 9 shows the sky-plot of satellites that were tracked.

Figure 8: Test set-up used in downtown data collection (top), configuration for external dual

helical antenna case (bottom)


Figure 9: Downtown test sky-plot - start and end location of satellites

Signal quality: C/N0 values of all PRNs during the first 200 s of each of the five data sets and configurations is now analysed. This represents approximately the same geometric location of the vehicle in each of the five data sets. The mean and standard deviation C/N0 values for the five configurations are given in Table 2. Comparative plots are shown in Figure 10. Red squares represent the mean values and vertical blue lines show the standard deviations. Predictably, the lowest C/N0 values occurred when the phone used its own internal antenna inside the metallic box in which case values as low as 14 dB-Hz were recorded.The overall C/N0 gain achieved in moving the phone from the box to the dashboard was 10 dB on average and reached 15 dB. The mean C/N0 gain achieved in going form dashboard placement to roof placement is only 1 dB, with occasional enhancements of 2 to 4 dB. By going from internal antenna reception with phone on dashboard to external single helical antenna on the rooftop, one can expect about 6 dB improvement. For the antenna diversity case (external dual helical antenna), the mean overall gain is about 5.5 dB, while satellite visibility and continuity is slightly better than the single helical antenna case. This will be discussed in later sections.


Table 2: Downtown tests – C/N0 means and standard deviations in dB-Hz for five configurations

PRN A

Internal, dashboard

B Internal, box

C Internal, roof

D External,

single helical

E External,

dual helical Mean Std Mean Std Mean Std Mean Std Mean Std

7 NA NA 14.0 5.5 20.4 9.0 NA NA 30.8 7.9 19 26.6 6.7 16.2 3.8 27.1 4.9 NA NA 24.0 6.6 13 31.7 5.1 20.1 3.8 31.3 4.7 41.8 3.9 39.2 4.6 1 27.9 4.3 25.4 3.3 30.2 5.4 40.2 2.9 30.2 7.1

15 28.0 6.1 18.7 6.2 38.5 2.7 35.5 9.9 36.1 8.6 11 28.6 9.9 20.9 5.5 28.7 5.9 36.2 7.8 40.1 2.5 17 35.0 5.3 25.3 4.0 37.6 6.9 38.1 7.2 36.9 9.1 30 37.3 7.0 21.3 3.9 33.9 5.6 37.8 6.8 38.1 7.3 28 36.7 3.7 20.4 4.5 31.5 3.4 41.9 1.0 38.2 4.4

Figure 10: Downtown tests - Observed C/N0 for five different PRNs and antenna configurations. Red squares show means and blue lines show standard deviations.


Multipath performance: The accumulated delta range (estimated by carrier measurements) and pseudorange (PR) measurements, both logged by the android app, were utilized to measure PR multipath errors. Multipath errors were estimated by computing the differences between delta PR and accumulated delta ranges, which is equivalent to computing multipath from the code-minus-carrier measurements. Only epochs where the accumulated delta ranges were valid are used {as indicated in the log file by a flag called ‘GPS_ADR_STATE = 1’ during which an accumulated delta range measurement is computed using a phase lock loop (PLL)}. The phone provided continuous carrier phase measurements (with PLL) for about 5 minutes when tested with the external pinwheel antenna in open-sky conditions. However, with internal the antenna configuration, carrier measurements with PLL were intermittent. In the downtown tests, the phone was in frequency lock loop mode 48% of time (GPS_ADR_STATE = 4), and in PLL mode about 27% of the time (GPS_ADR_STATE = 1); for the remaining time it was out of carrier tracking loop. The reason for intermittent carrier phase measurements is “duty cycling” operations in smartphones. The process of duty cycling powers off the GNSS receiver at given intervals (shorter than 1 s) to save power. Since the receiver is powered off, continuous carrier phase tracking is not possible (Banville 2017). The delta PR is nothing but the epoch-by-epoch difference between successive PR measurements beyond the first one and the first one. Figure 11 and Figure 12 show ‘delta PR-minus-accumulated delta range’ measurements for configuration A (dashboard) and C (roof top). In both cases, multipath errors experienced by the phone are of the order of 10 m and, in some cases, up to 150 m. Note that the inherent code noise for the phone (as observed in the simulator tests) is 2 to 15 m depending on the C/N0. Given the high multipath in pseudorange measurements, it is speculated that the phone is operating its code discriminator with 1 chip spacing between the early and prompt discriminator and/or mostly reflected signals are tracked.

Figure 11: Approximate multipath values, code-minus-carrier when phone placed on dashboard


Figure 12: Approximate multipath values, code-minus-carrier when phone placed on car roof

Satellite availability and continuity: The horizontal dilution of precision (HDOP) derived from the LS engine is shown for all five test runs in Figure 13, together with the number of satellites whose data is decoded by the phone measurement engine. As expected, the HDOP and satellite availability are poorest for configuration B when the phone was placed inside a metallic box and the phone antenna used. The number of satellites drops to less than 4 as the vehicle enters the dense downtown area (after 200 s). The car roof case of configuration C, has better availability than the dashboard case A. The external dual helical antenna configuration E was occasionally able to decode more satellites than the single helical antenna case D. This can also be verified by observing the time series of HDOP values, which are lowest, hence best, for the diversity antenna case (E), followed by single helical antenna (D), dashboard (A), car roof and finally the box case (B).

LS position and velocity estimation: Position and velocity LS estimates were derived from the pseudorange measurements and their RMSE values for the entire route are shown in Table 3 and Table 4 respectively. The box case configuration B was not able to provide LS solutions for most of the route, hence its value is not shown. The lowest errors occurred with external antenna configurations (D and E). The trajectory is in the east-west directioin (Figure 14) which is the reason why northing results are poorer.

Table 3: Least Squares RMS position results for different antenna configurations

Case East (m) North (m) Up (m)

Internal, dashboard 50.5 61.0 133.0 Internal, box NA NA NA Internal, roof 44.0 66.0 136.0

External, single helical 19.5 36.0 53.0 External, dual helical 17.5 55.0 50.0


Table 4: Least Squares RMS velocity performance for different antenna configurations

Case East m/s

North m/s

Up m/s

Internal, dashboard 3.3 3.9 6.3 Internal, box NA NA NA Internal, roof 4.7 4.6 7.6

External, single helical 0.9 1.6 2.2 External, dual helical 0.8 1.7 1.4

Figure 13: Downtown tests - HDOP and satellite visibility


Smartphones internal navigation outputs: The internal navigation outputs from two smartphones, namely the Huawei Mate9 and iPhone 5SE {both off the cellular network}, are now compared for each of the five test runs with different antennas. Table 5 shows the position state RMS error values and Table 5: Smartphone internal position RMS (m) results for different antenna configurations

Configuration iPhone 5SE Huawei Mate9 East North Up East North Up

Internal, dashboard 9.0 5.6 24.0 14.3 21.2 19.0 Internal, box 26.0 14.7 21.5 12.8 23.4 20.5 Internal, roof 6.8 3.8 22.0 11.5 8.8 20.0

External single helical 5.9 2.0 14.8 5.2 8.8 18.0 External dual helical 5.5 1.3 17.1 4.0 8.3 18.0

Table 6: Smartphone internal forward RMS (m/s) velocity results for different antenna configurations

Case iPhone 5SE Huawei Mate9 Internal, dashboard 0.85 1.8

Internal, box 3.4 3.8 Internal, roof 0.53 1.4

External single helical 0.37 1.2 External dual helical 0.40 0.85

CONCLUSIONS

A performance evaluation of GNSS measurements and positions using a modern Android smartphone was conducted under various test environments and with different antenna configurations. The simulation tests revealed that the Huawei Mate9 phone used herein has code noise ranging between 2 and 15 m, depending on C/N0 values. The smartphone was also tested with different antenna configurations without having to modify the internal hardware of the phone. The qualities of C/N0, pseudorange and position data show that the use of external single/dual helical antennas improves navigation performance in urban canyons. The same was observed using an iPhone 5SE internal position solution (No access to C/N0 and pseudorange measurements was possible with this unit).

shows the corresponding forward velocity RMS error values. Both phone internal software show an over 50% reduction in position and forward velocity errors when using the single or dual helical antenna configurations. The diversity reception showed a slight but not significant improvement over the single helical antenna case. The estimated trajectories for the five antenna configurations are shown in Figure 14.


Table 5: Smartphone internal position RMS (m) results for different antenna configurations

Configuration iPhone 5SE Huawei Mate9 East North Up East North Up

Internal, dashboard 9.0 5.6 24.0 14.3 21.2 19.0 Internal, box 26.0 14.7 21.5 12.8 23.4 20.5 Internal, roof 6.8 3.8 22.0 11.5 8.8 20.0

External single helical 5.9 2.0 14.8 5.2 8.8 18.0 External dual helical 5.5 1.3 17.1 4.0 8.3 18.0

Table 6: Smartphone internal forward RMS (m/s) velocity results for different antenna configurations

Case iPhone 5SE Huawei Mate9 Internal, dashboard 0.85 1.8

Internal, box 3.4 3.8 Internal, roof 0.53 1.4

External single helical 0.37 1.2 External dual helical 0.40 0.85

CONCLUSIONS

A performance evaluation of GNSS measurements and positions using a modern Android smartphone was conducted under various test environments and with different antenna configurations. The simulation tests revealed that the Huawei Mate9 phone used herein has code noise ranging between 2 and 15 m, depending on C/N0 values. The smartphone was also tested with different antenna configurations without having to modify the internal hardware of the phone. The qualities of C/N0, pseudorange and position data show that the use of external single/dual helical antennas improves navigation performance in urban canyons. The same was observed using an iPhone 5SE internal position solution (No access to C/N0 and pseudorange measurements was possible with this unit).


Figure 14: Huawei Mate9 position performance: internal phone antenna on dashboard (red), internal phone antenna on car roof (pink), internal phone antenna in a box (brown), external single helical antenna(blue), external dual helical antenna (purple), reference routing (green) [Not all trajectories seen due to overlapping]

REFERENCES

1. Pesyna, K. M., R. W. Heath and T. E Humpreys (2015) “Accuracy in the Palm of Your Hand: Centimeter Positioning with a Smartphone-Quality GNSS Antenna” in GPS World, Vol. 26, No. 2, February 2015, pp. 16–18 and 27–31.

2. Kirkko-Jaakkola, M., S. Söderholm, S. Honkala, H. Koivula, S. Nyberg and H. Kuusniemi (2015) “Low-Cost Precise Positioning Using a National GNSS Network” in Proceedings of ION GNSS+ 2015, Tampa, Florida, Sept. 14–18, 2015, pp. 2570–2577.

3. Banville, S. and F. Van Diggelen “Precise Positioning Using Raw GPS Measurements from Android Smartphones” in GPS World, Vol. 27, No. 11, November 2016, pp. 43–48.

4. Banville, S. (2017) Personal Communication, 12 October 2017

PERFORMANCE EVALUATION OF SMARTPHONE GNSS …...antenna, and (2) active external helical and (2)...

Documents

Transcript of PERFORMANCE EVALUATION OF SMARTPHONE GNSS …...antenna, and (2) active external helical and (2)...