Multi-Link Iridium Satellite Data Communication System

University of Kansas

Multi-Link Iridium Satellite Data Communication System

Overview, Performance and Reliability from Summer 2003

NGRIP, Greenland Field Experiments

June 23-July 17, 2003Abdul Jabbar Mohammad, Graduate Research AssistantDr.Victor Frost, Dan F. Servey Distinguished Professor

(September 24, 2003)

University of Kansas 2

Motivation Polar Radar for Ice Sheet Measurements (PRISM)

The communication requirements of PRISM field experiments in Greenland

and Antarctica include Data telemetry from the field to the University

Access to University and web resources from field

Public outreach to increase the interest of student community (K-12) in scientific

research and enable the science community to virtually participate in polar

expeditions

Generic data communication for Remote field research Mainstream communication system for polar science expeditions, field camps in

Arctic/Antarctic and other research purposes

Government and security use


Introduction – Commercial Satellite Systems

Polar regions do not have conventional communication facilities (dial-up, DSL, Cable Modem, etc) and are not serviced by most of the major broadband satellite systems.

Intelsat

Inmarsat

Globalstar


Introduction – Special Purpose Satellite Systems

NASA satellites like ATS3, LES9,

GOES, TDRS 1,and MARISAT2

provide broadband access to Polar

Regions

Geo-synchronous, they have a limited

visibility window at Poles – typically

10-13 hrs/day.

High satellite altitude and low elevation

angles (1-20) result in extremely large

field equipment.

May not be readily available

20 m diameter Marisat and GOES antenna at South Pole


Introduction - Iridium Satellite System

The only satellite system with true pole-to-pole coverage

66 low earth orbiting (LEO) satellites with 14 spares

It has onboard satellite switching technology which allows it to service large

areas with fewer gateways

Since it was originally designed as a voice only system, it provides a low

data rate of 2.4Kbps

Not practical to be used as a main stream/ life-line communication system


Introduction – Problem Statement

Problem Statement

A reliable, lightweight, portable and easily scalable data/Internet

access system providing true Polar coverage.

Solution

Implement a multi-link point-to-point Iridium communication system

to combine multiple satellite links to obtain a single logical channel

of aggregate bandwidth.


Background - Iridium

Satellite Type LEO

Satellite altitude 780 km

Minimum elevation angle 8.20

Average satellite view time 9-10 minutes

Access scheme FDMA and TDMA

Maximum number of located users 80 users in a radius of 318 km

Theoretical throughput 2.4 – 3.45 Kbps

Type of data services Iridium-to-Iridium, Iridium-to-PSTN


Background - Inverse Multiplexing

App 3

App 2

App 1M

uxHigh Bandwidth Link

Multiplexing

App 1

Inv-Mux

Low Bandwidth Links

Traditional Multiplexing - Data from a multiple applications/users sent over a single high bandwidth link.

Inverse Multiplexing - Data from a single application is fragmented and or distributed over multiple low bandwidth links.

Increases the available bandwidth per application significantly

Multi-link point-to-point protocol (MLPPP), an extension to the PPP is a packet based inverse multiplexing solution

Overhead of 12 bytes

Fragmentation of network layer protocol data units (PDU’s) into smaller segments depending upon the PDU size, link MTUs and the number of available links.

Inverse

multiplexing


Background - Multi-link point-to-point protocol

Layer 2 protocol that implements inverse multiplexing on a packet by packet basis

IP packets are encapsulated in to PPP frames with segment numbers – 12 byte overhead

Packet fragmentation depends on the number of available links and their capacity, packet size and MTU size

IP Packet

Modem i Modem 1 Modem 2 Modem 3 Modem 4

Not Fragmented Fragmented

IP Packet

IP PacketSH SH SH SH SHPDF PDF PDF PDF

SH-Segment header; PDF- packet data fragment


Multi-channel Iridium System – Design

The design requirements of the system are as follows.

The multi-channel implementation should maximize the throughput.

To support real-time interactions, the system should minimize the end-to-end delay.

The overall system should be reliable and have autonomous operation so as to handle

call drops and system/power failures in remote field deployment.


Multi-channel Iridium System – Protocol Stack

Application

HTTP, FTP, SSH

TCP

IP

PPP/MLPPP

Physical Modems

Application

HTTP, FTP, SSH

TCP

IP

PPP/MLPPP

Physical Modems

Remote System Local System

point-to-point satellite links


Multi-channel Iridium System – Network Architecture

NGRIP Camp, Greenland

ITTC Network, University of Kansas

World Wide Web

User 2

User 3

User 1

ppp0 eth0

PPP Server

ppp0 eth0

PPP Client

P-T-P Satellite link

ITTC Default Router

(Default gateway)(Default gateway)Guser 4

Guser 3

Guser 2G`user 1

Camp WI-FI

100 Mbps Ethernet

100 Mbps Ethernet


4-Channel Iridium System Implementation

Iridium Gateway

PSTN

US

B-S

ER

IAL

I. Modem 3

I. Modem 4

I. Modem 2

I. Modem 1 Antenna G

ridM

ulti-port P

CI card

Remote System

PPP client

Local System

PPP Server

Modem Pool

Remote Subsystem

Local Subsystem


4-Channel System – Implemented at KU


4-Channel System – Software Overview

Linux based system

Open source PPP package

Custom software developed at University of Kansas

Chat scripts for Iridium modems

PPP script to tune link parameters for Iridium satellite system

Client-Server configuration

Autonomous operation-connection setup, user authentication, detecting

failures, reconnections, handling power failures/system resets, generating

status information (text)


4-Channel System – Modem Flow Control

Check if any modem is

alive

Monitor Status

OKFAILED

FAIL Connect Modem 2

OK

Monitor Status

OKFAILED


OK

Monitor Status

OKFAILED


OK

STARTConnect Modem 1

Terminate All

Monitor Status

OK

FAIL

OK

FAIL

YES

NO


Field Tests and Results – Field implementation

Dimensions: 24x19x5

Antenna Connectors

USB Out

USB In

System with control software


Field Tests and Results – Antenna Setup

10 FT

3 FT


Results – Delay and Loss Measurement Ping tests between the two machines at the end of the of satellite link

One way propagation delay = (800+8000+6000)Km / (3e6)Km/sec= 49msec

Transmission time for 64 [email protected] = 64*8/2400=213msec

Theoretically, the RTT delay =2*(49+213)= 524msec+delay at the gateway

Test results show an average RTT delay of 1.8 sec, which may be attributed to the inter-satellite switching and delay at

the gateway

Packets sent

Packets received

% Loss

RTT (sec)

Avg Min Max mdev

50 50 0 1.835 1.347 4.127 0.798

100 100 0 1.785 1.448 4.056 0.573

100 100 0 2.067 1.313 6.255 1.272

200 200 0 1.815 1.333 6.228 0.809


Results – Delay MeasurementRTT Vs Time of the day

012345678

1:00 3:00 9:00 13:00 15:00 20:00

Time of the day in hh:mm

RTT

in s

ec Min

Avg

Max

Normalized RTT Vs Packet Size

32.05

26.50

9.247.51 7.52 7.04 7.13

12.3513.83

0

5

10

15

20

25

30

35

Packet Size (Bytes)

Nor

mal

ized

RTT

(mse

c/by

te)

Random variation of delay

High mean deviation

Delay increases linearly with

packet size

Normalized delay is almost

constant for MTU sizes > 800

bytes

Changing distance between the

user and satellite as well as

between satellites themselves

(ISLs)

Non-uniform traffic distribution,

varying delays on different routes 56 100 200 300 500 800 1000 1200 1500


Results – ThroughputMethod 1 Modem 2 Modems 3 Modems 4 Modems

Iperf 2.1 4.0 7.0 9.6

Iperf 1.9 3.9 7.0 9.3

Iperf 1.7 4.5 6.8 9.7

Ttcp 2.29 4.43 6.6 8.9

Ttcp 2.48 4.40 7.0 8.78

Average 2.1 4.25 6.88 9.26

Throughput Vs Number of Modems

2.1

4.25

6.88

9.26

0123456789

10

1 2 3 4Number of modems

Thro

ughp

ut

Tools used – TTCP, IPERF

Throughput varies to some

extend due to RTT variation

Efficiency > 90%

File Size (MB) Upload Time (min)

Throughput (bits/sec)

0.75 11 9091

3.2 60 7111

1.6 23 9275

2.3 45 6815

1.5 28 7143

2.5 35 9524

Effective throughputs during large file transfers

(K b

p s

)


Results – Reliability: 10th July 24 hr testCall drop events vs. Time

21:1

5

21:5

7

22:3

9

23:2

1

0:03

0:45

1:27

2:09

2:51

3:33

4:15

4:57

5:39

6:21

7:03

7:45

8:27

9:09

9:51

10:3

3

11:1

5

11:5

7

12:3

9

13:2

1

14:0

3

14:4

5

15:2

7

16:0

9

16:5

1

17:3

3

18:1

5

18:5

7

19:3

9

20:2

1

21:0

3

Time

Cal

l dro

p ev

ent

Modem up times during 24 hour test

0123

4

21:1

5

22:0

0

22:4

5

23:3

0

0:15

1:00

1:45

2:30

3:15

4:00

4:45

5:30

6:15

7:00

7:45

8:30

9:15

10:0

0

10:4

5

11:3

0

12:1

5

13:0

0

13:4

5

14:3

0

15:1

5

16:0

0

16:4

5

17:3

0

18:1

5

19:0

0

19:4

5

20:3

0

21:1

5

Time

Num

ber o

f onl

ine

mod

ems

Time intervalbetween call drops

(minutes)146 106 114 50 25 84 89 8 7 7 17 11 137 618

Total :

13 Call drops

80.691.894.796.8

Uptime %

Call drops on the first modem


Results – Reliability: 12th July 24 hr testCall drop events vs. Time

14:5

7

15:3

6

16:1

5

16:5

4

17:3

3

18:1

2

18:5

1

19:3

0

20:0

9

20:4

8

21:2

7

22:0

6

22:4

5

23:2

4

0:03

0:42

1:21

2:00

2:39

3:18

3:57

4:36

5:15

5:54

6:33

7:12

7:51

8:30

9:09

9:48

10:2

7

11:0

6

11:4

5

12:2

4

13:0

3

13:4

2

14:2

1

15:0

0

Time

Cal

l dro

p ev

ent

Modem up times during second 24 hour test

0

1

2

3

4

14:5

7

15:3

3

16:0

9

16:4

5

17:2

1

17:5

7

18:3

3

19:0

9

19:4

5

20:2

1

20:5

7

21:3

3

22:0

9

22:4

5

23:2

1

23:5

7

0:33

1:09

1:45

2:21

2:57

3:33

4:09

4:45

5:21

5:57

6:33

7:09

7:45

8:21

8:57

9:33

10:0

9

10:4

5

11:2

1

11:5

7

12:3

3

13:0

9

13:4

5

14:2

1

14:5

7

Time

Num

ber o

f onl

ine

mod

ems

Time intervalbetween call drops

(minutes)135 248 93 40 26 16 8 211 108 91 8 5 6 5 8 7 386

80929596

Uptime %

Total :

16 Call drops

Call drops on the first modem


Results – TCP performance of a single link

TCP Version – TCP SACK

Throughput of a segment is defined as the size

of the segment seen divided by the time since

the last segment was seen (in this direction).

RTT is defined as the time difference between

the instance a TCP segment is transmitted (by

the TCP layer) and the instance an

acknowledgement of that segment is received.

The average throughput of the connection is

2.45Kbps.

The average RTT was found to be 20 seconds

Bandwidth Delay Product (BDP) = 2400*20/8 =

6000 bytes = 4 segments.

Due to low throughput, the BDP is small.

-------- avg. of last 10 segments

-------- avg. of all the segments up to that point

Throughput vs. Time

RTT vs. Time


Results – TCP performance of a 4-channel system

-------- avg. of last 10 segments

-------- avg. of all the segments up to that point

The average throughput obtained - 9.4 Kbps

The average RTT observed - 16.6 seconds

Factors affecting throughput and RTT

TCP Slow start

MLPPP fragmentation

Random delay variation

TCP cumulative acknowledgments

------- Received Acknowledgements------- TCP segments transmitted------- Received window advertisement

RTT vs. Time

Throughput vs. Time

Time Sequence Graph


Results – TCP performance of a 4-channel system

------- Instantaneous outstanding data samples------- Average outstanding data------- Weighted average of outstanding data


Outstanding Unacknowledged

data and Congestion window

Time Sequence Graph

Time Sequence Graph

Outstanding Unacknowledged Data


Results – TCP performance degradation due to packet loss

Low packet loss, long time experiments needed to determine the performance degradation

Two packet losses were observed in the FTP video upload resulting in packet

retransmissions

Packet losses usually occur during call hand-offs

Retransmission time outs (RTO) is very large due to high RTT and high mean deviation

Retransmitted packetRetransmitted packet

Time Sequence Graph Time Sequence Graph


Results – TCP performance degradation due to packet loss

Effect on the TCP performance due to packet

loss

Decrease in the throughput following the packet

loss

RTT peaks around the packet loss

The average throughput of the connection is

observed to be 7.44 Kbps.Outstanding Unacknowledged Data

Throughput vs. Time

RTT vs. Time


Results – TCP performance degradation due to call drop

Packet loss due to a call drop on one links of the multilink bundle

A finite amount of time for the data link layer realize the link has failed

Large RTO timer The entire window of packets (12 in this case)

and acknowledgements that are in flight on that particular link are dropped.

Throughput of the connection – 7.6 Kbps.


------- Instantaneous outstanding data samples------- Average outstanding data------- Weighted average of outstanding data

Time Sequence Graph

Time Sequence Graph

Outstanding Unacknowledged Data


Results – Mobile tests

Test Location


Results – Mobile Performance

STOP

START

Avg. Throughput = 9 Kbps

Throughput vs. Time


Results – Mobile Performance (cont.d)

START

STOP

Avg. Throughput = 9.34 Kbps

Throughput vs. Time


Applications – Uploads and Downloads

The following files were downloaded from WEB and ITTC network. The size of these files, their importance (on a scale of 1-10, based on user survey) are shown

Title Downloaded/uploaded Size Imp

1 Spectrum Analyzer programmers Manual

Download from Agilent.com 7.2MB 9

2 Matlab Programs Download from ITTC 500KB 7

3 Voltage regulator data sheet Download from Fairchild.com 226KB 9

4 GPS software Download 800KB 9

5 Proposal submission Upload 600KB 8

6 Access point manager software Download from Orinoco.com 4.66MB 7

7 Drawing of machine spares to order

Upload to University of Copenhagen 1MB 9

8 Video of core, datasheet Upload for press release 2MB 8

9 Pictures, press release of longest core in Greenland

Upload to Kangerlussauq for press release

500KB 6


Applications – Internet at camp


Applications –WI-FI setup integrated with Iridium


Applications - Wireless Internet

Wireless Internet access was used

by many NGRIP researchers

Email access put the researchers in

touch with their home institutions

Scientist at NGRIP were able to

send information back to their home

institutions

The system was also useful for general camp purpose: sending drawings to order

spares for a broken caterpillar, excel spreadsheet for food order, general press

releases, downloading weather reports for planning C-130 landings


Applications – Outreach

Daily journal logs were uploaded to the University

of Kansas and posted on www.ku-prism.org

Two way video conference was held twice to test

the real time audio/video

The system not only helped PRISM group to

download critical software, but also made it

possible to obtain faculty guidance and expertise

on the field

It also helped the researchers back at the

University of Kansas to virtually view the field

experiments since the video clips of experiments

being carried out were also uploaded to the

University and posted on project website

Testing Video conference between the camp and the University

Uploading daily field logs

http://www.ku-prism.org/


Lessons Learned Multi-link PPP is relatively efficient over Iridium system. With 4-modems efficiency was observed to be

>90%

The RTT the system is significant ~ 1.8 seconds, which is an impairment to real time interactions

There is a large (random) variation in delay of the system

The average up-time of the overall connection is good > 95%

Mobile tests showed performance very similar to that of stationary system up to speeds of 20mph

A call drop on the first modem, results in a complete loss of connection, a potential bug in the PPP

networking code; could be fixed in newer version of the PPP software

Strong interference from a nearby source on the same frequency (1.6GHz) could cause little disturbance in

the system leading to either connection failures or call drops. This was observed when a radar sweeping

between 500MHz-2GHz with an output of 20dBm was placed at distance less than 15 meters


Conclusions

In order to provide data and Internet access to Polar Regions, we have developed a

reliable, easily scalable, lightweight, and readily available multi-channel data

communication system based on Iridium satellites that provide round the clock, pole-to-

pole coverage.

A link management software is developed that ensures fully autonomous and reliable

operation

An end-to-end network architecture providing Internet access to science expeditions in

Polar Regions is demonstrated.

This system provided for the first time, Internet access to NGRIP camp at Greenland and

obtained the call drop pattern.

The TCP performance and the reliability of the system is characterized


Future Work

Modify the MLPPP code so that the interface is attached to the bundle and not to the

primary link

Additional research to determine the cause of delay and develop methods to

overcome it.

Evaluate the new data-after-voice (DAV) service from Iridium

Evaluate different versions of TCP to determine the enhancements that can handle

the random variation and value of RTT

Improve the user friendliness of the system

GUI for connection setup

Self-test / diagnostic tools for troubleshooting

Research into the spacing and sharing of antennas to reduce the antenna footprint

Multi-Link Iridium Satellite Data Communication System

Documents

Transcript of Multi-Link Iridium Satellite Data Communication System