AccuverResource_attdrivetestresultsandreportpreview_1326383911

7/27/2019 AccuverResource_attdrivetestresultsandreportpreview_1326383911

http://slidepdf.com/reader/full/accuverresourceattdrivetestresultsandreportpreview1326383911 1/26

Signals AheadAT&T Drive Test Resultsand Report Preview



Septemer 18, 2011, Vol. 7 No. 10



2 September 18, 2011 | Signals Ahead, Vol. 7, Number 10

1.0 ExecutiveSummary

A& just launched commercial LE services in Dallas, Houston

Chicago, Atlanta, and San Antonio using its 700MHz spectrum assets

Coincidentally, we happened to fnd ourselves in Houston prior to the

launch where we had the chance to kick the tires o the operator’s network

Tis special-edition summary report is provided on a complimentary

basis to our existing Signals Ahead corporate clientele and it is being

made available to other interested parties. Unlike our regular Signals

Ahead research, which is strictly limited to paid subscribers, recipients o

this report may eel ree to orward this report to riends, colleagues and business associates – in act, we encourage you to do so.

In addition to quantiying the perormance o the network, based on some o the more requently

cited key perormance indicators (KPIs), we are using this summary report to promote a series o

orthcoming, ar more expansive, reports in which we will take our network benchmarking capabili

ties to a whole new level. In collaboration with Accuver, a leading supplier o drive test equipment

we have been using its XCAL-MO network benchmarking tool, which was specically designed to

acilitate the simultaneous testing o multiple networks, air interaces, and/or devices in a seamless

ashion.

Further, by careully selecting the markets in which we conducted our tests, we were able toinclude virtual ly every single combination o inrastructure vendor and technology, thus providing

invaluable competitive intelligence. In act, by testing in the Houston market on two dierent LE

networks we were able to compare and contrast the perormance o the two inrastructure supplier

who have the preponderance o the LE market share in North America. It goes without saying that

we tested all o the networks in the Houston area and that we contained great insight on how they

perorm, including the relative dierences in cell site density, which can impact perormance in the

near term and inherent capacity in the long term. Additional inormation about this series o reports

which is included as part o a normal Signals Ahead subscription, is contained later in this report.

Based on transerring 88.8GB o data over a three-day period with the Sierra Wireless

“USBConnect Momentum 4G” dongle, and ater covering 205.6 miles o the greater Houston area

we oer the ollowing observations about the perormance o A&’s LE network.

■ The average downlink Physical Layer throughput was 23.6Mbps with a peak

data rate o 61.1Mbps. Both results meaningully exceeded our expectations. Te data rate also

exceeded 40Mbp or 8.6% o the time and 21Mbps – the theoretical peak data rate o the operator’s

HSPA+ network – or 38.2% o the time. Most importantly, the data rate was greater than 5Mbps

or 95% o the time.

Arguably, the average data rate would have been even higher i we included a mix o stationary

testing rom optimal locations and/or included results rom pedestrian testing. Our approach

BytestingintheHoustonmarketontwodifferent

LTEnetworkswewereabletocompareandcontrast

theperformanceofthetwoinfrastructuresupplierswho

havethepreponderanceoftheLTEmarketshare

inNorthAmerica.




however, is ar more statistically signicant than stationary testing rom a couple o random loca

tions, which could result in highly erroneous results – either positive or negative. Tat being said

on at least one occasion we hit a data rate in excess o 60Mbps while traveling at 54mph (reerence

Figure 8).

■ The average uplink Physical Layer throughput was 15.2Mbps with a peak data

rate o 23.6Mbps. Te uplink results were a lso much higher than we anticipated. Sixty percent

(60%) o the time the uplink Physical Layer data rate exceeded 15Mbps and 98.2% o the time it

exceeded 5Mbps. Similar to the downlink throughput testing, we used the File ranser Protocol

(FP) to transer multiple les simultaneously to a server located in a third-party Cloud-based

server that we leased in the Chicago area as well as to FP servers in Dallas and Phoenix. Te

uplink throughput shown in Figure 9 (23.1Mbps) was achieved while traveling at 64mph.

■ The average latency, including the eects o the network and any transport

and IP routing delays which may have existed, was 49ms with a minimum value o

40ms. Tis result was higher [worse] than we were expecting based on the capabilities o LE

and our experiences with testing networks in Europe. Tat being said, the latency was on par with

other LE networks in the North American market. Over time, optimized routing and tighter

integration with the operator’s wireline assets and IP backbone can help reduce the latency outside

o the network. ransitioning to wider channel bandwidths (e.g., 20MHz channels) can also reduce

the latency in the radio access network. Worth noting, the average latency to this server using the

Verizon FiOS service in the Dallas area was 14ms and we observed latency as low as 27ms in the

Houston area to this server while using a commercial 3G network during the morning rush hour.

■ Intra-RAT handovers rom LTE to HSPA+ were relatively seamless or a t ypical

data application, requiring 2.4 seconds to exit the LTE network and establish a

dedicated connection on the HSPA+ network. Te handover time is based on the device

dropping the LE connection and establishing a connection on the HSPA+ network using the

CELL_DCH state. CELL_DCH is one o the RRC (Radio Resource Control) states used by an

HSPA+ device. An HSPA+ device uses the dedicated channel (DCH) when data is being transerredon an HSPA+ network. Te total time during which the data transer was interrupted was 6.5

seconds. Tese results are based on t wo instances where our device handed o to the 3G network

Handovers rom HSPA+ to LE currently require the device to return to the IDLE state.

We had no expectat ion on what these values should be but rom our perspective these values are

quite low, in particular in the inancy o LE, and a user will probably never notice the interruption

Obviously, applications, such as voice, will require a much shorter handover time, but since LE

is not currently being used to support voice services there is still ample time to lower the handover

latency. In our testing, the HSPA+ throughput ollowing the handover was quite good (median

MAC Layer throughput = 5.9Mbps), and or most applications it would have more than suced

Tat being said, we surmise that most users would preer to be on the higher throughput/ lower

latency LE network.

■ These results are specifc to AT&T’s LTE network in Houston and our test

methodology. Although we have no reason to expect materia lly dierent results in other A&

markets, we also know that several actors can, and do, impact the perceived network perormance

Tese actors include, but are not limited to the ollowing:

➤ Te use o a sophisticated drive test tool which can capture accurate, real-time inormation.




➤ Te combination o inrastructure suppl ier, device manuacturer, and chipset supplier. Without

question, in the early days o LE there are measurable perormance dierences that exist. Over

time the perormance capabilities and eatures o each important element should converge, but

this convergence does not exist today. In a pending Signals Ahead report we will be examining

the perormance dierences that exist with today’s LE chipsets. We will be doing this research

in collaboration with Spirent Communications.

➤ Access to a high-bandwidth server and/or servers with suciently low latency in order to ul ly

load the pipe. From what we have seen and documented, popular web testing portals may understate the throughput, in particular when testing very high bandwidth networks. Further, the

results that these services provide represent an average as measured over a very short period o

time during which only modest amounts o data are actual ly transerred.

➤ Te topology o the radio access network and the use o a distributed network architecture versus

a traditional macro cellular network. All things being equal, placing the radio electronics on top

o the tower and closer to the antenna mast will result in a higher throughput and better coverage

➤ Te processing capabilities o the host device and its associated operating system.

➤ Te topography o the market being tested.

➤ Te backhaul architecture, its throughput capabilities, as well as how the operator’s cellularnetwork interaces with its IP backbone and the location o its peering points with the Internet

Unless the end-to-end network is capable o supporting the peak data rates o LE, they will

never be achieved.

➤ Te mix o vehicular, pedestrian and stationary testing. Our methodology, which ocuses almost

exclusively on vehicular testing, tends to understand the true capabilities o the network.

➤ Te densication o the network cell sites. A denser deployment o cell sites should deliver higher

throughput i it has been properly tuned and optimized.

■ It will be quite some time beore network loading has a meaningul impact on

the throughput or a typical user. We have done more than our air share o drive testing in various LE networks and we have seldom detected the presence o other users, even with the use o

sophisticated drive test tools. Further, to the extent we did detect other people accessing the network

there wasn’t a material/long-term impact on the results we were obtaining. Tis observation does

not suggest that there are not any LE subscribers and that they are not using the network, but that

there are relatively so ew o them today and the combination o their distribution across the network

and their usage patterns tends to minimize or completely eliminate the impact o their presence

It also doesn’t hurt that LE is a very ecient technology when it comes to handling data and

signaling trac, that operators are committing a air amount o spectrum (e.g., 10-20MHz chan

nels) to LE, and that [at least some] operators are establishing taris that curtail bandwidth hogs

Following the Introduction, the Detailed Results section provides the throughput and latency

results or all o the A& LE test scenarios rom the greater Houston area – exclusive o theresults which involve other technologies and networks, as well as the user experience metrics. We

also include a est Methodology section, some closing remarks, and inormation about how you can

pre-order this series o reports.

BespokeServices Additional ly, we welcome the chance to speak with any organization that is interested in using our

services and the Accuver suite o benchmarking tools to conduct benchmark studies that are speci

cally tailored to a particular set o customer requirements.




Contents1.0 Executive Summary ………………………………………………………………………………………………………………………………………2

BespokeServices ………………………………………………………………………………………………………………………………………………4

2.0 Introduction …………………………………………………………………………………………………………………………………………………6

Volume1(NetworkandTechnologyPerformance) ……………………………………………………………………………………………7

Volume2(QuantifyingtheUserExperience) ……………………………………………………………………………………………………8

Volume3(DetailedPerformanceAnalysis)…………………………………………………………………………………………………………9

3.0 Detailed Results………………………………………………………………………………………………………………………………………… 11

3.1DownlinkResults ……………………………………………………………………………………………………………………………………… 13

3.2UplinkResults …………………………………………………………………………………………………………………………………………… 15

3.3Inter-RATHandoverResults ……………………………………………………………………………………………………………………… 17

3.4NetworkLatencyResults ………………………………………………………………………………………………………………………… 18

3.5TheStatisticalSignicanceof90GB ………………………………………………………………………………………………………… 19

4.0 Test Methodology …………………………………………………………………………………………………………………………………… 20

5.0 Conclusions ……………………………………………………………………………………………………………………………………………… 24

Volume 7, No. 10September 18, 2011

Michael W. Thelander(510) 338 [email protected]

WITHTHEEXCEPTIONOFTHISISSUE,ANUNAUTHORIzEDUSEOFOURRESEARCHATERIALWILLRESULTINTHENON-REFUNDABLECANCELLATIONOFOURSUBSCRIPTION.We also reserve the rightto post your company’s name, with logo, to the “SRG Wall of Shame.” If you received this issue from someone

outside of your organization and it did not come directly from SRG then the licensing terms for our research arebeing violated. If you forward this research to external organizations, either in whole or in part, or if you sharethe contents of the report beyond the authorized allocation within your organization then the licensing termsfor our research are being violated.

If you value the information and insight that we provide then I strongly urge you to respect our hard work andlivelihood and subscribe to our research. If you do not have a platinum license or a global license, you may wantto upgrade your license so that you can share this issue across your entire organization with our blessing.

If you or your organization is interested in distributing this report to outside organizations, please feel free tocontact us to discuss licensing terms and fees.

If you would like to leverage a gure or quote from this report and you have at least a global license, pleasecontact us for permission and we will be happy to provide it.

IndexofFiguresFigure1. “Oh the places we did go!” – Geo-plot of test routes with speed (mph) …………………………………………………… 11

Figure1A. “Oh the places we did go!” (Downtown Houston) – Geo-plot of test routes with speed (mph) ……………… 12

Figure2. AT&T Houston LTE Network Downlink PHY Layer Throughput Results – CDF and Pie Chart Distribution……… 13

Figure3. AT&T Houston LTE Network Vehicular Mode – Geo-plot of Physical Layer Downlink Data Rates ………………… 14

Figure4. AT&T Houston LTE Network Uplink PHY Layer Throughput Results – CDF and Pie Chart Distribution ………… 15

Figure5. AT&T Houston LTE Network Vehicular Mode – Geo-plot of Physical Layer Uplink Data Rates …………………… 16

Figure6. Inter-RAT Handover………………………………………………………………………………………………………………………………

17Figure7. AT&T Houston LTE Network Latency Results – Pie Chart Distribution………………………………………………………… 18




Volume1(NetworkandTechnologyPerformance)Next-generation network technologies are not created equal. On paper, there exist meaningul

perormance dierences, some o which are due to channel bandwidth considerations, but also to

the underlying technology itsel. Further, operator deployment philosophies and the maturity o the

solutions can have an over-arching impact on the results.

As operators around the globe struggle to make crucial strategic decisions regarding their network

technology evolution, it is imperative that they ully understand and appreciate the potential o

these technologies as well as their limitations. While this report is intended to address the needs o

operators worldwide by ocusing on the perormance o the technologies, as a secondary eature it

also provides valuable insight into the perormance o each major network deployment in the United

States. Everyone claims that they have the best network, but only one operator can be right.

Specic topics addressed in Volume 1 include, but are not limited to, the ollowing:

➤ Application and/or Physical Layer Troughput

♦ Mean, median and CDF plots

♦ Geo-plots o throughput or all test scenarios using Google Earth

➤ echnology comparisons, including

♦ DC-HSDPA versus LE (2x10MHz) with 2x2 MIMO

♦ LE (2x20MHz) versus LE (2x10MHz)

♦ Mobile WiMAX versus HSPA+, LE and DC-HSDPA

♦ DC-HSDPA versus HSPA+

♦ EV-DO versus LE, HSPA+, etc

➤ Single User Spectral Eciency Results

♦ Troughput normalized or channel bandwidth and duplex scheme

♦

Does LE with MIMO really outperorm narrow bandwidth solutions♦ LE network perormance with multiple devices

♦ DC-HSDPA and HSPA+ devices in the same 10MHz channel allocations

➤ Side-by-Side operator network coverage maps or drive routes used in each market

♦ Downlink throughput

♦ Uplink throughput

➤ Network Latency

♦ Variance based on time o day and network loading

➤ LE network deployment philosophies (LE cell site density relative to the legacy network) and

their implications or coverage and capacity

♦ A& HSPA+ versus A& LE

♦ Verizon Wireless EV-DO versus Verizon Wireless LE

♦ A& LE versus Verizon Wireless LE

➤ Mobile WiMAX (2500MHz) versus LE (700MHz)

Volume1iscriticalforoperatorsaroundthe

globewhoarecurrently

makingstrategicdecisionsregardingtheirnetwork

technologyevolution.




Volume2(QuantifyingtheUserExperience) Although mobile operators, industry pundits and most well-inormed consumers understand the

notion that a higher megabit-per-second throughput is preerable, the typical consumer is generally

clueless when it comes to understanding what these obscure marketing messages really mean or the

mobile Internet experience. Most operators recognize that they need to move away rom the “my

pipe is bigger than your pipe” marketing mentality, but it is easier said than done.

Further, it is readily apparent that the capabilities o these next-generation networks requently

exceed the requirements o the application and/or the capabilities o the Internet itsel. Very ew

applications and/or web site servers support high double-digit megabit-per-second throughput

Instead, it may actually be the combination o relatively high throughput and low network latency

– oset by transport latency – that really denes the user experience. But to what degree do these

relationships provide the most benet to the user?

Mobile video, be it Youube or Netix, is driving mobile data growth and the capabilities o next

generation networks will only serve as an impetus to even higher data usage. Te perceived quality

o the video playback also matters, not only or consumers, but also or mobile operators, content

owners, and video hosting services. As higher resolution video ormats with higher encoding rates

become more mainstream, this issue becomes even more important, especially i the perormance o

next-generation networks ails to keep up with the requirements o the video content.

Tis report is cr itical or operators try ing to understand how to market their broadband wireless

service oering as well as how they should prioritize their network optimization activities in order

to achieve the best possible user experience or their subscribers. In addition to mobile operators

this report provides invaluable insight to application developers and content providers who require a

greater appreciation or how network perormance characteristics impact the user experience.

Specic topics addressed in Volume 2 may include, but are not limited to, the ollowing:

➤ Quantiying the user experience based on HP web page download times

♦ Popular websites, including Yahoo, CNN, iunes, Amazon, Youube, etc.

♦ Results down to the millisecond, based on dev ice/chipset signaling messages

♦ Network/technology comparisons

♦ Comparisons based on network loading – same location over a 12-15 hour period o time

➤ Determining i perceived dierences in network/technology perormance have more to do withnetwork loading than the actual capabilities o the network/technology itsel

➤ Determining how the combination o throughput and latency impact the HP web pagedownload time results

♦ 3 axis plot, showing maximum achievable throughput and network latency versus webpageload time and required throughput

♦ Which matters most – latency or throughput

♦ Does DC-HSDPA really oer a quantiable benet over HSPA+

♦ Determining the crossover point when higher throughput become irrelevant

➤ Quantiying the user experience based on downloading Google email attachments

➤ Quantiying the user experience based on downloading video and audio content rom iunes

♦ Determining the crossover point when higher throughput become irrelevant

➤ Netix v ideo streaming requirements

➤ Determining the chokepoints in the network (rom end user to the original source o the contenthow they vary as a unction o loading, and their impact on the user experience

Volume2iscriticalforoperatorstrying

tounderstandhowtomarkettheirbroadbandwirelessserviceoffering

aswellashowtheyshouldprioritietheirnetworkoptimiationactivities.




Volume3(DetailedPerformanceAnalysis)In our third and nal installment we delve much deeper into the KPIs that we captured with the

Accuver suite o drive test tools. As we have witnessed in the past there are discernible dierences

in how each vendor implements a technology. Frankly, some vendors do a much better job than

their peers.

Given that we collected network perormance data in a number o key markets and that we know

ingly included virtually every single vendor + technology combination that exists in North America

this report provides invaluable competitive intelligence or vendors, subsystem suppliers and mobile

operators. Further, by peeling back the layers o the proverbial technology onion it becomes possible

to gain a greater appreciation or how each technology delivers its results.

Specic topics addressed in Volume 3 may include, but are not limited to, the ollowing:

➤ Modulation Utilization (QPSK, 16QAM, and 64QAM) – by primary and/or secondary carriersas appropriate

➤ MIMO RI 1 and RI 2 – how MIMO perorms at 700MHz

➤ CQI (average and median) – by primary and secondary carriers

➤ HS-PDSCH Codes (average, % > 10, distribution) – by primary and secondary carriers

➤

HS-SCCH Scheduling Success Rate – by primary and secondary carriers➤ Average PHY Layer Served Rate – by primary and secondary carriers

➤ Maximum PHY Layer Scheduled/Served Rate – DC-HSDPA only

➤ UL ransmit Power (average and median)

We will also leverage the capabilities o the XCAP-M tool to analyze these KPIs by severadierent means, potentially including, but not limited to the ollowing:

➤ MAC-HS Troughput versus RSCP scatter plot

➤ MAC-HS Troughput versus Reported CQI Values scatter plot

➤ Reported CQI Values versus 64QAM Availability scatter plot

➤ MAC-HS Troughput versus Cell ID (real time)

➤ MAC-HS Troughput versus # o Assigned HS-PDSCH Codes (real time)

➤ MAC-HS Troughput (primary, secondary, and combined)

➤ HSPA+ MAC-HS Troughput versus DC-HSDPA MAC-HS Troughput (primary, secondaryand combined)

➤ CINR versus RSSI (scatter plot and real-time)

➤ Troughput versus CINR (scatter plot and real-time)

➤ Troughput versus Cell ID (e.g., handover perormance)

➤ CINR versus RSSI (scatter plot and real-time)

➤ CINR versus Modulation Scheme and/or MCS

➤ UL ransmit Power versus Cell ID

➤ UL Troughput versus ransmit Power

➤ Modulation Scheme (antenna 1 and antenna 2)

Volume3providesinvaluablecompetitiveintelligencewhilealso

allowingreaderstoobtainagreaterappreciation

forhoweachtechnologydeliversitsresults.




LICENSECOSTS(pre-publishing and post-publishing prices) Volume 1 – Network and Technology Performance ($1,495, $1,995)Volume 2 – Quantifying the User Experience ($1,295, $1,495)Volume 3 – Detailed Performance Analysis ($1,295, $1,495)FullReport–all3volumes($3,300,$3,995)

CONTACTINFORATIONYou may call us at +1 (510) 273-2439 or email us at [email protected] and we will contact you for your billing informationor respond to any further inquiries that you may have. Subscriptioninformation for our Signals Ahead research product, which includesthese reports, can be found on the last page of this report. You canalso visit our website at www.signalsresearch.com or write us at

Signals Research Group, LLC10 Ormindale Court

Oakland, CA 94611

PRE-ORDEROURREPORTLICENSENOW(includedaspartofa Signals Ahead subscription)

Coming Soon!




3.0DetailedResults Although we have titled this chapter Detailed Results, it rea lly ocuses only on three KPIs, namely

downlink throughput, uplink throughput, and latency, including Inter-RA handover latency. We

will cover the more substantive analysis, including how A&’s LE network compares with its

HSPA+ network and the Verizon Wireless LE network with a similar test methodology, in our

orthcoming series o reports.

As previously indicated, our testing took place over a three day period. Since we had relatively

exclusive access to the pre-commercial network, we were not concerned about the implications onetwork loading. Tat being said, based on our testing o other commercial LE networks we

believe that it will be quite some time beore network loading has any discernible bearing on the

perormance o the network. In order to load the pipe, we tapped into multiple servers across the

United States, including a 150Mbps server that we provisioned in the immediate Chicago vicinity

and a server in Dallas, exas.

Figure1.“Ohtheplaceswedidgo!”Geo-plot of test routes with speed (mph)

50 <= x40 <= x <50

30 <= x <40

20 <= x <3015 <= x <20

10 <= x <15

5 <= x <100 <= x <5

VehicularSpeed(mph)

Source: Signals Research Group, LLC




4GWorld, Chicago, ILOct. 24-27

LTEAmericas, Dallas, TXNov. 8-9Invited speaker

RCRWirelessOCEvent, Orange County, CANov. 10Invited Speaker

ConsumerElectronicsShow, Las Vegas, NV Jan. 10-13

obileWorldCongress, Barcelona, SpainFeb. 20-23


In total, we covered 205.6 miles while testing A&’s LE network in the greater Houston area

Including testing that we did on other networks, we covered nearly 350 miles. Figure 1 shows the

routes that we covered during our downlink and uplink testing. Te second gure enhances the

testing that was done in the downtown area o Houston, while the long pinkish line extending outrom Houston reects the reeway heading back to Bush International Airport while we were testing

the uplink (3.6GB o transerred data; Average PHY Layer uplink throughput = 16.9Mbps). o pu

things into perspective, the main ocus o our testing spanned a 15 square mile grid, with the drive

back to the airport (the long pinkish line) covering roughly 20 miles outside o the grid.

Most o the results that we present in this report are based on the Physical Layer throughput

Troughput at the Application Layer will be modestly lower and based solely on the eciency o

the application/protocol that is being used. In looking at all o our data, including the results rom

tests in other markets, a good rule o thumb is that the Application Layer throughput using the FP

protocol is 10% lower than the Physical Layer throughput.

Figure1A.“Ohtheplaceswedidgo!”(DowntownHouston)Geo-plot of test routes with speed (mph)

50 <= x

40 <= x <50

30 <= x <40

20 <= x <30

15 <= x <20

10 <= x <15

5 <= x <10

0 <= x <5

VehicularSpeed(mph)




3.1DownlinkResults Te downlink results that we present are based on transerring 73.9GB o data. All o the testing

took place rom a vehicle that was constantly moving unless we were stuck in trac or sitting at a

trac light. In addition to testing in the greater downtown area, we included reeway testing, area

that can best be described as urban sprawl, and residential areas in seemingly remote portions o th

city. Based purely on luck, we believe that we hit all o the major universities and traveled by Relian

stadium on at least a couple o occasions.

Figure 2 shows the results across all test scenarios. Te average Physical Layer throughput was

23.6Mbps and the maximum data rate was 61.1Mbps. We note that the theoretical maximum dat

rate o a Category 3 device in a 10MHz channel with 2x2 MIMO and under pristine and completel

unrea listic network conditions is 73.6Mbps. Te data rate also exceeded 40Mbp or 8.6% o the tim

and 21Mbps – the theoretical peak data rate o the operator’s HSPA+ network – or 38.2% o th

time. Most importantly, the data rate was greater than 5Mbps or 95% o the time. Although it is no

specically shown in the gure, during our tests the average Physical Layer throughput exceede

30Mbps – we downloaded at least 3GB during each o these tests while driving.

TheaveragePhysicalLayerdownlinkthroughputwas

23.6bpsandthemaximumdataratewas61.1bps

whileweachievedatleast5bpsfor95%ofthetime.

> 50Mbps 4.2%

45 -50Mbps4.4%40 -

45Mbps6.4%

35 - 40Mbps5.9%

30 - 35Mbps7.9%

25 - 30Mbps9.4%

20 - 25Mbps12.1%

15 - 20Mbps22.8%

10 - 15Mbps14.6%

5 -10Mbps7.3%

2.5 - 5Mbps 3.7%

0 - 2.5Mbps1.3%

0%

20%

40%

60%

80%

100%

605550454035302520151052.5

Mbps

CDF

Average PHY Layer Data Rate = 23.6MbpsMax PHY Layer Data Rate = 61.1Mbps

Total Distance Traveled = 153.9 milesTotal Data Transfer = 73.9GB

Figure2.AT&THoustonLTENetworkDownlinkPHLayerThroughputResults–CDF and Pie Chart Distribution

Source: Signals Research Group, LL




Figure 3 contains a Geo-plot o the downlink throughput. Te second gure shows an enhanced

view o the downtown area o Houston.

Figure3.AT&THoustonLTENetworkVehicularode–Geo-plot of Physical Layer Downlink Data Rates


55 <= x

50 <= x <55

45 <= x <50

40 <= x <45

35 <= x <40

30 <= x <35

25 <= x <30

20 <= x <25

15 <= x <20

10 <= x <15

5 <= x <10

2.5 <= x <5

0 <= x <2.5

DownlinkPHLayerThroughput(bps)




3.2UplinkResults Te uplink throughput results are based on transerring 14.9GB o data while driving 51.7 miles

Our test methodology was identical to our downlink testing. Te average Physical Layer uplink data

rate was 15.2Mbps and the peak data rate was 23.6Mbps. Te maximum theoretical Physical Layer

uplink data rate or a Category 3 device in a 10MHz channel is 26.8Mbps. Sixty percent (60%) o

the time the uplink Physical Layer data rate exceeded 15Mbps and 98.2% o the time it exceeded

5Mbps.

Like the downlink results, the uplink throughput is relatively impervious to vehicular speed. In

act, during the 26.2 mile drive back to the airport (average vehicular speed = 53mph), the average

uplink throughput was 16.9Mbps – we transerred 3.7GB during this test.

Figure 4 contains the results rom our uplink testing and Figure 5 contains a geo-plot o the same

results. Once again, the second gure expands the results rom the downtown area o Houston.

20 - 25Mbps11.5%

15 - 20Mbps48.5%

12.5 - 15Mbps16.5%

10 - 12.5Mbps11.9%

7.5 -10Mbps6.2%

5 - 7.5Mbps 3.5%2.5 - 5Mbps 1.6%1 - 2.5 Mbps 0.2%

25201512.5107.552.51

0%

20%

40%

60%

80%

100%

Mbps

CDF

Average PHY Layer Data Rate = 15.2MbpsMax PHY Layer Data Rate = 23.6Mbps

Total Distance Traveled = 51.7 milesTotal Data Transfer = 14.9GB

Figure4.AT&THoustonLTENetworkUplinkPHLayerThroughputResults–CDF and Pie Chart Distribution





Figure5.AT&THoustonLTENetworkVehicularode–Geo-plot of Physical Layer Uplink Data Rates

20 <= x <25

15 <= x <20

12.5 <= x <15

10 <= x <12.5

7.5 <= x <10

5 <= x <7.5

2.5 <= x <=5

1 <= x <=2.5

0 <= x <=1

UplinkPHLayerThroughput(bps)





3.3Inter-RATHandoverResults We had original ly intended to test Inter-RA handovers between LE and HSPA+ on our way

back to the airport, but as it turned out the operator had coverage the entire way. We suspect, bu

didn’t conrm, that the coverage primarily existed along the reeway and not in the residential

areas extending out rom the reeway. However, during the process o reviewing the data we

stumbled upon two instances where our device handed o rom the LE network to the HSPA+

network. Although two tests may not be statistically signicant we did observe very similar results

between the two handover events – one occurred during downlink testing and one occurred during

uplink testing.

Figure 6 shows the MAC-layer throughput or the device when it was on the LE network and

ater it switched to the HSPA+ network. We used the MAC layer throughput since it provided

greater resolution and since it was a good common denominator between the two air interaces

According to our analysis o the log les, there was ~6.5 seconds o down time in the data transer

as the device was switching network. It actually took the device only 2.4 seconds to establish a

connection to the HSPA+ network. Tis latter calculation was based on the time between the device

signaling that it was leaving the LE network until the time that it entered the CELL_DCH state

CELL_DCH is the radio resource control (RRC) state used by an HSPA+ device when it is sending

large amounts o data.

0

10

20

30

40

50

60550500450400350300250200150100500

AT&T LTE Median Throughput (MAC Layer) = 16.8MbpsAT&T HSPA+ Median Throughput (MAC Layer) = 5.9Mbps

LTE to HSPA+ CELL_DCH Handover Time = ~2.4 seconds(~6.5 seconds of interrupted data flow)

AT&T LTE MAC Layer Throughput (Mbps)

AT&T LTE HSPA+ MAC Layer Throughput (Mbps)

Mbps

Time (sec)

Figure6.Inter-RATHandover





From our perspective, the handover latency was quite good, in particular given the early days o

the network and the inancy o LE. A typical user would never notice the interruption with most

applications – network drive testing being a notable exception. Even applications, such as video

streaming, generate relatively large buers which can endure a ew seconds o interrupted data

transers. Tat being said, Inter-RA handover latency will have to get much better in order to

deliver a seamless voice service between the two networks. Since VoLE on A&’s network is stil

several years away there should be ample time to address the issue. At the moment, handovers rom

HSPA+ to LE require that the device return to the IDLE state.

3.4NetworkLatencyResultsOur methodology or testing network latency leveraged local servers in the markets that we were

testing. On occasion we tried multiple servers in order to obtain a mix o results. In order to test all

networks simultaneously we also had to use servers that resided outside o an operator’s network

Tis requirement can impact the latency, in particular i the peering points between the operator’s

mobile network and the Internet are less than optimal.

In order to be as objective as possible, we also measured latency to these servers using a local

broadband service provider. For example, we used Verizon FiOS rom a residence in Dallas to test

the server that we used or latency testing in Houston and Dallas. In our orthcoming reports we

also encourage readers to ocus on the relative results versus the absolute results.

Figure 7 shows results rom two latency tests that we did on A&’s LE network in Houston

One test was done rom a xed location while the other test was done rom a moving vehicle. As

indicated in the two pie charts, the average latency was 49-50ms over an extended period o time

(>60 seconds) with 40ms being the minimum latency. At one point we saw a low o 35ms latency

but this result is not reected in the below pie charts.

In general, we have been disappointed with the latency on the LE networks in North America

Tis opinion is based on our experience with LE networks in Europe (15ms average latency to

an external server) and the results rom some o the 3G networks that we have been testing in

North America. Without naming names, we documented a minimum round trip latency o 27ms

while testing in Houston using this very same server, albeit when we were testing a commercial 3G

network with a large number o subscribers. Latency is an area or improvement, and A& isn’talone in that regard.


>65ms

4%61 - 65ms

8%

56 - 60ms21%

51 - 55ms

17% 48 -

50ms

6%

45 - 47ms

8%

43 - 45ms

18%

40 - 42ms

18%

>65ms

6%61 -

65ms

6%

56 - 60ms

23%

51 - 55ms

9%

48 - 50ms

8%45 - 47ms

15%

43 - 45ms

22%

40 - 42ms

17%

AT&T LTE Latency (Drive Test) Average Latency = 50ms AT&T LTE Latency Stationary Test) Average Latency = 49ms

Figure7.AT&THoustonLTENetworkLatencyResults–Pie Chart Distribution




3.5TheStatisticalSignicanceof90GBIn total, we transerred 90 GB during our testing o A&’s LE network, including the user

experience tests that we conducted – the results rom these tests are not included in this report

Readers may or may not agree that 90GB o transerred data can result in statistically signicant

results, but consider the ollowing:

➤ 90GB equates to nine months o usage without incurring overage ees.

➤ Tere are 5,946,800 people in the Houston metropolitan area, meaning that our data usage was

the equivalent o each Houstonian transerring 15.9 kilobytes o data.

➤ A typical downlink/uplink/latency test using the Speedtest.net portal when the network being

tested supports a 50/15Mbps pipe uses 52.4MB o data [we did the test and examined the Accuver

log les to reach this conclusion]. Doing the math, we did the equivalent o 1,750 bandwidth tests

using this service. Presumably, the amount o transerred data is a unction o the capabilities o

the network, suggesting that the actual number o implied tests is considerably higher.

➤ According to a somewhat recent study by Google, a typical web page consists o 320kB o data

suggesting that we loaded the equivalent o 294,912 web pages.

➤ We transerred 5GB or each gallon o gas that our rental SUV consumed. Tis metric may say

more about the uel ineciency o a Ford Explorer than the eciency o LE.

➤ For each and every Katy Perry song that we listened to on the local radio station (27 times) we

transerred 3.33GB o data. Tis last metric may say more about the popularity o Katy Perry

than the perormance o A&’s LE network.




Figure8.XCAL-DriveTestToolinAction–DLperformance

Source: Accuver XCAL and SRG

4.0 TestethodologyFor the drive tests that we have been conducting this summer we primarily used the recently released

Accuver XCAL-MO network benchmarking tool, augmented by the Accuver XCAL drive test

tool to collect the underlying perormance indicators and to conduct the user experience tests. Fo

purposes o our tests, we “limited” the XCAL-MO to only our dongles – one dongle or each

network/technology that we wanted to test. In theory we could have installed multiple dongles or

each network/technology.

We used the Accuver XCAP post-processing tool to analyze the data and to help us create thegures which appear in this summary report. Tanks to a combination o the powerul tool and

countless hours spent on the road, we are convinced that we have witnessed network perormance –

both good and bad – that would have otherwise not been observed.

We used the Accuver XCAL-MO and XCAL-M tools to collect the underlying perormance

indicators and the Accuver XCAP-M post-processing tool to do the analysis o the data that we

collected.

Figure 7 and Figure 8 illustrate a typical user display that we used when collecting the data. We

have included two gures since they also help prove that we observed downlink data rates greater

than 61Mbps (Figure 7) and uplink data rates in excess o 23Mbps (Figure 8). Interestingly, we

were traveling in excess o 60mph when we achieved both results, or 55mph and 64mph, respectively




At our request, A& provided us with an LE dongle and unlimited access to its LE network

For our “Mother o all Network Benchmark ests” each operator provided us with at least two

dongles, although in the case o operators, such as Clearwire, with multiple network/technology

deployments (e.g., 2x20MHz LE and 1x10MHz Mobile WiMAX), we received multiple dongles

In order to ensure that we ully loaded the air link channel or the networks/technologies that we were testing, we leveraged multiple high-bandwidth servers, including servers in Phoenix

(>100Mbps), Dallas (300Mbps) and Chicago (150-300Mbps). Further, we established multiple

sessions in order to oset some o eects o transport latency and the CP ACK window associated

with FP.

For purposes o these tests, we used the Windows 7 operating system, which uses a dynamic

window allocation methodology to theoretically deliver the best possible throughput or the given

combination o latency and channel conditions. As we have documented in the past, we believe

Windows 7 leaves something on the table, meaning that the super-high data rates associated with

LE, in particular with a 20MHz channel, are not always achieved. We believe our approach

is still appropriate given that it is logistically impossible to purchase a notebook computer with

the Windows XP operating system. We also note that the connection manager associated with atleast one operator’s technology/network does not change the CP window size when it is installed

meaning that the data rates could already be limited, even with the legacy operating system.

For all o our testing, network latency tests were done to servers located in the vicinity o the market

in which we were doing the tests. Since we can’t rule out the eect o transport latency, readers

should ocus their attention on the relative perormance dierences versus the absolute results.

esting in each market took place rom as early as 4AM local time until the early evening hours

We also conducted user experience tests rom the connes o our hotel room in the dead o night

Since we were using test equipment we had the ability to determine whether or not network loading

WeusedtheWindows7operatingsystem,whichusesadynamicwindow

allocationmethodologytotheoreticallydeliverthebest

possiblethroughputforthegivencombinationoflatency

andchannelconditions.

Testingineachmarkettookplacefromasearlyas4Alocaltimeuntil

theearlyeveninghours.

Figure9.XCAL-DriveTestToolinAction–ULperformance

Source: Accuver XCAL and SRG




➤ 7/6/11 “Mobile Platorms – the center o mobilenetworks” In this report we discuss the recent trendsimpacting the various mobile platorms that exist and what hastranspired since our piece rom three years ago on Web 2.0.

We address the state o the mobile platorms that exist, provide

our thoughts on the current and uture prospects and look atthe various trends that are driving the industry.

➤ 6/8/2011 “United we stand, ragmented we ail” Weprovide the key takeaways rom the LE World Summit, heldin Amsterdam. Spectrum ragmentation tops the list o key LE topics, although a growing ocus on the use o 1800MHz or those operators that have access to it is encouraging.VoLE, or the lack thereo, is still on everyone’s minds, butin the interim CSFB isn’t even working as promised. Finally,there was a lot o talk about Mobile WiMA X, but the emphasisseemed to be on how to best move away rom the technology and adopt D-LE.

➤ 5/16/2011 “HetNet: When big cells and small cellscollide” In addition to covering the basics o heterogeneousnetworks (HetNet), a key LE-Advanced (R10) eature, wepresent a compelling series o analytical studies which demon-strate the need or macro network ofoad, starting as early as2015. We also get into the technical details o how HetNet

works, including discussions on eICIC, ABS and the impor-tance o intererence cancellation in the handset. Finally, welook at what is being done with legacy 3G emtocells to limitintererence-related problems that they introduce, both withthe macro network and between each other.

➤ 4/26/2011 “Chips and Salsa XIII: Now Seasoned with

Soy Sauce” In collaboration with Spirent Communications we provide results rom the industry’s only independentperormance benchmark tests o HSPA+/HSPA chipsets.In the most recent benchmark study we tested 16 dierentdevice congurations, representing chipsets rom 9 dierentsuppliers, including new entrants, such as Samsung (HSPA+),Intel (HSPA+), Mediaek and HiSilicon. We provide theresults, based on a total o 42 HSPA+ test scenarios and 26HSPA test scenarios.

➤ 3/15/2011 “Looking beyond HSPA+: keeping up withthe Joneses” Based on interviews with 3GPP membercompanies and a thorough review o 3GPP submissions, we

oer an in-depth look at the uture o HSPA+ (Release 11 andbeyond). Ultimately, we conclude that many o the eaturesthat are being incorporated into LE will nd their way intoHSPA+, thus blurring the perormance dierences betweenthe two technologies. Latency and the impact o new eatureson legacy devices are two areas o prime importance whereHSPA+ could ace challenges relative to LE.

➤ 1/12/2011 “DC-HSDPA: Double the Bandwidth, Doublethe Pleasure, Part II” In collaboration with Accuver, whoprovided us with its XCAL-W drive test tool and XCAP-W post-processing sotware, we provide results and analysis roman extensive drive test o elstra’s DC-HSDPA network. We

compare DC-HSDPA with HSPA+ perormance in a numbero side-by-side tests.

➤ 1/12/2011 DC-HSDPA: “Double the Bandwidth, Doublethe Pleasure, Part I” In collaboration with Accuver, whoprovided us with its XCAL-W drive test tool and XCAP-W post-processing sotware, we provide results and analysis roman extensive drive test o elstra’s DC-HSDPA network. Wecompare DC-HSDPA with HSPA+ perormance in a numbero side-by-side tests.

➤ 12/10/2010“Can you schedule me now?” In collabora-tion with Sanjole we examine how some o today’s commer-cial LE eNodeBs allocate network resources when servingmultiple devices. We determine that while LE may delivera compelling user experience, it is largely due to an empty network and the large channel bandwidths, and that urtherimprovements are necessary i LE is going to supportmultiple users in an ecient manner.

➤12/3/2010 “A Perspective rom LTE Americas and theGSMA Mobile Asia Congress” We provide and discuss

various data points which stem rom our participation at theLE Americas event in Dallas and the GSMA MAC eventin Hong Kong. We provide an LE market update, including

D-LE, discuss the debate about a smart or dumb pipestrategy, and the impact o smartphones and social networking

services, including the use o cloud computing, intelligentnetworks, network ofoading and data caching.

➤11/4/2010 “A G-Wiz World” We provide drive test resultsor eliaSonera’s HSPA+ network in Sweden and provide datapoints rom this year’s 4G World event in Chicago. We alsodiscuss the growing trend o operators who are intelligently adding more capacity to their networks through the use o higher perorming devices/chipsets and upgrades to theirnetwork inrastructure.

➤10/7/2010 “2x20MHz o LTE and the HeisenbergUncertainty Principle” We provide an update on

LE network perormance based on extensive testing o eliaSonera’s LE networks in Stockholm and Gothenburg,Sweden. Te 60+ page report provides detailed results andanalysis based on more than 600GB o transerred data.

In Case You Missed It




was impacting the results. Suce it to say that in the early morning hours network loading was not

a concern or any o the networks. Later in the day, network loading impacted the perormance o

certain networks/technologies while it was not even a consideration with other networks/technolo

gies. We take this phenomenon into consideration when doing our analysis.

A large percentage o our test data was collected rom a moving vehicle. Tis approach ensured tha

we achieve statistically meaningul results since as we have demonstrated in past reports, moving

a ew eet or turning 90 degrees can meaningully impact the achievable throughput. Further, we

based our analysis and conclusions on literally hundreds o Gigabytes o transerred data. Tis

approach is markedly dierent rom the more commonly used method which involves using popular

web-based “speed testing” sites and transerring tens o Megabytes o data. From our perspective

this latter approach achieves anecdotal results which are statistically meaningless and not neces

sarily reective o the overall network perormance. Tis sampling o the network perormance

also provides no insight whatsoever into how/why the throughput was achieved since the KPIs

are limited to throughput and latency, versus KPIs, such as modulation type, number o assigned

resource blocks, MIMO avai lability, scheduling requency, etc.

One drawback o our approach is that it does tend to understate the perormance o the network

since the ading conditions rom a moving vehicle are more challenging than rom a stationary

position or someone walking down the street. Given the emergence o next-generation smartphones

(LE, HSPA+, Mobile WiMAX, etc.), accessing the broadband wireless network rom a moving

vehicle will be more commonplace now than in the past. Ideal ly, we would include stationary tests

rom hundreds o locations in a given market and all times o day, but this approach goes well

beyond something that we can reasonably do or these studies.

By testing rom a moving vehicle and by transerring hundreds o Gigabytes o data, we believe

that our conclusions are statistically signicant, even i the results may sl ightly understate the true

capabilities o the network.

Although we do not include results rom our user experience tests in this summary report, we used

popular websites or our HP web page download tests and Google Mail or the email applica

tion. We also used popular applications, such as Skype, and content providers, such as Netix and

Youube, to evaluate the impact o various video streaming services.

Like all Signals Ahead reports, we received no sponsorship or unding rom the participatingcompanies, in order to maintain our independence. As such, we oot the bill or all o our travel

expenses not to mention an inordinate amount o time and eort collecting the data and writing

these series o reports.

We also could not have done this report without the support o Accuver who provided us with its

suite o drive test tools and post-processing sotware. SRG takes ull responsibility or the analysis

and conclusions that are documented in this report and in our orthcoming series o reports.

Onedrawbackofourapproachisthatitdoestendtounderstatethe

performanceofthenetwork.

Wereceivednosponsorshiporfundingfromtheparticipatingcompanies.




5.0ConclusionsNow that we have nally nished our testing we will in short order be releasing the rst in a series

o reports. We expect that the rst report will be released beore the end o September and that the

ollowing two reports will ollow in October and potentially extending into November.

I you are currently a subscriber to Signals Ahead t hen you will receive these reports as part o your

subscription. I you are not a subscriber then you can purchase the reports on an individual basis o

purchase a corporate subscription to our research. Te latter option is more economically attractive

since it includes at least ourteen additional issues. Until next time, be on the lookout or the nextSignals Ahead….

ichaelThelanderMichael Telander is the CEO and Founder o Signals Research Group. In his current

endeavor he leads a team o industry experts providing technical and operator economics anal-

ysis or clients on a global basis. Mr. Telander is also responsible or the consultancy’s Signals

Ahead research product, including its widely acclaimed “Chips and Salsa” series o reports that

ocus on the wireless IC industry.

Previously, Mr. Telander was an analyst with Deutsche Bank Equity Research. Prior to joiningDeutsche Bank, Mr. Telander was a consultant with KPMG (now known as BearingPoint)

and a communications ocer with the United States Army. Mr. Telander has also published

numerous articles or leading trade publications and engineering journals throughout his career.

He has been an invited speaker at industry conerences around the world and he is requently

quoted by major news sources and industry newsletters, including he Economist, Te Wall Street

Journal, Investors Business Daily, Reuters, Bloomberg News, and Te China Daily. Mr. Telander

earned a Masters o Science in Solid State Physics rom North Carolina State University and a

Masters o Business Administration rom the University o Chicago, Graduate School o Business.



please note disclaimer: Te view s expre ssed in thi s news letter reect those o Signa ls Research Group, LLC and are based on our under standing o pa st and current events shaping the wi relessindustry. Tis report is provided or inormational purposes only and on the condition that it will not orm a basis or any investment decision. Te inormation has been obtained rom sources believedto be reliable, but Signals Research Group, LLC makes no representation as to the accuracy or completeness o such inormation. Opinions, estimates, projections or orecasts in this report constitutethe current judgment o the author(s) as o the date o this report. Signals Research Group, LLC has no obligation to update, modi y or amend this report or to otherw ise notiy a reader thereo in theevent that any matter stated herein, or any opinion, projection, orecast or estimate set orth herein, changes or subsequently becomes inaccurate.

I you eel our opinions, analysis or interpretations o events are inaccurate, please ell ree to contact Signals Research Group, LLC. We are always seeking a more accurate understanding o thetopics that inuence the wireless industry. Reerence in the newsletter to a company that is publicly traded is not a recommendation to buy or sell the shares o such company. Signals Research GroupLLC and/or its aliates/investors may hold securities positions in the companies discussed in this report a nd may requently trade in such positions. Such investment activity may be inconsistent withthe analysis provided in this report. Signals Research Group, LLC seeks to do business and may currently be doing business with companies discussed in this report. Readers should be aware that SignalResearch Group, LLC might have a conict o interest that could aect the objectivity o this report. Additional inormation and disclosures can be ound at our website at w ww.signalsresearch.com Tis repor t may not be reproduced, copied , dist ributed or publ ished wit hout the prio r writt en authoriz ation o Sign als Research Group, LL C (copyright ©2011, all rights rese rved by Signals R esearchGroup LLC)

Signals Ahead SubscriptionThe Signals Ahead newsletter is available on a subscription basis. We offer four distinct packages that have been tai-lored to address the needs of our corporate users. The Group License includes up to ve users from the same company. TheGlobal License is the most attractive package for companies that have several readers since it is offered to an unlimitednumber of employees from the same organization. Finally, the Platinum package includes the Global License, plus up to vehours of analyst time. Other packages are available.

Corporate Rates (18 issues)❒Group License ($3,995) ❒Global License ($7,995) ❒Platinum ($9,495) Payment Terms

❒American Express ❒Visa ❒MasterCard Credit Card # Exp Date / /

❒Check Check Number

❒Purchase Order PO Number

Name: Title:

Afliation: Phone: ( )

Mailing Address:

Mailing AddressSignals Research Group, LLC – ATTN: Sales10 Ormindale CourtOakland, CA 94611

Our ax number is (510)338-1284.

Alternatively, you may contact us at (510)273-2439 or at [email protected] and we will contact you for your bill-ing information. We will not process your payment until after the trial subscription period is completed.

Terms and Conditions: Any copying, redistributing, or republishing of this material, including unauthorizedsharing of user accounts, is strictly prohibited without the written consent of SRG.

AccuverResource_attdrivetestresultsandreportpreview_1326383911

Documents

Transcript of AccuverResource_attdrivetestresultsandreportpreview_1326383911