device performance and show 51% greater 802.11ac device ...a Samsung Galaxy S4 smartphone, 1SS a...

Lab Testing Summary

Report

Key findings and conclusions:

Purpose-built Aruba 802.11ac APs double the 802.11n device performance and show 51% greater 802.11ac device performance compared to Cisco APs with an 802.11ac upgrade module

Rate versus range tests exhibit comprehensive throughput advantage in three Wi-Fi technologies: 802.11ac, 802.11n 5 GHz and 802.11n 2.4 GHz

Performance testing was conducted with Aruba AP-225 that uses a purpose-built 802.11ac approach which exhibited consistent and reliable performance in a real world office environment

October 2013 Report 13096

Product Category:

Wireless Access

Point

Vendor Tested:

Products Tested:

Aruba Networks AP-225

Cisco Aironet

3602i

Average throughput of downstream traffic from the Aruba AP-225 access point to a single 3SS client, five feet away, was 768.83 Mbps. A 51% percent increase over the Cisco Aironet 3602i.

Aruba Networks engaged Miercom to provide an independent evaluation of the performance capabilities of its AP-225 802.11ac indoor wireless access point, and the Cisco Aironet

3602i. Both access points have 802.11ac gigabit Wi-Fi functionality.

Layer 3 traffic, generated by Ixia IxChariot, was used in all tests. One of the tests validated maximum 802.11ac throughput between each access point and a single client device. Results are shown in Figure 1and discussed further on page 2.

The other three tests were rate versus range. What is the impact on throughput when the distance between APs and client devices is increased, and does physical obstruction between points affect throughput? For clients or endpoints, three different devices were used. Each client supported one, two or three spatial streams.

Source: Miercom, October 2013

Figure 1: Aruba AP-225 Access Point Maximum Throughput, 3SS Dell Laptop, 5 Feet LOS

768.83

507.86

0

100

200

300

400

500

600

700

800

Aruba AP-225 Cisco Aironet 3602i

Th

rou

gh

pu

t (M

bp

s)

Copyright © 2013 Miercom Aruba Networks AP-225 Access Point

Aruba began shipping its AP-200 Series, of which the AP-225 is a member, in May 2013. All are designed to deliver gigabit Wi-Fi performance to 802.11ac-capable mobile devices deployed in high-density wireless LAN environments.

The 220 Series consists of two access points, 224 and 225. Each is available in two versions, AP and IAP (Instant AP). The AP is designed to connect with a dedicated mobility controller while the IAP can be cloud-managed or managed as a local cluster of APs. IAPs can easily be converted into controller-managed APs from the management console.

While announced in June of 2012, the Cisco 802.11ac module and its software (Version 7.5) became available in August of 2013. The 3600 Series access points with which the module works are more than a year old, having started shipment in the first quarter of 2012.

Aruba AP access points are managed by a cloud-based management system, by Web console or by a dedicated on-premise hardware appliance. An Aruba Mobility Controller enables the AP access point to provide centralized configuration, data encryption, policy enforcement and network services, as well as distributed and centralized traffic forwarding.

Key specifications of the AP-225 (and IAP-225) include a software-configurable dual radio with maximum data rate of 600 Mbps in the 2.4-GHz band and 1.3 Gbps in the 5-GHzband, for a total of 1.9 Gbps across both radios. The radio also has 3x3 MIMO capability with three spatial streams and three integrated omni-directional down-tilt antennas.

Maximum Throughput Test This test verified the maximum rate of downstream traffic each access point could transmit in a Line of Sight (LOS) environment to a single client device five feet away. No physical obstructions were between the access point and the client device, a 3SS Dell Latitude laptop.

The access points operated in the 5 GHz radio band and transmitted test traffic generated byIxia IxChariot.

The results of three one-minute tests were averaged to create the throughput value for each access point.

The Aruba AP-225 demonstrated the ability to transmit 51% more traffic. Figure 1 onpage 1 shows the average throughput for the Aruba AP-225 to be 768.83 Mbps and Cisco Aironet 3602i was 507.86 Mbps.

Rate Versus Range Tests Three sets of tests were conducted to determine what the impact had on throughput.

In testing, the access points operated in the 802.11ac radio band, 5 GHz, as well as in both 802.11n bands, 5 GHz and 2.4 GHz.

Throughput values for three types of 802.11ac/n-capable client devices were recorded. Four different distance and obstruction scenarios were used.

The combination of distance and/or physical obstruction between the access point and the client device for each location was as follows:

5 feet LOS

30 feet Non-LOS, one sheetrock wall with metal studs

75 feet Non-LOS, two walls with metal studs

120 feet Non-LOS, three walls with metal studs

The maximum theoretical throughput possible for each device in each test varied based on the type of 802.11 functionality and/or radio band being utilized.

Each client device was tested in each set one at a time. Also, each has a different spatial stream (SS) capacity:

a Samsung Galaxy S4 smartphone, 1SS

a MacBook Air notebook, 2SS

and a MacBook Pro laptop or a Dell Latitudelaptop, 3SS.

The traffic generated by Ixia IxChariot in all rate versus range tests was bidirectional, downstream and upstream flows. The expectation was that throughput would decline as the distance and physical obstructions increased between the access point and each client device.

As in the test for maximum LOS throughput, each rate versus range test for each client lasted one minute. Three tests were performed for an average throughput figure.


802.11ac Rate Versus Range

The results of the 802.11ac throughput tests are broken down showing the results on each client. Additionally, the four scenarios with Line of Sight and non-LOS with obstruction are shown.

The maximum theoretical data rate that could be achieved by spatial streamis as follows:

1SS = 433 Mbps

2SS = 866 Mbps

3SS = 1,300 Mbps

The 3SS client device was a Dell Latitude laptop which was also deployed in the 802.11ac LOS maximum throughput test.

The Aruba AP-225 exhibited an advantage in average throughput in all 12 scenarios, three client devices tested individually with four different scenarios.

The largest difference in throughput occurred with 75 feet and two obstructions between each access point and the 1SS Samsung Galaxy. Average throughput for the Aruba AP-225 was 125.40 Mbps or 72.4% better than the Cisco Aironet 3602i at 72.72 Mbps.

The smallest difference was observed with 30 feet and one wall between each access point and the 2SS MacBook Air. Average throughput for the Aruba AP-225 in this case was 406.89 Mbps, a 12.9% improvement over Cisco’s 360.35 Mbps.

Detailed results are shown to the left.

514.9

406.9

325.3

122.0

418.1

360.4

255.4

74.00

100

200

300

400

500

600

5 LOS 30 Non-LOS 75 Non-LOS 120 Non-LOS

Th

rou

gh

pu

t (M

bp

s)

Range (Feet) and Environment

Aruba AP-225

Cisco Aironet 3602i

Figure 3: Rate Versus Range Throughput, 802.11ac 2SS MacBook Air Notebook, Four Scenarios

Figure 4: Rate Versus Range Throughput, 802.11ac 1SS Samsung Galaxy S4 Smartphone, Four Scenarios


207.5 212.8

125.4

62.9

137.4 134.5

72.7

49.2

0

50

100

150

200

250


Th

rou

gh

pu

t (M

bp

s)


Aruba AP-225

Cisco Aironet 3602i

572.3

470.3

336.5

214.2

443.6

363.7

269.4

136.9

0

100

200

300

400

500

600


Th

rou

gh

pu

t (M

bp

s)


Aruba AP-225

Cisco Aironet 3602i

Figure 2: Rate Versus Range Throughput, 802.11ac 3SS Dell Latitude Laptop, Four Scenarios




Figure 5: Rate Versus Range Throughput, 802.11n 5 GHz 3SS MacBook Pro Laptop, Four Scenarios

Figure 6: Rate Versus Range Throughput, 802.11n 5 GHz 2SS MacBook Air Notebook, Four Scenarios

Figure 7: Rate Versus Range Throughput, 802.11n 5 GHz 1SS Samsung Galaxy S4 Smartphone, Four Scenarios

802.11n Rate Versus Range, 5 GHz

In these tests, each access point was forced to utilize 802.11n mode. This was accomplished in the Aruba AP-225 by turning off 802.1ac mode and in the Cisco Aironet 3602i by disabling the Cisco 802.11ac module.

Also, both access points used the 5 GHz radio and turned off the 2.4 GHz band during these tests.

The maximum theoretical data rate that could be achieved by spatial stream is as follows:

1SS = 150 Mbps

2SS = 300 Mbps

3SS = 450 Mbps

The 3SS device was a MacBook Pro laptop.

Aruba AP-225 outperformed the Cisco Aironet 3602i in all 12 scenarios.

There was minimal through-put difference at 5 and 30 feet with the Samsung Galaxy. See Figure 7.

The throughput for AP-225 was 99.4% to 123.9% better than the 3602i in all 75 feet Non-LOS tests.

Aruba AP-225 had a verified average throughput for each device for 120 feet Non-LOS:

41.7 Mbps for the Samsung Galaxy

80.4 Mbps for the MacBook Air

97.7 Mbps for the MacBook Pro

We were unable to record any results for Cisco in the 120 Non-LOS 802.11n 5 GHz tests.

Full results are in Figures 5 through 7.

294.3 293.8

237.8

97.7

259.5

173.7

106.2

NA0

50

100

150

200

250

300

350


Th

rou

gh

pu

t (M

bp

s)


Aruba AP-225

Cisco Aironet 3602i

223.0

180.6191.4

80.4

199.4

160.0

96.0

NA0

50

100

150

200

250

300

350


Th

rou

gh

pu

t (M

bp

s)


Aruba AP-225

Cisco Aironet 3602i



100.4 103.4

79.8

41.7

95.0100.5

38.0

NA0

20

40

60

80

100

120


Th

rou

gh

pu

t (M

bp

s)


Aruba AP-225

Cisco Aironet 3602i



802.11 Rate Versus Range, 2.4 GHz

The access points again used 802.11n mode in this series of tests. They used the 2.4 GHz radio by turning off the 5GHz band.

The maximum theoretical data rate that could be achieved by spatial stream is as follows:

1SS = 72 Mbps

2SS = 144 Mbps

3SS = 216 Mbps

The 3SS client device used here was the MacBook Pro laptop.

The Aruba AP-225 completed its sweep of the rate versus range tests by recording a higher average throughput for all three client devices in all four scenarios.

The advantage for the Aruba AP-225 ranged from a high of 105.1% (45.76 Mbps versus 22.31 Mbps) for the Samsung Galaxy at 120 feet Non-LOS to a low of 13.5% (101.62 Mbps versus 89.52 Mbps) for the MacBook Air at 30 feet Non-LOS.

Full results are in Figures 8 through 10.

Figure 8: Rate Versus Range Throughput, 802.11n 2.4 GHz 3SS MacBook Pro Laptop, Four Scenarios

Figure 9: Rate Versus Range Throughput, 802.11n 2.4 GHz 2SS MacBook Air Notebook, Four Scenarios

Figure 10: Rate Versus Range Throughput, 802.11n 2.4 GHz 1SS Samsung Galaxy S4 Smartphone, Four Scenarios

159.7154.1

129.3

87.0

111.1

89.5

64.7

39.7

0

20

40

60

80

100

120

140

160


Th

rou

gh

pu

t (M

bp

s)


Aruba AP-225

Cisco Aironet 3602i


47.9 48.1 46.8 45.8

41.0 42.038.9

22.3

0

5

10

15

20

25

30

35

40

45

50


Th

rou

gh

pu

t (M

bp

s)


Aruba AP-225

Cisco Aironet 3602i


111.0101.6 101.5

80.287.6 89.5

64.8

40.0

0

20

40

60

80

100

120

140

160


Th

rou

gh

pu

t (M

bp

s)


Aruba AP-225

Cisco Aironet 3602i



Aruba AP-225 Throughput Comparison by Wi-Fi Technology

A byproduct of the competitive testing is the ability to compare average throughput values for the Aruba AP-225.

The comparison consists of all client devices (1-, 2- and 3SS) in four scenarios for the three Wi-Fi technologies utilized in testing: 802.11ac5 GHz, 802.11n 5 GHz and 802.11n 2.4 GHz.

As expected, a number of general patterns became apparent. The most important was that average throughput for the AP-225 operating in 802.11ac mode was higher for all devices in all scenarios than the 802.11n modes,5 GHz.

In five of the 12 sets, the percentage difference was greater than 100%.

The largest individual percentage increase was 130.9% for the 2SS client device in 5 feet LOS, 514.88 Mbps for 802.11ac versus 222.98 for 802.11n 5 GHz.

The smallest individual percentage increase was 41.5% for the 3SS client device in 75 feet Non-LOS. The respective average throughput figures were 336.48 Mbps for 802.11ac versus 237.76 Mbps for 802.11n 5 GHz.

These figures prove the increased throughput provided by 802.11ac over 802.11n 5 GHZ.

For complete results, see Figures 11 through 13.

Figure 11: Comparison of 3SS in Four Scenarios 802.11ac with Dell Latitude Laptop 802.11n 2.4 and 5 GHz with MacBook Pro

Figure 12: Aruba AP-225 Comparison with 2SS MacBook Air Notebook, Four Scenarios

Figure 13: Aruba AP-225 Comparison with 1SS Samsung Galaxy S4 Smartphone, Four Scenarios



572.3

470.3

336.5

214.2

294.3 293.8

237.8

97.7

159.7 154.1129.3

87.0

0

100

200

300

400

500

600


Thr

ough

put (

Mbp

s)


802.11ac

802.11n 5 GHz

802.11n 2.4 GHz

514.9

406.9

325.3

122.0

223.0180.6 191.4

80.4111.0 101.6 101.5

80.2

0

100

200

300

400

500

600


Th

rou

gh

pu

t (M

bp

s)


802.11ac

802.11n 5 GHz

802.11n 2.4 GHz


207.5 212.8

125.4

62.9

100.4 103.4

79.8

41.747.9 48.1 46.8 45.8

0

50

100

150

200

250


Th

rou

gh

pu

t (M

bp

s)


802.11ac

802.11n 5 GHz

802.11n 2.4 GHz


Test Bed Diagram

How We Did It The network architecture provided no radio frequency interference while the access points were under test. Twospectrum analyzers were used to ensure that no outside radio traffic was present on the band used in testing. They wereAirMagnet from Fluke Networks and inSSIDer Office from MetaGeek. In all tests, Layer 3 traffic was generated by IxiaIxChariot Server, running on a Lenovo W520 laptop with 16 GB RAM.

IxChariot utilized the script “High-Perfrmance_TCP.scr” to generate maximum TCP traffic. In the 5 Feet Line of Sight test case, the performance of the Aruba AP-225 and Cisco Aironet 3602i was assessed while handling downstream (unidirectional) traffic. In the rate versus range tests, the performance of both access points was assessed handling bidirectional traffic. The run mode was batch and run time was one minute.

Each access point was managed by a separate hardware-based controller, the AP-225 by an Aruba 7210 Mobility Controller, Version 6.3.0.0 Build 38660, and the 3602i by a Cisco 5508 Wireless Controller, Version 7.5.102.0. The wireless controller governs the connection between Ethernet and wireless, configuration and management of lightweight access points, traffic control and network access policy.

The heart of the test network, an Aruba S3500 Mobility Access Switch, was connected to the Lenovo laptop hosting the server portion of IxChariot as well as to each combination of access point and wireless controller. Each wireless client hosted the client portion of IxChariot.

Each rate versus range test used three different clients, each with a different spatial stream. Testing was conducted in an office building with multiple rooms, partitions, walls and hallways. Steps were taken to ensure the absence of noise, human and Wi-Fi, that could interfere with the unidirectional and bidirectional traffic generated by IxChariot and transmitted by the access points. Each access point was attached horizontally against a dropdown ceiling.

Miercom recognizes IxChariot by Ixia (www.ixiacom.com) as a leading test tool for simulating real-world applications for predicting device and system performance under practical load conditions. Consisting of the IxChariot Console, Performance Endpoints and IxProfile, the IxChariot product family provides network performance assessment and device testing by testing hundreds of protocols across several kinds of network endpoints. IxChariot is used to accurately access the performance characteristics of any application running on wired and wireless networks.

The tests in this report are intended to be reproducible for customers who wish to recreate them with the appropriate test and measurement equipment. Current or prospective customers interested in repeating these results may contact [email protected] for details on the configurations applied to the Device Under Test and test tools used in this evaluation. Miercom recommends customers conduct their own needs analysis study and test specifically for the expected environment for product deployment before making a product selection.


Product names or services mentioned in this report are registered trademarks of their respective owners. Miercom makes every effort to ensure thatinformation contained within our reports is accurate and complete, but is not liable for any errors, inaccuracies or omissions. Miercom is not liable fordamages arising out of or related to the information contained within this report. Consult with professional services such as Miercom Consulting forspecific customer needs analysis.

About Miercom’s Product Testing Services

Report 130916 [email protected] www.miercom.com

Miercom has hundreds of product-comparison analyses published over the years in leading network trade periodicals including Network World, Business Communications Review, Tech Web - NoJitter, Communications News, xchange, Internet Telephony and other leading publications. Miercom’s reputation as the leading, independent product test center is unquestioned. Miercom’s private test services include competitive product analyses, as well as individual product evaluations. Miercom features comprehensive certification and test programs including: Certified Interoperable, Certified Reliable, Certified Secure and Certified Green. Products may also be evaluated under the NetWORKS As Advertised program, the industry’s most thorough and trusted assessment for product usability and performance.

Before printing, please consider electronic distribution

Aruba Networks, Inc. 1344 Crossman Avenue

Sunnyvale, CA 1-408-227-4500

www.arubanetworks.com

Miercom Performance Verified The purpose-built Aruba AP-220 series access point hassignificant advantages which were shown during our reviewof the product. In basic testing, AP-225 model was ableto deliver 768.8 Mbps throughput, well above the minimumof 500 Mbps as required in the specs. When impairmentsand distances were increased, the throughput performancewas maintained at high levels, while using 1-, 2- and 3SSclient devices.

Aruba AP-225 also has built-in gigabit Wi-Fi functionalitythat can be deployed in high density wireless LANenvironments. The AP-225 access point is a noteworthyproduct supporting the 802.11ac specifications.

Based on our testing, Aruba AP-225 has earned theMiercom Performance Verified Certification.

Aruba Networks AP-225

Access Point

device performance and show 51% greater 802.11ac device ...a Samsung Galaxy S4 smartphone, 1SS a...

Documents

Transcript of device performance and show 51% greater 802.11ac device ...a Samsung Galaxy S4 smartphone, 1SS a...