A new benchmarking study of five commercially available LTE smartphones relied on over-the-air testing to spotlight data performance differences among them, and put Motorola Mobility’s Moto X on top in OTA data performance measures.
The independent study was a collaboration of Signals Research Group, test company Spirent Communications and ETS-Lindgren, which produces systems for shielding, measuring and controlling electromagnetic and acoustic energy including RF.
“Once a great-performing chipset is embedded into a mobile device, there are innumerable factors that could result in the smartphone performing poorly with respect to cellular connectivity, including implementation of MIMO antennas,” said Michael Thelander, CEO of Signals Research Group, in a statement. “For this reason we decided to conduct this latest round of benchmark testing over the air, as this is perhaps the best methodology available today to fully characterize device performance.”
Some of the study’s conclusions included:
- Poor-performing smartphones “can dramatically degrade the user experience, make inefficient use of valuable network resources and [increasing] subscriber churn.
- Data speeds were “generally substantially higher with MIMO than without, but the benefits were far less meaningful with less favorable network conditions.”
- The study also found that a smartphone’s protective cover can have a “modest impact on … performance, in particular when using MIMO,” but noted that the research was “far from conclusive.”
“Our study found substantial OTA cellular performance differences in the smartphones tested,” said Thelander. “However, the results show that one device – the Moto X from Motorola Mobility – came out on top by a country mile.”
Which past research studies from Signals Research Group have used traditional lab testing, which did not provide as complete a picture as the OTA testing conducted in an anechoic chamber. The OTA tests exposed each device to radiated signals from probes within the chamber while eliminating external interference that could influence performance.
Spirent’s CS8 LTE network emulator, VR5 channel emulator and DatumLab server were used in the testing, along with ETS Lingren’s AMS-8055 MIMO Single Cluster Environment Simulator. All devices were purchased directly from a mobile operator’s store or website, and no handset manufacturer had prior knowledge of the testing or results, according to the three companies who collaborated on the study.
“OTA test methodology is the only way to thoroughly assess the real-world data performance of LTE devices,” said Bryan Sayler, senior vice president at ETS Lindgren. “Performance is highly dependent on device design, especially with complex antenna schemes such as MIMO. As LTE continues to roll out we believe that the industry will place much greater reliance on testing mobile device performance over the air, utilizing an approach similar to the one used in this important SRG study.”
Nigel Wright, VP of Spirent Communications added that “in benchmarking commercial devices that all used chipsets from a single vendor, we exposed the reality that use of a top-performing chipset does not necessarily guarantee a top-performing mobile device. The industry has a significant and complex challenge to address when validating LTE device performance to ensure a successful commercial rollout.”
The “Chips and Salsa XVII: When Iconic Meets Anechoic” report can be previewed over at Signals Research Group.
