When the ball dropped on New Year’s Eve, three teams from Global Wireless Solutions were testing carrier networks in New York’s Times Square.
It took hours of waiting in line along to get the teams in place to do their ground-level testing, according to GWS CEO Paul Carter. The New Year’s Eve testing is the first of a new series of public benchmarking efforts that GWS has planned, centered around evaluating carrier networks’ performance during specific events.
“We’ve put a lot of focus on OneScore and our customer-centric ranking methodologies, and ranked AT&T the best overall network nationwide. We thought we would do something a bit more fun, let’s say, but also relevant — and try and come up with an event-scoring methodology that people could understand very quickly,” Carter said. “This is the first of what we want to do with many events throughout the country this year.” GWS has dubbed the new event-centered metric its Event-O-Meter, calling it “a barometer for how American consumers experience wireless connectivity when networks are trying to accommodate heavy traffic.”
Carter said that the team’s data from New Year’s Eve generated a picture of how things changed over the course of the day and night in Times Square. Some metrics held relatively steady, and others rose and plunged along with the ball. Overall, GWS gave mobile network performance during the event a score of 77 out of 100 — about a C-level score, Carter explained. Compared to other events that GWS has tested in the past, according to the company, the 2018 Mobile World Congress in Los Angeles had a OneScore of 89, President Donald Trump’s 2017 inauguration ranked at 75 and the 2018 Boston Marathon scored 86.
Carter noted that in the lead-up to the ball drop, there are hours of musical entertainment and time during which the crowd gathers. The GWS teams began testing from about 5 p.m. in the afternoon.
“There’s not necessarily one big rush, as it were,” during that time period, he said, comparing the environment to a sports game where there is an extended period of high crowds and high traffic with breaks in the action. But in Times Square, Carter added, there are many people (an estimated 1 million) packed into a relatively small area, all very close together — a much different case than in a stadium, where traffic might be concentrated around the field but people are also moving through the rest of the building.
But then, he said, “things got pretty hairy around the midnight hour,” when people in the crowd ramped up their social media sharing of the event. Uploading a test video file went from taking about 10 seconds to taking about 16, while photo upload time increased from about 5 seconds up to 9 seconds, he said. After 1 a.m., when the throng had largely cleared out, those numbers fell back to as good or better as they were earlier in the day, Carter added. Interestingly, he pointed out, download speeds held quite steady over the course of the night — it was on the upload side where the network impacts of the spikes in traffic were seen.
Among the other data results:
- Nearly 100% of voice calls made in Times Square were completed successfully.
- Data-related tasks had normal success rates (in the high 90 percent range), even if they took longer to complete
- “Overall reliability of the network was solid and unchanged throughout testing,” GWS said.
GWS’ testing also revealed some of the network features at work to support the boom in traffic. He said that all four national carriers had two-component and three-component carrier aggregation at work, and a couple of them had four-component CA. There was a “fair amount” of 256 QAM modulation, and several carriers had a small amount of 4×4 multiple-input multiple-output at work alongside 2×2 MIMO, Carter noted.
GWS said its three test teams used Rohde & Schwarz’s SwissQual QualiPoc on Samsung Galaxy S9 collection devices to measure the performance of the four Tier 1 mobile networks. GWS collected nearly 13,000 voice and data samples on which it based the event’s score; data collected began at 5 p.m. on December 31 and ended at 2 a.m. on January 1.
