As mobile network coverage demands move indoors and into large venues, carriers are stuck with the conundrum of how to cost-effectively test and monitor the performance of distributed antenna systems, particularly through the lens of the customer experience.
AT&T developed its own solution, the EchoBot. DAS installations have been a major part of AT&T’s three-year infrastructure investment program called Velocity IP. Shane Elliott, manager with AT&T’s antenna solutions group and the inventor of the EchoBot, came into the ASGÂ in 2010. He said that in the
course of AT&T’s DAS builds across the country, the company began putting more intensive focus on the performance of its network designs and how that design translated to customer experience.
Typical assessment of DAS, he said, meant using various probe vendors and third-party testing offerings, mostly in the form of people on the ground testing during live events. But the economics of boots-on-the-ground testing across venues ranging from professional stadiums to college stadiums around the country, spanning many events over the course of the year, simply weren’t feasible.
“It really didn’t allow for us to meet the economics to continue to do all of that, because of the expense,” Elliott said. “We had to measure the customer experience and meet the economics, and do it on a large scale.”
Enter the EchoBot.
Elliott developed the device to solve the issue of testing and monitoring in distributed antenna systems. His brainchild consists of a 10-inch-by-10-inch box, large enough to hold a couple of devices: a typical Android handset, and two other wireless modules in order to cover LTE, GSM and Wi-FI. A bot is placed in each sector of a DAS – typically 15 to 20 sectors per venue – and it generates traffic similar to a typical end user.
Elliott said that initially, the EchoBots were hand-assembled devices put together within the ASG as a proof of concept. Those initial devices were deployed in 2012, and eventually that design became more sophisticated and standardized, with a metal cover and external antennas deployed as the EchoBot 2.0 earlier this year. The bot itself is part of an orchestrated monitoring software system – Echo – that provides a central architecture for management and the ability to react quickly to data coming from the bots.
EchoBots can operate separately from the main architecture, Elliott said, and will simply continue to run the latest programmed queries. He noted that although traditional network statistics on the radio frequency environment are useful, the EchoBot allows a view into the customer experience – whether YouTube streaming is working, or whether a user can post a video or send a text message.
The EchoBot is already in use at a handful of venues, according to AT&T. At the recent LTE North America conference in Dallas, Jim Parker, senior manager with AT&T’s antenna solutions group, said the technology was instrumental during the 2013 Super Bowl in New Orleans. The DAS, he said, was tuned before the game and AT&T was able to respond quickly during the event when it realized that the DAS was encountering large amounts of interference from the macro network. Parker described the bots as having both cellular and Wi-Fi capabilities, which can be activated via scripts to generate calls, text messaging, uploads and downloads.
“It’s been very eye-opening in regards to what we were able to see before an event, as well as during the event,” Elliott said. “As you can imagine, in a big game, you have all your customers in the seating area, then during halftime they’re moving to the concourse area and the concession stands.” That movement drives the need to move capacity from the concourses to the bowl and back again over the course of the event, he added.
DAS installations are handling stunning amounts of data traffic, even with Wi-Fi often available for offload.
During the 2013 Super Bowl, AT&T reported 3G and LTE usage totalling 388 GB, plus 73,000 voice calls during the game. Those numbers seem almost tame compared to those of 2014’s Super Bowl, when  usage skyrocketed even more, to 624 GB – the highest data usage the company had ever seen from a one-day sporting event, according to a company blog post. According to AT&T numbers from five DAS installations at professional baseball stadiums during the playoffs, the average cellular network usage for each game was 352 GB, up 86% from the regular season.
At AT&T Park, the company said, its DAS and Wi-Fi systems handled 2.1 TB of data in a single game earlier this year, with more than 477 GB on the DAS and more than 1,626 GB on Wi-Fi.
Elliott said that the primary challenges of developing the EchoBot were computing power and power consumption. ARM-based computing allowed the device to be sufficiently small, he said, while he sought to avoid the standard rack equipment design requiring 120 volts of power. Instead, the EchoBots rely on power over Ethernet so that they can be installed without the need for additional power outlets.
AT&T has pondered whether to make the EchoBot available to other companies for DAS monitoring, Elliott said, but for now it remains a proprietary technology.
