Reader Forum: Wireless network performance metrics – are you measuring reality?

Editor’s Note: Welcome to our weekly Reader Forum section. In an attempt to broaden our interaction with our readers we have created this forum for those with something meaningful to say to the wireless industry. We want to keep this as open as possible, but maintain some editorial control so as to keep it free of commercials or attacks. Please send along submissions for this section to our editors at: dmeyer@ardenmedia.com or tford@ardenmedia.com.
If you ask any RF manager at a wireless operator how their network performs, you will likely be presented numerous metrics such as drop call rate, accessibility and coverage metrics that together yield statistics in the range of one to three problems for every 100 calls. The reality from a consumer standpoint is however a tangential shift from the network-based reported metrics. Data from a recent study showed that when viewed through consumer lenses, problems experienced are in the range of five to 10 for every 100 calls, and operators are missing as much as 40% to 60% of problems when focusing only on network-based information.
Network-based metrics may have served well historically; we should challenge whether these same metrics are valid for today’s wireless operators given two significant changes: The evolution in complexity of devices leading to device-detected problems and changes in actual consumer usage behavior leading to user-detected problems.
The claim that up to 40% to 60% of consumer problems are missed by network statistics was based on a recent study conducted by NIL-Labs with a panel of wireless consumers. Device-based agents were deployed for the users to capture all call failure events and associated surveys were triggered after each call to gauge consumer perception of service quality.
The results led to the conclusion that device-detected problems with no relationship to the network contribute 15% to 30% towards poor consumer experience. An explanation for this phenomenon is linked to evolution of the actual consumer device. These terminals are no longer just wireless phones; they are mini-computers with complex and evolving mobile operating systems tied to ever changing embedded and cloud-based applications. This presents a lot of areas for failure or integration glitches. Hence it is realistic to anticipate that similar problems and frustrations experienced with PC’s related to battery, OS hanging, apps hanging, processing speed, memory shortage, etc., will make their way to smart phones.
Another interesting revelation from the study, captured via the associated survey agent, was the problems that are only user-detected and reported, and contribute 15% to 30% towards poor consumer experience. Change in consumer usage behavior is a big driver. One of the important findings was the extent of poor quality experienced by consumers and how they reacted. There was a large disparity between users that stated they experienced quality problems via the survey verses actual quality problems that led to a failure as detected by the agent. The explanation of this disconnect may be related with how network managers are compensated on their bonuses and the effect this has on how they set parameters in the network in order to minimize network-based recorded drops. It is possible to set call quality timers high enough, so that the call will not drop even if the speech quality is unintelligible. The effect this has is the consumer terminates the call herself, and the network records that call as a normally terminated call – everyone is happy, except the consumer. The NIL-Labs dataset sample also showed almost half of the calls to be placed indoor (at home or work), which is an environment not easy to simulate and test for RF engineers. The challenging radio environment adds to the quality perception vs. reality disconnect. Drive-testing, an existing method to simulate and uncover consumer problems is done only on the roads, and is unlikely to uncover these problems. This data justifies the trend that consumers and businesses are increasingly relying on wireless services as an alternative to fixed line at home and work.
In conclusion, a few thoughts and noted trends are offered for solutions for operators to contemplate to accurately capture, analyze, make fact-based decisions and take action to improve consumer satisfaction:
1. Invest in device-agent based CEM solutions that provide true insight into end user experience. The CEM solution needs to be integrated to drive customer-centric decision making within management, marketing, customer care and network operations to realize full benefit.
2. Make sure you understand how your customers value the benefits your network provides – connect reality and perception. This will help focus capital expenditures and operating expenditures spend, and ensure relevant cross-selling promotions.
3. Ensure smart phone’s, tablets and other wireless consumer devices are well tested through lab and field testing. Available technology ensures testing can be automated to minimal negative impact in product launch schedules delays or associated increased testing costs.
Developing a consumer-centric operational mindset needs to start from the top at an executive level. Technology has evolved where it is now possible to evolve traditional CRM platforms to CEM platforms, and let consumer-centric data drive important decisions in the areas of customer life time value improvement, churn management, customer acquisition and capex and opex spend.
Sanjay Ambekar is the SVP and Chief Client Partner at Nil-Labs and can be reached for comments or questions at sanjay.ambekar@nil-labs.com.