Fickle Quality of Mobile Data Roaming
From 12 June until 13 July Brazil hosts the most followed sports event in the world, the FIFA World Cup. Although many travel to Brazil to follow the matches and experience the essence of football at first hand, most fans support their teams from their home countries. In The Netherlands, for instance, and especially in Amsterdam it is very popular to gather around big screens in parks and squares, or fill the pubs around the canals to cheer and support the home team in a friendly atmosphere.
Big sports events often generate a tsunami of mobile traffic as fans share experiences and watch video replays online with smartphones. Mobile networks are in a true stress test. To understand the traffic peak impact on customer experience, Omnitele conducted a football-flavoured mobile data benchmark of Dutch mobile operators in typical football watching locations of Amsterdam. All three operators (KPN, T-Mobile and Vodafone) were tested in 38 crowded pubs and parks with big screens. Test cases included web browsing tests in 7 local and international websites closely related to WC2014 (FIFA official website, telegraaf.nl sport news, etc.) File transfer downlink bitrates were also measured to obtain deeper visibility into network performance dynamics.
Although T-Mobile’s average bitrate (12.8Mbit/s) was 45% higher than Vodafone, the results show that in fact, Vodafone provided the fastest web browsing experience in most locations (39%) and KPN the shortest average www page download time (5.5s). The results clearly underline the eroding importance of bitrate in mobile networks with double digit Mbit/s.
For competitive positioning, Omnitele concludes that the three operators offer practically equal web browsing customer experience. Only marginal differences are observed. Looking at the absolute values, the results still leave a desire for improvement. Average web site waiting time longer than five seconds is not extremely good.
For years, operators have relied almost solely on bitrate as the single KPI to measure, benchmark and optimise their networks for mobile data. But in modern mobile networks the bitrates are often so high that in many cases there are other factors imposing more limits on mobile data customer experience. For some use cases, e.g. VoIP, high bitrate is not required at all, and sometimes not even a very high bitrate allows for acceptable customer experience. Thus, true customer oriented test cases and KPIs are imperative for modern network optimisation.
For customer centric optimisation projects, special measurement tools are required. The tooling should allow for simultaneous recording of customer oriented service quality indicators and network performance KPIs in be-the-customer fashion. By cross-correlating the network performance and customer experience KPIs, the optimisation engineer can truly understand the network dynamics and focus the optimisation work where it is needed and where improvement is available. This measurement campaign was conducted with a combination of Anite’s recently launched Nemo Observer measurement system and Nemo CEM. The tooling facilitates full radio interface parameter recording while conducting customer experience tests, and hence gives practical means for mobile subscriber customer experience optimisation.
Download full paper in PDF format: WC2014-customer-experience-1-pager