Network benchmarking

For benchmarking, sophisticated multi-channel tools such as Focus Infocom's DMTS and XGMA, DingLi Communications' Pilot Fleet, Ascom's Symphony, Rohde & Schwarz-SwissQual's Diversity Benchmarker, Keysight Nemo Invex II, Aaronia RTSA Suite or RantCell application based QoE drive test benchmarking tool are used to measure several network technologies and service types simultaneously to very high accuracy, to provide directly comparable information regarding competitive strengths and weaknesses. Results from benchmarking activities, such as a comparative coverage analysis or comparative data network speed analysis, are frequently used in marketing campaigns. Drive testing to gather network bench-marking data is the only way mobile network operators can collect accurate competitive data on the true level of their own and their competitor's technical performance and quality levels.

Optimization and troubleshooting

Optimization and troubleshooting information is more typically used to aid in finding specific problems during the rollout phases of new networks or to observe specific problems reported by consumers during the operational phase of the network lifecycle. In this mode drive testing data is used to diagnose the root cause of specific, typically localized, network issues such as dropped calls or missing neighbour cell assignments.

Service quality monitoring

Service quality monitoring typically involves making test calls across the network to a fixed test unit to assess the relative quality of various services using Mean opinion score (MOS). Quality monitoring focuses on the end user experience of the service, and allows mobile network operators to react to what effectively subjective quality degradations by investigating the technical cause of the problem in time-correlated data collected during the drive test. Service quality monitoring is typically carried out in an automated fashion, using devices that run largely without human intervention carried in vehicles that regularly ply typical drive testing routes such as garbage collection vehicles, taxis or buses.

Drive testing can be conducted at any time on a live network and very rarely will there be any network intrusion.

Classification of drive testing based on use cases

• Single site verification (SSV) or Single Cell Function Test (SCFT)
• Multiple site verification (MSV) or cluster drive test
• Operator benchmarking drive test (Market level drive test)

Single site verification (SSV) or Single Cell Function Test (SCFT)

SSV drive test or SCFT drive test is all about performing 5G/4G/3G drive test analysis on a particular site (Example newly installed cell tower) and this is usually performed by drive testing or by walk testing around the site.

In site level testing, site is ready for performing SSV testing after completion of engineering, installation and integration and no active alarms are observed. The main aim of SSV testing is to validate functional performance of the site and identify/flag workmanship issues, product issues & provisioning issues before turning the site on for commercial users.

Multiple site verification (MSV) or Cluster drive test

As the name suggests, network tests when performed on a group of cells is termed as a ‘Cluster drive test’ or ‘MSV’.

Initially, in traditional drive testing, network testers used to drive along target routes to collect information regarding coverage data through various iterations and field tests with cellular RF drive test equipment. But, as telecom businesses are expanding, it is cumbersome to refine expanded networks in terms of size, capacity, and number of users. With a cluster drive test, operators can perform network tests by taking a group of cells and deploy it in a particular location and investigate the network accordingly.

Cluster drive test is executed when the network is in active mode (i.e. providing service to customers) and inspect the interference amid two cells and handover taking place or not. Network parameter details like Drive Route, Quality plots of RSRQ, SINR, PUSCH, Coverage plot, Download and Upload throughput is accumulated. Operators use this data to get output which is further utilized to optimize the mobile network and deliver efficient service to users.

Multiple site verification or a 4G LTE drive test must have measurements of radio parameters such as RSRP, RSRQ, SINR, PCI, EARFCN, and WCDMA drive test parameters like RSCP, RSSI, ECNO, PSC, UARFCN etc. at least at basic level to identify primary network issues.

Operator benchmarking drive test (Market level drive test)

In this test, comparison is done in terms of number of call drops, data throughput, and other key KPIs on 4G / 5G networks between various network operators and analysing who is providing the better user experience. Usually market level drive testing is performed by operators themselves/ independent bodies/telecom regulators. For example, telecom regulators from various countries constantly perform market level drive tests to identify under-performing operators whose user experience KPIs are not meeting regulatory requirements.

‘Market level drive test’ analysis is carried out based on a particular market or an entire geographical area with the use of 5G, 3G, CDMA, LTE drive test tools. Operators conduct drive testing on an area to analyse user experience with respect to other cellular service providers. Market level drive testing can identify their strengths (e.g., high data throughput compared to competitors) and this provides them an opportunity to up-sell their services in advertising campaigns.

In recent times, 5G deployment across the globe has increased the demand for 5G drive test parameters such as 4x4 mimo, massive mimo, etc., and now 5G NR to a must-have feature on a RF drive test tool.

Typical features of RF drive test tools

• Drive test solution must support capture of GNSS information along the drive route.
• Gapless full frequency recording
• Support of 2G, 3G, 4G LTE and 5G NR measurements.
• Data testing such as HTTP, latency and FTP speed tests.
• Voice and SMS testing.
• Video streaming testing support on mobile network.
• Frequency scanning function for all RAN technology.
• Layer 3 and Layer 2 messaging capture capabilities if RF deep diagnosis information.
• Script based call and data session manual and automatic.
• RF tool should be portable, measurement software on smart phone will ideal.
• Real-time upload of test data to post-processing tool.
• Post-processing tool supporting analysis of data on map-based view.
• Post-processing tool providing graphical and tabular reports for various features of GSM / UMTS/LTE related performance of the network.
• Post-processing software tool working on PC or web-based interface.
• Export of excel, maps or PDF based reports.
• 5G MIMO measurements to verify during the SSV stage.

Types of RF drive test tools currently available in the market

RF Spectrum Analyzer - A device used to measure the entire radio spectrum concurrently.

RF Scanners - A device used to measure radio frequency (RF) of multiple operators concurrently.

RF layer 2 and RRC layer 3 capable tools - These are required for deep analysis on 4G and 5G networks and are usually expensive.

App-based RF measurement tools - App-based RF tools are compatible on Android smartphones and can be used to conduct 3G, 4G, 5G drive tests. Mass-scale deployment of such apps is possible due to low cost and compatibility of Android phones. There are solutions on the market that have big data capabilities for analysing huge geographical test data received from app-based RF tools.^[2]

Voice quality measurement - Such tools are used for assessing voice quality (MOS, POLQA), echo minimization, and reduction in noise while permitting easy speaker recognition.

Load Generator – Load generators are deployed to generate high voice and data traffic into a network and test its ability to handle high-level congestion/traffic in a network.^[3]

Voice quality measurement – Such tools are used for assessing voice quality (MOS, POLQA, sQLEAR), echo minimization, and reduction in noise while permitting easy speaker recognition.^[4]