Testing and Quality Assurance to Validate 5G Performance

Executive Summary 

Every 5G device in a cell is connected to the internet or phone network via an antenna built into the cell. With the deployment of standalone 5G networks accelerating in 2021, the cloud will need to reinvent itself. Technology providers want a way to easily and effectively test 5G devices across a set of parameters, including speed, application output, scale, and more, as the world embraces 5G technologies. Wireless service providers will spend and expand their footprint on 5G in the next months, making quality assurance and testing a critical component for competitive advantage, customer satisfaction, and market sustainability. The 5G network has opened up new opportunities for companies as new gadgets will be required to endure the same network capabilities. This will impact testing because all apps will be needed to test on a network-eligible device. Machine learning (ML) or artificial intelligence (AI) will be required in 5G network and service assurance solutions. To analyze the reams of data flowing in from the network in real time, AI or ML will be required, and it must be implemented at every stage of the assurance stack. As a result, the algorithms will be much more tailored to the individual application and will be able to deliver more reliable insights and forecasts about service quality. According to the Spirent 2021 5G Report, as operators accelerated their early 5G rollouts, service provider participation grew by more than 50% year over year (200 by the end of 2021). As market rivalry grew, over half of service provider activities were on service assurance and experience (including competitive benchmarking). Hence, 5G accelerated the deployment of new tests and assurance procedures in 2021.

Essential Types of Quality Assurance Testing

Quality assurance (QA) is a method of preventing errors and flaws in the production process, as well as problems in the delivery of products or services to clients.

In a multi-vendor network context, standards compliance is one of the most important parts of quality assurance and design validation of 5G products. QA testers would need to adjust their acceptance test strategy and confirm that the system was constructed following the 3GPP specification by the associated vendor. Furthermore, interoperability between 5G equipment from different vendors is only achievable if they are both compliant with the same specification and protocol. A 5G User Data Management product, for example, should be created to the 3GPP specification number 29.503, regardless of which manufacturer makes it.

When it comes to the 5G test life cycle, regression testing is very crucial, especially if a company practices DevOps and the Continuous Integration and Continuous Development (CI/CD) model. Each 5G software drop must be extensively regressed, and each iteration must be capable of capturing all types of software and related hardware problems, which must be rectified and confirmed with the highest level of quality in following DevOps software drops.

Customer happiness is heavily influenced by device interoperability. All Android and iOS-based user devices, such as smartphones and tablets, must be certified across the various 5G radio access and core network entities. Because Samsung Galaxy and iPhone smartphones have the largest market share, all generations of these devices must be properly researched for network vendor and original equipment manufacturer interoperability. 5G Wi-Fi calling, simulating a home Wi-Fi router as a cell site tower, must be tested on all iPhone versions.

Load and stress testing are required to ensure that the 5G system can handle the planned traffic load at a high volume. Busy Hour Calls (BHCA) that exceed the system's peak transaction per second (TPS) ensure that the system does not crash or the software does not dump a core. While trying to undertake load and stress testing to identify a 5G system's Key Performance Indicators, open-source tools like J-Meter and seagull, as well as free/trial versions of SOAP UI and Postman, could come in help (KPIs).

5G Testing: Framework, Parameters, and Phases

If the QA engineers create systems for the next generation of wireless networks, mobile app testing organizations will benefit. Due to the lack of cable access, devices must be evaluated via over-the-air (OTA) testing. For example, mobile app testing services firms confront a variety of test issues, including configuration, coverage, and repeatability. New techniques and methods must be used to evaluate 5G components and devices. Standard test architecture and measurement for multi-vendor wireless for 5G networks is critical for moving forward with design, configurations, characterization, production, and validation for hardware and software. Maintaining optimal performance with novel 5G testing methods can help stay ahead of the competition by reducing chamber testing while maintaining test coverage and accuracy.

The International Mobile Telecommunications Vision 2020 defines three scenarios for 5G telecommunications that differentiate it from 4G telecommunications:

• Ultra-reliable, low-latency communications (URLLC)
• Enhanced mobile broadband (eMBB)
• Massive machine-type communications (mMTC)

Agencies are looking into how to use 5G to better mission delivery and business operations, as well as deliver new apps and services that are impossible with 4G technology.

Source: Seragon.com

The 5G testing architecture is separated into four main phases based on the timeframe for introducing 3GPP 5G standards and the availability of 5G equipment and devices from vendors. For example, the first step builds a 5G non-freestanding architecture that uses the current 4G core network infrastructure, while the second phase updates the network core to 5G in a standalone mode. The framework's modular components are divided into architecture, application traffic, spectrum, network, and 5G innovations. Each paragraph offers a description of the test element as well as associated test and measurement equipment, as well as considerations (e.g., protocol analyzer). Not all elements are necessary for all testing; for example, if a real Radio Access Network (RAN) is unavailable, a simulated RAN might be employed. The framework finishes with a description of its security considerations and security metrics that an agency could collect using it. After implementing the framework, an agency could build/lease a test-bed from a carrier-grade equipment manufacturer, perform testing using existing external labs/testbeds (e.g., government lab, university lab, coordination with the Department of Defense), or use a combination of the two.

What are the Parameters Used to Test 5G Networks?

Phases of Testing Capabilities

1. Verification & Validation: Verify virtual network functionalities and network services (5G Lab System Verification & 5G Field Validation) to assure immediate quality and dependability once the network is implemented. Network virtualization, network performance, baseline & benchmarking, and network automation will be the emphasis areas for quality assurance in this phase.
2. Deployment and Activation: Activation of 5G test tools and tools/platforms to monitor network performance is required after successful 5G network and infrastructure deployment. Performance monitoring, advanced fiber testing, radio testing (C-RAN), and compliance testing are the main areas of concentration in this phase.
3. Assurance & Optimization: Verification of virtual network functionalities (5g Lab system Verification & 5G Field Validation) and network services to ensure immediate quality and dependability once the network is implemented. During this phase, the focus is on user experience, IoT assurance, robotic automation, device testing, and user acceptability testing.

Source: VIAVI Solutions

How are Major Companies Coping With the 5G Revolution?

1. EXFO, a Canadian business, launched its Nova Adaptive Service Assurance (A|SA) platform in 2020. Nova A|SA was created expressly to facilitate the delivery of 5G signals and is described as the industry's first service assurance-centric automation platform. The new platform combines data from EXFO and third-party service-assurance solutions, as well as data from relevant layers, domains, services, and data sources, to provide operators with timely data to detect, diagnose, and resolve – or predict and prevent – network-based events that could harm customers. In response to the exponential expansion in network traffic, EXFO created Nova A|SA, which today encrypts over 80% of it (such as OTT video). The tremendous throughput of 5G makes keeping up with it much more difficult for older monitors.
2. Infovista's TEMS Network Testing Portfolio has been updated to safeguard connectivity for traditional network operators who require 5G solutions for testing and improving networks and industries that require assistance maintaining and ensuring application connectivity. TEMS also provides mobile network operators with a number of rapid-support 5G solutions that allow them to walk-test, drive-test, and dynamically assess new mobile 5G devices and networks in real-world scenarios.
3. PathWave Test, by Keysight, is crucial in 5G and mmWave design, verification, and test techniques, expediting the process from system-level modeling to RF/MW circuit co-simulation and analysis. To boost test efficiency, Keysight is working with Nokia on a new software strategy incorporating Artificial Intelligence (AI) and advanced data analytics into the company's 5G base-station manufacturing processes.
4. Viavi provides a broad range of network tests, activation, and assurance equipment and software. With its optical and 5G expertise, as well as an aggressive acquisition strategy, the company has been able to deliver a wide range of solutions, including 5G deployment and activation, the industry's first test suite for O-RAN specifications, fiber test tools, 3D sensing filters for smartphones and cars, and IoT monitoring and assurance.
5. Tutela recently added machine-learning techniques to its datasets, indoor record classification, and cell-site geolocation functionality. In addition, the company is significantly involved in 5G deployments, providing operators with insight into locations with high data demand or congestion so they can make cost-effective decisions.

Trends to Keep an Eye on in 2022

1. Automation is gaining popularity: Network equipment vendors, service providers, and businesses are working to improve their agility and operational efficiency. Automated testing and assurance will become key building blocks across all layers as 5G pushes CI/CD into the mainstream. Use cases like new service activation, continuous monitoring, and automatic troubleshooting will now be smoothly linked with network management systems owing to automation. The ultimate goal is for operations to be completely self-contained. Furthermore, in 2021, a market shift was witnessed as hyperscalers, and cloud providers began to build out their own 5G core infrastructures, prompting them to focus on automated 5G core lab tests.
2. Legacy visibility methods are just insufficient in cloud-native, highly disaggregated, and completely virtualized network environments, according to network operations teams. Individual Open RAN infrastructure components cannot be tested since new service delivery models are required for successful 5G deployments. New services such as 5G, SD-WAN, SASE, and ORAN are also being created, deployed, and optimized for market consumption in fundamentally different ways. Instead, true end-to-end testing and assurance across all layers (physical, transport, network, and service) are required, from a user device to the core network.
3. Hyperscalers will need to achieve demanding and predictable 5G performance criteria when operators transfer functions to the cloud in 2022. Because every program is different, each can produce its own network issues. In the pre-deployment phase, hyperscalers, operators, and network suppliers rely extensively on emulation to reproduce accurate, real-world data center settings to stress test and verify application performance.

Recent Innovations in the Industry

• On 4th May 2021, HARMAN, a Samsung Electronics Co., Ltd. wholly-owned subsidiary focused on connected technologies for the consumer, automotive, and enterprise markets, announced the launch of a new end-to-end 5G Testing Lab for 5G CP devices. This would enable technology suppliers such as device manufacturers, chipmakers, telcos, and others to run a wide range of protocol and functional tests, as well as validate and analyze applications in a true 5G radio environment.  The HARMAN 5G Testing Lab is a neutral environment where technology vendors can verify favorable performance, pre-validate any 5G CP equipment, as well as examine applications across various parameters, third-party integrations, and use scenarios.
• HPE opened a new 5G lab in July 2020, where telcos and suppliers can test and certify the performance and compatibility of key network core operations utilizing a variety of vendors. Although the lab is in Fort Collins, Colorado, clients and HPE partners worldwide can use it remotely. Personnel to manage and maintain the on-site environment, as well as assistance with integration and testing, is given.
• Qualitest, the world's premier AI-powered quality engineering firm, today announced that its 20-year partnership with BT Group, the UK's largest mobile and internet provider, has been strengthened. BT has announced its plans to roll out 5G services across the UK by 2028 and build the country's first fully-integrated network by the mid-2020s. While this generational network change takes place, Qualitest's Managed Test Service (MTS), which considerably widens the scope of the engagement, will help to ensure a consistent, positive user experience for both consumer and commercial clients. Qualitest will handle testing tasks using automation and in-house talent, resulting in considerable cost savings and enhanced value for BT. The company's Qualisense AI suite of testing tools enables data-driven decision-making by providing a 360-degree approach to machine learning.
• Ericsson launched a 5G testing facility and training center in Moscow's Metropolis Business Center in May 2019. A demo studio is available for testing 5G and IoT applications and use cases at the facility. The facility, according to Ericsson, will allow Russian operators to test 5G and IoT technologies and standards that have been suited to Russian requirements. On a live network, service providers would be able to test 5G use cases and consumer devices, as well as solutions like fixed wireless access, spectrum sharing, Private LTE networks, and autonomous car remote control.
• In January 2022, VVDN Technologies announced the launch of a new end-to-end 5G testing lab in India for 5G open radio access network (ORAN) based resource unit (RU) devices. The 5G lab will allow technology providers such as telecom device manufacturers and system integrators to conduct protocol and validation testing for ORAN, as well as radio conformance testing (RCT) and interoperability testing in a real 5G radio environment.
• Intertek, a major provider of Quality Assurance to industries throughout the world, announced the debut of assurance and testing services for 5G equipment in August 2019. Manufacturers from a variety of industries will be able to test and validate unique products for the next generation of IoT devices and mobiles owing to the solutions, which were made possible by staff education and investment in new testing hardware, technology, and software.
• Rohde & Schwarz will demonstrate its automotive 5G and C-V2X test solutions for development in the lab and field verification to assure the performance and reliability of vital vehicle communications at MWC Barcelona in February 2022. Additional focus applications include Rohde & Schwarz's UWB test solutions for R&D, certification, and production, as well as flexible GNSS simulation using the new R&S CMX500 one-box tester, which covers the complete development cycle from chipsets to TCUs and entire vehicles. The R&S CMX500 has a wide range of testing capabilities, including LTE, 5G NR FR1 and FR2 deployments.

About the Author

Roshni Bajaj works for Knowledge Sourcing Intelligence LLP as a Market Research Analyst. Roshni specializes in data collecting, analysis, and translation into actionable insights that give businesses a competitive advantage. Visit www.knowledge-sourcing.com to read more articles by Roshni and to learn more about a variety of global and regional markets.