CPE Testing in Three Key Areas

Wi-Fi Testing for a Major Network Operator

Wi-Fi testing can be problematic for operators whose lab environment is oversaturated with CPEs, rendering Wi-Fi performance testing impossible. When performing Wi-Fi testing for a major international network operator, a portion of the engagement was at the operator’s premises and the majority at Resillion’s labs.

This meant we:

• could provide a test environment with perfect control over the Access Points
• can use different types of testing environments (including an apartment, terraced house – and others – in addition to the Resillion Lab)
• had access to Resillion’s extensive collection of devices for interoperability testing
• could leverage our team of experts in all aspects of Wi-Fi device testing including access points, repeaters, Mesh networks, all variations of Wi-Fi clients, and many more.

The testing was focused around characterising the maximum Wi-Fi throughput supported by the Wi-Fi Product under test, in up- and downstream using different antenna positioning, varying distance between access point and client, different WAN/LAN configurations, single & multi-client usage, etc.

We also verified:

• Basic Settings (Radio, SSID, Channel settings, Transmission mode, Bandwidth)
• Security (Pre-Shared Key, Security protocol encryptions: WPA, WPA2, WPA3)
• Access Control (MAC restrict mode for each Wi-Fi interface)
• WPS
• Band Steering
• Wi-Fi Event Logs
• Time Control
• QoS

The client was delivered reliable and reproducible test results from a set of environments similar or identical to those of the end users. This enabled the operator to deliver the best possible Wi-Fi experience for their subscribers, preventing a high customer churn due to a poor customer experience.

End-to-end Voice Service Quality Testing

Our client, a large European network operator, asked us to provide objective end-to-end voice quality testing. Working with the client, we aimed to go beyond a simple subjective quality check of the audio channel by a tester. We also wanted to go further than a Mean Opinion Score (MOS) computation by taking network impairments such as packet loss and jitter into the equation (using the E-Model as defined in ITU-T Rec. G.107). The net result is that we were able to create a comprehensive set of voice quality tests appropriate for this operator.

The tests comprise objective MOS measurements, undertaken in accordance with PESQ (ITU-T Rec. P862), augmented with an explicit evaluation of echoes (echo return loss), Gateway input and output attenuation, frequency distortions, DTMF recognition, etc.

Voice Quality measurements are first conducted under ideal conditions: in impairment-free lab conditions, and then with simulated network impairments (including packet loss, jitter and jitter variations) introduced to check the degradation of the voice quality under real network conditions.

At the conclusion of the tests the client was provided with reports that included a detailed measurement report as well as graphical interpretations to help quickly assimilate results and determine service quality degradation root causes. An unacceptable echo, for example, can be traced back to an incorrect input or output attenuation or insufficiently working echo cancellation.

In summary, the testing comprised:

Voice Quality Measurements

• Mean Opinion Score (MOS) based on PESQ LQ
• Echo return loss, I/O attenuation, frequency distortion
• Voice quality impact of network impairments (packet loss, jitter, jitter variation)

Analogue (a/b) interface measurements

• Voltage
• Ringing
• DTMF recognition
• Tone generation
• Hook flash recognition

Cyber Security Testing for Modems and Routers

A client, a large ISP in the Netherlands, had experience of conducting cyber security testing internally. However, for the deployment of some new Gateways (ZTE and Huawei DSL modem and routers) they were looking for an external partner to manage this process, who could bring a fresh perspective on the testing. The client was aware of the work that Resillion had previously undertaken for other clients regarding cyber security testing for end-user devices. They requested that we compile a test strategy and perform all relevant tests.

Several services and protocols were included in the scope of the testing: PPP over ATM and Ethernet, DHCP, Telnet, UPnP and TR-069 remote management.

The project specifically addressed questions such as:

• Are there ways to obtain administrative access to the devices?
• Could UPnP/TR-069 flaws lead to malware infections (such as botnets)?
• Do the devices show signs of backdoors or hidden features?

The testing team took the approach of regarding the devices as black boxes: no OS level access was assumed, and no documentation was provided.

Testing was both manual and automated with VoIP traffic manipulation. Additionally, we applied fuzz testing and software reverse engineering.

Outcomes

The client was provided with a comprehensive test report, describing the test coverage, major and minor findings. All findings were given with risk classification, potential impact, probability, and suitable recommendations for mitigation actions. The recommendations were given per finding, but also on a high level since looking at the wider picture can highlight trend patterns, such as design issues in the network or problems with the patch and vulnerability management process.

Based on our recommendations, the operator was able to prevent some major security issues from slipping into the end customer product.

For consumers and end users to enjoy the best connectivity experience, they need the best performance from their Customer Premises Equipment (CPE): the Voice and Data Gateways used to connect their home to the broadband network provide by their ISP, Cable or DSL network providers. Those consumers are also putting increasing pressure on the Gateway thanks to video streaming, online gaming, and large file transfers.

Ensuring that gateways are up to the task has never been more important.

Read more about Voice and Data Gateways here.