Written by Meik Kottkamp | September 1, 2023
In the round trip time (RTT) measurements, a mobile device initiates the process by sending a signal towards a server connected to the cellular network and then back to the device. Industrial applications that use private 5G NR networks require both low latency and high service stability (high connection availability and reliability). Requirements vary and range from 99.99% to 99.9999% service availability. Note that 99.9999% service availability equals 31 seconds of downtime per year and raises the question as to whether testing can verify such requirements. This article explores the interactivity test as a possible way to meet such stringent testing requirements.
Test method and solution to determine service availability
The test solution uses the commercially available interactivity test with 5G STS or Qualipoc Android. The setup below illustrates the measurement in a 5G smart factory. RTT measurements with interactivity test are initiated by the user device, e.g. a 5G module connected to the 5G STS:
The interactivity test comes with a wide range of predefined traffic patterns. Note that the TWAMP-based solution can vary packet size, packet frequency and delay budget. The delay budget is the acceptable delay for data packets, those arriving outside the time in the budget are labeled “discarded” in the analysis. Details for certain traffic patterns can be seen in the following.
Method for determining service availability
Certain use cases can have very high service availability requirements. The table below provides concrete examples from 5G-ACIA, ZVEI or ABI research.
Note that a 99.9999% requirement equals 31 seconds of downtime a year, raising the question of whether such stringent requirements can be tested in a reasonable amount of time. Fortunately, interactivity tests send out an enormous number of packets at very short intervals (0.77 ms to 16.1 ms) so that even stringent requirements can be verified in line with the calculations below.
Multiple sessions (M) can be run in sequence while multiple packets (N) are transmitted during one session within the observation period.
Figure 2: Multiple sessions (M) can be run in sequence while multiple packets (N) are transmitted during one session within the observation period
Different applications tolerate a different number of packets, which are lost or arrive too late. In our analysis we distinguish four cases. If 99.99% service availability is required (SAreq), a service will fail because a (1) single packet is lost or fails to arrive in time (2) two consecutive packets are lost or delayed or (3) four consecutive packets are either lost or arrive later than specified in the time budget. Error cases (E) simply describe the case, if the latency for each packet is too high relative to the specified delay limit based on whether just a single packet or multiple packet delays can be tolerated. The following formula determines whether the service availability requirement has been met:
Accordingly, a specific number of error cases (E) will lead to a service availability rate (SA) as per
Results
We tested a private 5G NR SA network at our plant in Teisnach, Germany. Rohde & Schwarz operates this private 5G SA network in the n78 spectrum band with 80 MHz bandwidth (3.72 to 3.8 GHz). We applied the long traffic pattern to our measurements to transmit/receive 4500 packets within 30 seconds. We further configured our test so that a single test was run every 10 minutes over a total period of 24 hours. 607500 packets were sent during our one-day observation period.
The following picture illustrates the measurement results taken in mid-April 2023. First note the very stable performance over the day even though the latency values are sometimes quite high.
Let’s take a closer look at the absolute values using the method for determining service availability described in section 2. The median RTT for the day is 16.39 ms. Note that latency optimizations were not applied to the 5G NR SA network and none of the 3GPP Rel16 features, which are specifically designed to support URLLC, were implemented yet. The device under test was a commercial high-end smartphone running the Qualipoc Android interactivity test solution. Depending on the requirements, we assumed 40ms RTT for moderate use cases with a tolerance level at the application for service availability of better than 99.99%.
Table 3: The absolute values using the method for determining service availability described in section 2
To verify 99.9999% service availability and see at least 10 error cases (single packet outside the delay budget) during measurement, a total of 10 million packets would need to be sent. Using the long traffic pattern as a baseline (4500 packets in 30 seconds), total testing time would exceed 18.5 hours per day without any pause between sessions. Note that multiple parameters influence required overall test time, such as the traffic pattern (how many packets are sent over a time period), the application of interest (how many consecutive packets may be lost/delayed) and statistical certainty (the minimum number of errors to be measured). However, reviewing the sample calculations in this section, we find that even tough requirements only need days of testing.
Conclusions
5G NR performance measurements remain essential when setting up and operating commercial networks, specifically when private/local networks must meet very low latency requirements. The interactivity test combines testing round-trip latency, packet delay variation and packet error rate in a single test. The inbuilt flexibility for various traffic patterns lets the solution verify service availability in a relatively short amount of time, even when very stringent service availability requirements apply.
Read more about interactivity test in part 1 - 8 of this series and download our white paper "Interactivity Test"
Interactivity test: QoS/QoE measurements in 5G (part 1)
Interactivity test: Concept and KPIs (part 2)
Interactivity test: Examples from real 5G networks (part 3)
Interactivity test: Distance to server impact on latency and jitter (part 4)
Interactivity test: Impact of changing network conditions on latency and jitter (part 5)
Interactivity test: Packet delay variation and packet loss (part 6)
Interactivity test: Dependency of packet latency on data rate (part 7)
