Editor’s note: This article is a follow-on to “5G Energy Efficiency Metrics, Models and Systems Tests” which was published by RCR Wireless News on August 28, 2024
Introduction
The Aether SMaRT-5G initiative (originally started by the Open Networking Foundation (ONF)), is focused on research and development of innovative energy measurements and modeling for 5G mobile networks. The project also includes demonstration and validation of advanced energy savings techniques in collaboration with a multi-vendor/operator ecosystem.
The National Telecommunications and Information Administration (NTIA) of the US Department of Commerce has been supporting the SMaRT-5G initiative with a research grant through their Public Wireless Supply Chain Innovation Fund (PWSCIF). The $2M grant is funding SMaRT-5G in collaboration with Rutgers University’s WINLAB to research, develop, and validate accurate and effective test methods and to create metrics and models to measure the energy efficiency of 5G network components as well as the effectiveness of end-to-end Open RAN based energy optimization strategies.
The platform we have created for this purpose is called POET: Platform for O-RAN Energy Efficiency Testing. Details of POET can be found in a previously published RCR Wireless article.
We have adopted different approaches for power measurement, which is the first and the most fundamental step in the project. Just to recap, we use a multi-pronged approach for this purpose.
(a) PDU: The power, current, voltage supplied to PNFs and servers is obtained by regularly querying the Power Distribution Units (PDUs) (nominally every 15 secs), and exporting this data to Prometheus/Grafana. This provides a critical ground-truth measurement of power consumption.
(b) IPMI: Power and environment variables reported by the servers are also monitored using the Intelligent Platform Management Interface (IPMI). Queries are made to the server Baseboard Management Controller (BMC) and exported to Prometheus/Grafana.
(c) Scaphandre: We deployed the Scaphandre open-source energy monitoring functionality on bare-metal servers running CU/DU software. Scaphandre measures process utilization (based on Intel Running Average Power Limit (RAPL)) and estimates the power consumption using an estimation model.
(d) Kepler: The Kubernetes deployment in the testbed uses Kepler exporters on each server sending power metrics to Prometheus/Grafana. Kepler is an open-source energy monitoring functionality for Kubernetes systems. It measures node and container utilization (based on Intel RAPL) and estimates the power consumption using an estimation model.
Though PDU gives the ground truth in terms of energy consumption, deploying PDUs in a network and collecting data is expensive and laborious. Furthermore, total energy consumption alone is insufficient to achieve the desired goals of energy management efforts, hence granular and detailed energy consumption measurements are required. For instance, when a disaggregated RAN (such as the O-RAN architecture) is considered, it is important to know how energy is consumed as a function of traffic load by RU, CU, DU and Core, along with infrastructure overhead which houses virtual network functions. Therefore, if O-RAN interfaces can provide energy consumption information with sufficient details and granularity, from which various energy efficiency KPIs can be derived, that would be ideal.
However, the current scope of O-RAN interfaces is inadequate for this purpose, and we have to rely on other feasible and economical approaches to supplement O-RAN interfaces, to begin with. Bearing this in mind, other approaches like IPMI, Scaphandre, and Kepler are highly relevant, and correlating data collected via these methods is very important. Efforts are now underway to collect and analyze data from POET along these lines. Preliminary results were recently published as a technical paper and presented at the RitiRAN conference by project participants from the Linux Foundation’s Aether SMaRT-5G project in collaboration with WINLAB and Cognizant. This article summarizes the key results in the paper.
5G network energy consumption – key observations
We have observed that PDU and IPMI readings track well, and PDU readings typically vary around 5% above IPMI readings, depending on the load. Thus, IPMI readings can be calibrated to estimate PDU readings. This is valuable in the context of energy consumption of cloudified and virtualized network functions. It is also noteworthy that IPMI readings are quantized where the readings are reported in steps, while PDU readings are continuous. This should also be considered in the calibration/modelling efforts.
We used Kepler to get energy consumption of Kubernetes containers/pods, as an attempt to measure the energy consumed by Cloudified Network Functions (CNFs). Kepler uses models based on CPU utilization to estimate energy consumption, and the models can vary for different CPU/server families which can be tweaked to improve estimation accuracy. The current results are by using open-source Kepler models (as-is) and we plan to study methods to improve accuracy of models. Understandably, Kepler readings are significantly below PDU/IPMI readings since they do not capture non-CPU-related energy consumption, which needs to be modelled and calibrated. Nonetheless, we observed that Kepler readings track the PDU and IPMI readings quite well, with respect to total energy consumption.
It is important to see how the energy consumption is for various virtual network functions, namely Core, CU and DU, as reported by Kepler. We did some TCP tests, first with one UE and then with two UEs. We made the following observations:
1. The power consumption of the 5G Core, O-CU, and O-DU in a Kubernetes deployment can be separately estimated using Kepler. The total consumption of the Core NF, and CU and DU containers are only a part of the energy consumption of the total O-RAN system due to significant energy consumption from other Kubernetes system functions.
2. The effect of load with respect to energy consumption is the most on DU as opposed to Core and CU.
We also looked at the energy consumption reported by Scaphandre (bare metal), and compared it with Kepler, PDU, and IPMI readings. Specifically, we used Scaphandre to get energy consumption of O-RAN NF (OAI) and other processes. Also, like Kepler, Scaphandre also uses models to estimate energy consumption based on CPU utilization, and we plan to study methods to improve the accuracy of models.
Our key observations are summarized below:
1. Scaphandre readings tracks well with respect to Kepler, PDU and IPMI readings
2. The power consumption of the OAI O-RAN system could be separately estimated using Scaphandre. The total consumption of the OAI O-RAN system is significantly below PDU/IPMI readings since the Scaphandre estimates do not capture non-CPU related energy consumption, which needs to be modelled and calibrated
Performance measurements
One important aim for this R&D effort is to correlate energy consumption to network performance and to derive KPIs, connecting both these aspects. Therefore, performance measurements are also important in this context. We have been adopting various approaches for this purpose, such as:
1. End-to-end measurements (for example, data volumes and throughput)
2. O1-based KPIs (we used the telnet-based OAI DU O1 solution to get uplink/downlink throughput, and downlink PRB load and expect more support for O1 in future tests).
3. E2-based KPIs (Several KPIs available over E2 interface, e.g. E2-SM KPM)
Next steps
POET is operational now, and initial observations indicate that IPMI is a useful estimate for total server energy consumption, while Kepler, Scaphandre are promising tools for O-Cloud VNF/CNF power estimates. Currently, the team is in the process of calibrating various results, augmenting RU power measurements using commercial RUs, and correlating power consumption measurements with performance measurements to develop power consumption KPIs and models. In addition, dialogue with other labs is underway, specifically ones funded by NTIA (such as ORCID and ACCoRD), with the goal of expanding experiments and getting closer to real-life network deployment scenarios.
Register to attend the virtual Telco Sustainability Forum, hosted by RCR Wireless News on December 10th. Sarat Puthenpura will be participating in a lively panel discussion “From the core to the RAN – how to make networks more sustainable”.
About the author
Sarat Puthenpura, Ph.D., is the Chief Architect – Open RAN, SMaRT-5G and Aether projects hosted by the Linux Foundation. He has over 35 years of experience in the industry spanning the application of mathematical optimization and artificial intelligence/machine learning techniques in telecommunications network planning, resource optimization, operations automation, and performance management with leadership in R&D, software development and deployment in these areas. He is the inventor of several technologies along with fundamental technical contributions in these fields with 86 patents, 40 papers, and a graduate level textbook, and recipient of the AT&T Science and Technology Medal.
