Sarat Puthenpura, Chief Architect – Open RAN; Aether (previously ONF) will be speaking at this year’s Open RAN Global Forum. Read more and register here.
Energy costs for telecom operators can be as high as 10% of their total operating expenditure. This is especially true as they roll out 5G. Even though the 5G-NR standards are more energy efficient than their predecessors, the overall power consumption of 5G networks can be significantly higher than previous generation networks due to a number of compounding factors.
Massive MIMO and higher MIMO modes, which are highly beneficial to support mobile broadband at high frequency bands (FR2), require more power. Furthermore, with mobile edge computing, the number of data centers will increase and computationally demanding use cases will further intensify energy usage. Another aspect of 5G is the higher operating bandwidth for each channel, which has increased from 20 MHz (LTE) to as high as 400 MHz. The sub-division of larger bandwidths into Bandwidth Parts (BWPs), which is aimed at reducing power consumption of devices, will cause incremental energy consumption for the base stations. Thus overall, the energy consumption in mobile networks is expected to go up significantly as 5G is rolled out. With all this, mobile operators are under intense pressure to deploy intelligent solutions to optimize the energy consumption of their mobile networks.
Energy efficiency and energy savings use cases have been identified as a top priority by mobile network operators and this drives ongoing standardization discussions in the O-RAN Alliance and 3GPP. Energy efficiency testing, metrics and estimation models provide the foundation for solutions for monitoring and optimizing energy usage. Existing specifications and methods in the context of Open RAN are being improved and updated. The overall logical steps here are straightforward and intuitive: (1) measure energy consumption accurately with sufficient granularity; (2) create KPIs and models to capture the characteristics of energy consumption; and (3) use the KPIs and models to optimize energy consumption.
The problem with the current landscape of energy saving use cases and associated optimization strategies is that they are at Step 3 of the above-mentioned logical steps, somewhat bypassing Steps 1 and 2. For example, cell on/off approaches (turning cells on/off based on load) do make sense to save energy in general, since one could temporarily turn some capacity off under low load conditions if a certain level of quality of service (QoS) can be maintained. However, doing the same and similar operations based on quantifiable measures of energy efficiency and associated models makes more sense, because different strategies and approaches are influenced heavily by network configurations, operating conditions and performance requirements.
Advancing energy savings capabilities
An open-source anchored effort has emerged to pursue advancement of energy savings use cases.
The SMaRT-5G initiative — originally started by Open Networking Foundation (ONF) — has been building a series of PoCs that demonstrate progressively advanced energy savings techniques on both Open-RAN and traditional RAN architectures. The phased series of PoCs is to progressively deliver power usage models, APIs and open-source software, with the goal of accelerating MNO adoption and industry impact. We have successfully demonstrated two SMaRT-5G PoCs in the recent past. The first was at Fyuz in October 2023 and the second took place at the NTIA-hosted RIC forum in March 2024. The key collaborators of these PoCs were Rimedo Labs, Tietoevry, Intel Labs, Viavi Solutions and i14y Labs. More PoCs are on the way.
Recently, the National Telecommunications and Information Administration (NTIA) of the U.S. Department of Commerce has graciously supported the SMaRT-5G initiative with a research grant through their Public Wireless Supply Chain Innovation Fund (PWSCIF). The $2 million grant is funding SMaRT-5G in collaboration with Rutgers University’s WINLAB to research, develop and validate accurate and effective test methods and to create metrics and models to measure the energy efficiency of 5G network components as well as the effectiveness of end-to-end Open RAN based energy optimization strategies. The focus is on finding a minimal set of parameters and scenarios needed to achieve effective results, in the context of Open RAN networks employing a multi-vendor, disaggregated architecture with physical, virtual and cloud-native network functions. Experimental research is being conducted as a collaborative effort between members of Aether and the SMaRT-5G project and Rutgers WINLAB, leveraging WINLAB’s lab infrastructure and lab’s synergy with other multi-vendor Open RAN projects.
The expected outcome of this R&D effort is to develop and validate: (a) innovative effective measurements for energy consumption of various RAN and core components; (b) energy consumption metrics, KPIs and APIs to be supported by RAN and core equipment; (c) energy consumption models which can be used in simulation and analytics studies; and (d) methods to assess end-to-end energy efficiency optimization using different algorithms and dedicated applications.
For energy consumption measurements and modeling, we need to consider both physical functions (such as RU) as well as cloudified network functions (such as CU, DU, RIC and x/rApps). Understanding the interplay among these network functions under various traffic and environmental conditions affecting energy consumption is very important.
The Open RAN standards/testing community is particularly relevant to this project since they have the scope for specifying the metrics and test methodology. Our objective is to be actively engaged in the process. We have designed our plan based on current and trending standards and aim to contribute our findings back to the same process.
Energy efficiency testing — Platform and methodology
The platform we have created for this purpose is called POET: Platform for O-RAN Energy Efficiency Testing and its functional architecture is shown in the figure below.
POET Architecture (Platform for O-RAN Energy Efficiency Testing)
POET includes bare metal servers as well as a Kubernetes cluster for the O-DU, O-CU and Core. It includes physical O-RUs as well as SDR-based RUs.
The Kubernetes system includes near-RT RIC, Kepler power monitoring and support for SDR RU and simulated RAN. The testbed includes an Amarisoft UE-emulator and modem/phone UEs. Power supplied to all components is monitored using smart PDUs (Server Technology PRO3X and STV-6521V.) Power and performance metrics are captured and exported to Prometheus and displayed on Grafana dashboards. POET heavily leverages open-source components to implement end-to-end O-RAN scenarios. Open-source software is free and openly accessible, thus enabling teams to replicate results in a common reference framework. This aligns with our project objective of contributing our results to influence the industry. The open source software we are using and plan to use is from Aether,OAI, srsRAN and O-RAN Software Community.
Kubernetes servers are Dell R730xd (Intel Xeon E5-2680 v4 2.4GHz, 56 CPUs) while the bare-metal servers are Dell R740 (Intel Xeon Gold 6226 2.70GHz, 24 CPUs) and Penguin Relion 1900 (Intel Xeon E5-2620 v4 2.10GHz, 32 CPUs).
Our broad objective is to research the best methodology for measuring energy consumption, energy efficiency and energy savings. While energy measurement is not a new field, there is relatively little available in terms of standardized methodology, especially when addressing cloudified NFs. Our objective is to determine the test methodology rather than compare the efficiency and performance of various equipment. We intend to explore different methods as part of our work, including:
(a) Testing under different user traffic and load and radio scenarios (e.g., frequency bands, channel bandwidth, path loss, MIMO modes, etc.)
(b)Testing different types of O-RU (e.g., indoor/outdoor,) and server architectures (e.g., with and without accelerators)
(c) Automating the collection of power and performance metrics and test metadata, and reporting of results
Measuring the power and energy consumption is at the heart of the energy efficiency testing and validation of energy optimization solutions. We have adopted different approaches for power measurement:
(a) PDU: The power, current and voltage supplied to PNFs and servers is obtained by regularly querying the Power Distribution Units (PDUs) (nominally every 10 secs), and exporting this data to Prometheus/Grafana. This provides a critical ground-truth measurement of power consumption.
(b) IPMI: Power and environment variables reported by the servers are also monitored using the Intelligent Platform Management Interface (IPMI). Queries are made to the server Baseboard Management Controller (BMC) and exported to Prometheus/Grafana.
(c) Scaphandre: We deployed the Scaphandre open-source energy monitoring functionality on bare-metal servers running CU/DU software. Scaphandre measures process utilization (based on Intel Running Average Power Limit (RAPL)) and estimates the power consumption using an estimation model.
(d) Kepler: The Kubernetes deployment in the testbed uses Kepler exporters on each server sending power metrics to Prometheus/Grafana. Kepler is an open-source energy monitoring functionality for Kubernetes systems. It measures node and container utilization (based on Intel RAPL) and estimates the power consumption using an estimation model.
For energy efficiency, we essentially have a high-level relationship given by desired network performance divided by energy consumed in the relevant portion of the network. We plan to measure energy consumption (denominator of the ratio) using the multi-pronged approach described before. The numerator, desired network performance, is a loaded term and depending on the context, its definition can vary. For example, MNOs broadly use throughput and cell-edge performance as well as accessibility and retainability to measure performance. Our plan is to collect performance measures such as number of UEs / RRC connections per cell, DL and UL PRB utilization per cell, DL and UL throughput (Mbps) per cell and per UE, DL and UL data volume (bytes) per cell and per UE, latency per UE and aggregate across UEs per cell, MCS value per UE and aggregate across cell, and characteristics of UE traffic mix – and we will explore correlations that are relevant to energy efficiency.
Energy efficiency testing and energy optimization monitoring require accurate and convenient performance KPI measurements. We have the following approaches for KPI metrics:
(a) E2E: We use iPerf application end-to-end to obtain the data volume, throughput and latency of a session between the UE and a test server in the packet data network.
(b) O1-based KPI: We use an “O1-like” approach using a telnet-based solution that provides uplink/downlink throughout and downlink PRB load for DU. This is a preliminary version of ongoing open-source work to implement an O1 interface for CU/DU.
(c) E2-based KPI: The E2 interface from the O-CU and O-DU to the Near-RT RIC and xApps can provide several KPIs using the E2-SM KPM. Our Kubernetes-based O-RAN system includes a KPM xApp on Near-RT RIC with an E2 interface to the CU/DU.
We plan to use our test results to develop models to estimate the power consumption of O-RAN networks, including:
1) Models provide estimates based on available energy and performance metrics when a given deployed system may not support all the latest self-reported energy metrics
2) Models which can be calibrated with initial lab-based tests of equipment which are later deployed in the network
3) Models to provide input for rApp/xApp energy saving applications
It is important to note that POET leverages open-source components to implement end-to-end O-RAN scenarios. Open-source software is free and openly accessible, thus enabling teams to replicate results in a common reference framework. This aligns with our project objective of contributing our results to influence the industry.
It is also notable that this project is getting collaboration support from BubbleRAN and Cognizant with more companies in the pipeline.
Concluding remarks
POET is operational now, and we have started getting initial measurements from the platform. These measurements are in-line with our expectations and plan. Analysis, interpretation and modeling activities are progressing along with the expansion and enhancements of POET. This will be discussed in a future article.
Sarat Puthenpura, Chief Architect – Open RAN; Aether (previously ONF) will be speaking at this year’s Open RAN Global Forum. Read more and register here.
