Authors: Hatim Chergui, Juan Camargo, Miguel Catalan
Organization: i2CAT
As 6G looms on the horizon, the complexity of resource management has grown exponentially. In Open Radio Access Networks (O-RAN), virtualized Radio Access Networks (vRANs) must efficiently allocate both radio and computational resources to meet diverse performance requirements. A novel approach developed under the NANCY project leverages multi-agent protocol learning or emerging communication to offer a powerful solution to these challenges. By enabling agents to dynamically develop a communication protocol, the NANCY AI Virtualizer improves resource orchestration, resolves conflicts, and enhances overall network performance.
This blog explores the innovative integration of decentralized multi-agent deep reinforcement learning (MADRL) in O-RAN environments, focusing on emergent communication’s role in optimizing CPU utilization, reducing latency, and fostering collaboration between agents.
Intelligent Inter-Slice Resource Management in O-RAN
O-RAN systems face a non-linear relationship between bitrate, vRAN bandwidth, and CPU utilization. Even with adequate radio resources, insufficient computational resources in the O-Cloud can degrade network performance. Effective Radio Resource Management (RRM) must address this by coordinating CPU allocation across network slices. In a typical vRAN use case, i) Each slice operates a server at the edge, managed by a Virtual Infrastructure Manager (VIM) for computing, storage, and network resources and ii) Latency factors include CPU preemption at the edge and per-slice transmission queues at the O-CU level, combining to determine end-to-end slice performance. Achieving optimal latency and utilization requires seamless coordination between slices.
In this respect, the NANCY AI Virtualizer leverages emergent communication in a decentralized MADRL setup, where agents develop a common communication protocol through on-the-fly learning, guided by a reward function that penalizes conflicts and underutilization. This enables the dynamic adjustment of CPU frequencies to minimize local latency as well as the utilization of underutilized resources from other slices to enhance system-wide efficiency. By communicating via learned messages, agents align resource allocation strategies without relying on predefined agreements. This protocol learning approach ensures adaptability and efficiency, fostering collaboration among agents for optimal performance.
Cloud-Native Design
In practice, the AI Virtualizer employs a cloud-native architecture to enable scalable network operations. Communication between agents and the server is facilitated through a Kafka Bus running in a dedicated container. This setup allows agents and the server to exchange various types of data, including messages for on-the-fly protocol learning and the broadcast of rewards and state updates. Kafka’s ability to scale horizontally and support asynchronous communication makes it well-suited for distributed systems. It efficiently handles large data volumes, enabling independent message production and consumption, which improves system resilience, reduces bottlenecks, and ensures real-time, reliable data processing.
Performance
The AI Virtualizer significantly outperforms baseline approaches by enhancing CPU utilization, achieving up to 1.4 times better efficiency while maintaining optimal usage levels. It reduces resource conflicts dramatically, averaging just 8.44 conflicts in early episodes and eventually eliminating them entirely, outperforming other methods by factors of 3.4 to 6.06. Additionally, its emergent communication capabilities enable agents to adapt collaboratively, ensuring efficient resource sharing, reduced latency, and robust performance in dynamic environments.
Conclusion
The integration of multi-agent protocol learning under the NANCY project represents a paradigm shift in resource orchestration for O-RAN. By enabling agents to develop a shared communication protocol, the NANCY AI Virtualizer optimizes CPU utilization, minimizes latency, and resolves inter-slice conflicts.
Emergent communication offers a flexible, scalable approach that aligns with the cloud-native principles required for future 6G networks. As networks grow increasingly complex, frameworks like the NANCY AI Virtualizer pave the way for autonomous, self-optimizing systems capable of adapting to ever-changing demands, driving the evolution of next-generation wireless communication.
