Speaker
Description
Recently, the landscape of computational infrastructure is in dramatic changes under the pressure of application requirements. The suit of the properties of modern applications can be summarized as follows: distributed, self-sufficient, work in real time, elastic, cross-platform, actively interact and synchronize, and are easy to update. The definitions of these terms are in [1]. For further understanding, it is important to recognize that an application is made up of interrelated components, which we will refer to as application functions. The analysis of requirements of modern application to the computational infrastructure presented in [1] shows the trend of ubiquitous application deployment. We are moving to the era when data processing resources and data transmission resources form a single space for computing - computational infrastructure. In other words, the time has come for the implementation of the slogan "Network is a Computer". Further we will call such computational infrastructure Network Powered by Computing (NPC).
Several versions of Functional Architecture for such new generation of computational infrastructure were proposed. Briefly NPC functional architecture presented in [1] can be described as following (see figure 1). It consists of data processing (DP) plane, data transmission (DT) plane, data processing control (DPC) plane, data transmission control (DTC) plane, administration, orchestration and management plane (AOM plane). DP plane covers all computational resources of NPC. DT plane is an overlay network over underlying physical network. Actually data transmission plane is data transmission network (DTN). DPC plane is responsible for preparation of the application for execution, planning the placement of application components, calculation of the quality of service (QoS) requirements based on the service level agreement (SLA) specified by the user, generation of DTN control plane instructions for setting up overlay tunnels in accordance with the application function interaction topology of the application. DTC plane is responsible for control and monitoring of DTN. AOM plane orchestrates interactions between application components in accordance with application topology, collects NPC resource consumption statistics by every application component, secures management and administration of NPC.
In [2] it has been proposed the new method for optimal traffic routing in overlay DTN of NPC based on decentralized multi-agent reinforcement learning (MARL) with hashing (MAROH). In the cited paper are considered three approaches to MARL optimization: centralized, decentralized with communication and fully decentralized. It is assumed in all these approaches that agents construct their local state based on the environment observations. The agents behavior in these approaches are different:
- Centralized approach: a single entity named control center is responsible for the operation of every agent. This center collects the local states from all agents and defines the actions for every agent by solving an optimization problem.
- Decentralized approach with communication (or just decentralized approach): assumes that the local state of every agent can be broadcasted to all neighboring agents. Then the agent by himself chooses an optimal action based on the knowledge of local and neighboring agent states.
- The fully decentralized approach: do not suppose any communication between agents. Decision should be made based only on the history of the agent local behavior observations.
The main problems of MA methods for traffic control are poor scalability; there are no mathematical models that guarantee convergence to the optimal solution; it is difficult to mathematically frame optimization functional; the extent of deviation from the optimal solution is unknown. It is shown that the new proposed method surmount the problems listed above. MAROH method can be used for optimal traffic engineering is also applicable in traditional data network.
The experiments (see figure 2) show that the traditional approaches in load balancing like Equal Cost Multi-Path (ECMP [3]) or Unequal Cost Multi-Path (UCMP [4]) are ineffective in NPC environment. ECMP assigns the same weight to each possible path to the same destination and balances the flow or packets among these paths evenly. As NPC overlay channels may go through different Internet Service Providers (ISP), the assumption that these channels are similar is wrong. It is only flow load balancing is considered, because packet load balancing disturbs the congestion control algorithm operation and multi-path transport protocols are out of scope of this work. UCMP allows to assign the weight according to its available resources (e.g. bandwidth) and balances flows in the same ratio. However, UCMP doesn’t provide the coordinated choice of weights. It means that NPCRs may overload some channel by simultaneously forwarding flows to it. These shortcomings lead us to consider multi-agent machine learning approaches.
In this paper the MAROH method briefly is presented with NPC architecture. The main contribution of the paper is the comparison of ECMP, UCMP and MAROH.
[1] Smeliansky R. Network Powered by Computing: Next Generation of Computational Infrastructure. In Edge Computing – Technology, Management and Integration. Intechopeт 2023, DOI: 10.5772/intechopen.110178
[2] Stepanov E.P., Smeliansky R.L., Plakunov A.V., Borisov A.V., Xia Zhu, Jianing Pei, Zhen Yao On Fair Traffic allocation and Efficient Utilization of Network Resources based on MARL (preliminary on Researchgate)
[3] ECMP Load Balancing https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/mp_l3_vpns/configuration/xe-3s/asr903/mp-l3-vpns-xe-3s-asr903-book/mp-l3-vpns-xe-3s-asr903-book_chapter_0100.pdf
[4] UCMP Load Balancing https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/mp_l3_vpns/configuration/xe-3s/asr903/17-1-1/b-mpls-l3-vpns-xe-17-1-asr900/m-ucmp.pdf
