On April 20, the V International Scientific and Practical Conference “Scientific Progress…

On April 20, Saratov hosted V International Scientific and Practical Conference “Scientific Progress and Sustainable Development” . The participants' articles will be included in the RSCI collection...

So, wait, why am I saying this? Well, okay, okay, it’s clear that since I’m talking about it, it means I’m somehow connected with this conference. You guessed it ( you won't be fooled, I'm proud ) 🧠

During the conference, I presented my new work - a continuation ( and deepening ) that research about which has already been wrote on the channel. Only if before we had one agent, now we have five. The article is called “Multi-agent access control in NB-IoT networks based on deep reinforcement learning” ( and it is just being prepared for publication in a collection of conference reports )

Let me tell you what's new and add introductory notes 🧐

Imagine thousands of devices trying to transmit data over the same radio channel ( and this is true in the case of NB-IoT ). This will cause a bunch of collisions, timeouts and retransmissions. All this consumes energy and interferes with the transfer of data from devices. The problem is especially noticeable in scenarios like “a city with hundreds of thousands of different sensors”, where during peak hours all devices suddenly wake up

Previously, my colleagues and I tried to cope with this using one RL agent. And we were able to reduce collisions by 74% and improve energy efficiency. But as the load increased, the agent became heavily overloaded. Well, because it cannot immediately proactively determine traffic, configure slots and take into account device types. In scenarios for several hundred devices - quite. But if we bring all this closer to the real quantity ( hundreds of thousands or millions ), it can no longer boast the same efficiency 😏

This is where I come out with my proposal. And I, in fact, decided to assemble an architecture of five specialized agents:

🙂 The first controls the number of access slots

🕗 The second predicts activity peaks over time

🙂 The third differentiates the access policy depending on the type of device

📊 The fourth balances the load across sectors

🙂 The fifth optimizes the power consumption of devices

I implemented all this in NS-3, connected it with Python agents via ns3-gym, trained it on DQN/A2C/DDPG and tested it in two scenarios ( and in the previous study we took into account only one urban scenario ). This time I modeled a city with a high density of devices and a rural area with a low one. The results came out beautiful:

Collisions in the city decreased from 26% ( when static ) up to 7%

Connection delay decreased from 5 seconds to 1.5 seconds

Energy consumption reduced by 35%

At the same time, the system remains consistent - the actions of the agents do not conflict, but, on the contrary, complement each other. This was confirmed by correlation and stability metrics in different conditions 🥳

I believe that all this can potentially be scaled. If earlier we could simply improve RACH, now it is possible to build real intelligent distributed access systems, where each agent is responsible for his or her area of responsibility

Ahead is an experimental test of this entire architecture on real hardware ( and the safety of the RL agents themselves ). Perhaps even with attacking agents to test resilience. However, I will talk about this in future posts. ❤

Thanks to the conference for the opportunity to share the results. And to you for reading this post ( and read everyone else ) 🥂

#Internet_of_things #graduate studies