Mohamed-Aymen Chalouf - Intelligent Security Management and Control in the IoT

Wang, Y.-H., Huang, G.-R., Tung, Y.-C. (2014). A handover prediction mechanism based on LTE-A UE history information. In International Conference on Computer, Information and Telecommunication Systems (CITS). IEEE, Jeju.

Wu, Q., Ding, G., Xu, Y., Feng, S., Du, Z., Wang, J., Long, K. (2014a). Cognitive Internet of Things: A new paradigm beyond connection. IEEE Internet of Things Journal , 1(2), 129–143. doi:10.1109/JIOT.2014.2311513.

Wu, Y., Hu, F., Kumar, S., Zhu, Y., Talari, A., Rahnavard, N., Matyjas, J.D. (2014b). A learning-based QoE-driven spectrum handoff scheme for multimedia transmissions over cognitive radio networks. IEEE Journal on Selected Areas in Communications , 32(11), 2134–2148.

Xenakis, D., Passas, N., Di Gregorio, L., Verikoukis, C. (2011). A context aware vertical handover framework towards energy-efficiency. In Proceedings of the IEEE Vehicular Technology Conference (VTC) .

Xiao, K. and Li, C. (2018). Vertical handoff decision algorithm for heterogeneous wireless networks based on entropy and improved TOPSIS. In 2018 IEEE 18th International Conference on Communication Technology (ICCT) . IEEE, Chongqing.

Xu, L.D., He, W., Li, S. (2014). Internet of Things in industries: A survey. IEEE Transactions on Industrial Informatics , 10(4), 2233–2243.

Zekri, M., Jouaber, B., Zeghlache, D. (2012). A review on mobility management and vertical handover solutions over heterogeneous wireless networks. Computer Communications , 35, 2055–2068.

Zhu, C., Leung, V.C., Shu, L., Ngai, E.C.H. (2015). Green IoT for smart world. Special Section on Challenges for Smart Worlds . doi:10.1109/ACCESS.2015.2497312.

Zou, L., Javed, A., Muntean, G. (2017). Smart mobile device power consumption measurement for video streaming in wireless environments: WiFi vs. LTE. In 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB). IEEE, Cagliari.

Yassine HADJADJ-AOUL and Soraya AIT-CHELLOUCHE

Inria, CNRS, IRISA, University of Rennes 1, France

Communications between objects in the Internet of Things (IoT) and particularly machine-to-machine (M2M) communications are considered as one of the most important evolutions of the Internet. Supporting these devices is, however, one of the most significant challenges that network operators need to overcome (Lin et al. 2016). In fact, the considerable number of these devices that might attempt to access a network at the same time could lead to heavy congestion, or even a total saturation, with all the consequences this might cause. In fact, as can be seen in Figure 2.1, a very limited number of devices trying simultaneously to access the network can reduce the network’s performance to zero, independently of the access opportunities available (Bouzouita et al. 2016). In these circumstances, it seems clear that effective access control mechanisms are needed to maintain a reasonable number of access attempts.

The Third Generation Partnership Project (3GPP) group has identified the overload on the Random Access Network (RAN) as a priority at a very early stage and has suggested several solutions. Among the approaches suggested, the Access Class Barring (ACB), suggested in version 8, and its extension, the Extended Access Barring (EAB), suggested in version 11, are certainly the most effective strategies (3GPP 2011). In fact, these approaches tackle the problem at its roots by preventing any attempts to access the network. However, these approaches merely provide a framework for congestion management, without providing a ready-made solution.

Figure 2.1. Impact on performance of the number of IoT devices simultaneously attempting access

The idea introduced in this chapter is quite simple, because it involves calculating a blocking factor. Nevertheless, a good implementation would require a good knowledge of the number of terminals ready to attempt access, so as to deduce from this the probability of the optimal blocking. This information is unfortunately not available in the network.

To solve this problem, two significant challenges should be taken into account: (1) designing an access control strategy for dynamic generation of the blocking factor and (2) estimating the number of devices simultaneously attempting to access the network.

In this chapter, we tackle these questions using an estimator that was suggested in earlier research (Bouzouita et al. 2019). Since this estimation is very noisy, we exploit the potential of the most advanced reinforcement learning techniques, to take account of this complex reality (the state of the network is not observable) and deduce a sub-optimal control strategy. More especially in this chapter, we use the deep reinforcement learning algorithm Twin Delayed Deep Deterministic policy gradient algorithm (TD3) (Fujimoto et al. 2018) to produce the optimal blocking factor from these past estimations (Hadjadj-Aoul and Ait-Chellouche 2020).

The remainder of this chapter is organized as follows. Section 2.2briefly presents the NB-IoT standard considered in our proposal. Section 2.3gives an overview of the main congestion control techniques in IoT networks, and more especially cellular IoT networks. Section 2.4describes the model for IoT terminals to access the network. Section 2.5describes the suggested control solution, based on the TD3 algorithm, adapted to solve the problem of calculating the blocking factor. Section 2.6describes the environment for simulating the suggested approach and shows its effectiveness compared to the existing approach. Finally, the chapter ends with a summary recapitulating the main advantages and achievements of the system suggested in section 2.7.

NB-IoT is a cellular access technology of the Low Power Wide Area Network (LPWAN) type, specified in the document “Rel-13” by the 3GPP (3GPP 2016). Improvements to it were then provided in the documents “Rel-14”, “Rel-15” and “Rel-16” (Mwakwata et al. 2019). Several key players in the industry, such as Ericsson, Nokia, Intel or Huawei, have shown a keen interest in this standard and have broadly participated in its standardization (Bicheno 2015).

NB-IoT technology is also presented as a promising candidate for covering one of the three main pillars of 5G, that is the massive Machine Type Communication (mMTC) (3GPP 2018a; Narayanan et al. 2018). In fact, the ITU has defined different requirements in terms of the density of the IoT terminals, battery life, coverage, price and mechanisms and features supported, to which the suggested technologies must respond to implement mMTCs (Kafle et al . 2016). In this direction, the 3GPP has suggested several mechanisms and NB-IoT features to meet these demands through different releases (Mwakwata et al. 2019). The options for integrating the NB-IoT with the core 5G network and its coexistence with the other services suggested in the framework of this standard are also studied by the 3GPP (3GPP 2019a). A better use of physical resources via the virtualization of network functions (NFV) or the SDN, as discussed in Migabo et al. (2020), can also facilitate this integration.