دانشگاه تربیت دبیر شهید رجائی
انتشارات دانشگاه تربیت دبیر شهید رجائی

  اخبار و اعلانات

 آمار

تعداد نشریات11
تعداد شماره‌ها210
تعداد مقالات2,098
تعداد مشاهده مقاله2,877,815
تعداد دریافت فایل اصل مقاله2,085,693
Khazaei, M.. (1401). Reinforcement Learning-based Load Controller in IP Multimedia Subsystems. فناوری آموزش, 11(1), 21-32. doi: 10.22061/jecei.2022.8723.546
M. Khazaei. "Reinforcement Learning-based Load Controller in IP Multimedia Subsystems". فناوری آموزش, 11, 1, 1401, 21-32. doi: 10.22061/jecei.2022.8723.546
Khazaei, M.. (1401). 'Reinforcement Learning-based Load Controller in IP Multimedia Subsystems', فناوری آموزش, 11(1), pp. 21-32. doi: 10.22061/jecei.2022.8723.546
Khazaei, M.. Reinforcement Learning-based Load Controller in IP Multimedia Subsystems. فناوری آموزش, 1401; 11(1): 21-32. doi: 10.22061/jecei.2022.8723.546

Reinforcement Learning-based Load Controller in IP Multimedia Subsystems

Journal of Electrical and Computer Engineering Innovations (JECEI)
مقاله 22، دوره 11، شماره 1، فروردین 2023، صفحه 21-32    اصل مقاله (695.58 K)
نوع مقاله: Original Research Paper
شناسه دیجیتال (DOI): 10.22061/jecei.2022.8723.546
نویسنده
M. Khazaei*
Computer Engineering Department, Kermanshah University of Technology, Kermanshah, Iran.
تاریخ دریافت: 09 بهمن 1400،  تاریخ بازنگری: 02 اردیبهشت 1401،  تاریخ پذیرش: 11 اردیبهشت 1401 
چکیده
Background and Objectives: IP multimedia subsystems (IMS) have been introduced as the Next Generation Network (NGN) platform while considering Session Initiation Protocol (SIP) as the signaling protocol. SIP lacks a proper overload mechanism. Hence, this challenge causes decline in the multimedia QoS. The main propose of overload control mechanism is to keep the network throughput at the same network capacity with overload.
Methods: NGN distributed with IMS is a complex innovative network consisting of interacting subsystems. Hence, multi-agent systems (MAS) receiving further attention for solving complex problems can solve the problem of overload in these networks. To this end, each IMS server is considered as an intelligent agent that can learn and negotiate with other agents while maintaining autonomy, thus eliminating the overload by communication and knowledge transfer between the agents. In the present research, using MAS and their properties, the intelligent hop by hop method is provided based on Q-learning and negotiation capability for the first time.
Results: In the proposed method, parameters of overload controller are obtained by reinforcement learning. In order to check the validity of controller performance, a comparison is made with the similar method in which the optimal parameters are achieved based on trial and error. The result of the comparison confirms the validity of the proposed method. In order to evaluate the efficiency of the learner method, it is compared with similar and standard methods, for which the results are compared to show performance. The results show, the proposed method has approximately improved the throughput by 13%, the delay by 49% and the number of rejected sessions by 17% compare with methods, passing control messages through the network such as CPU occupancy methods. While compare with external controller methods like holonic, throughput is improved by 1% and the number of rejected requests is decreased by 10%, but delay is increased by 6% due to the convergence time of the learning and negotiation process.
Conclusion: To overcome overload, complex IMS servers are considered as learner and negotiator agents. This is a new method to achieve the required parameters without relying on expert knowledge or person as well as, heterogeneous IMS entities can be inserted into the problem to complete study in future.
کلیدواژه‌ها
IMS؛ SIP؛ Overload؛ Multi-agent System؛ Reinforcement Learning
مراجع
[1] P. Agrawal, Y. Jui-Hung, C. Jyh-Cheng, Z. Tao, "IP multimedia subsystems in 3GPP and 3GPP2: overview and scalability issues," IEEE Commun. Mag., 46: 138-145, 2008.

[2] K.K. Guduru, U. Jayadevappa, "Overload control in SIP signalling networks with redirect servers," Int. J. Wireless Mobile Comput., 19: 124-132, 2020.

[3] V.S. Vaishnavi, Y. M. Roopa, P. L. Srinivasa Murthy, "A survey on next generation networks," in Proc. ICCNCT 2019: 162-171, 2020.

[4] C.Shen, H. Schulzrinne, E. Nahum, "Session Initiation Protocol (SIP) server overload control: Design and evaluation," in Principles, Systems and Applications of IP Telecommunications. Services and Security for Next Generation Networks, S. Henning, S. Radu, and N. Saverio, Eds., ed: Springer-Verlag: 149-173, 2008.

[5] V. Hilt, I. Widjaja, "Controlling overload in networks of SIP servers," in Proc. IEEE International Conference in Network Protocols: 83-93, 2008.

[6] J. Davin, P. Riley, M. Veloso, "CommLang: Communication for coachable agents," in Proc. RoboCup 2004: Robot Soccer World Cup VIII. vol. 3276, D. Nardi, M. Riedmiller, C. Sammut, and J. Santos-Victor, Eds., ed: Springer Berlin Heidelberg: 46-59, 2005.

[7] M. Wooldridge, An Introduction to MultiAgent Systems, Newyork: Wiley, 2009.

[8] B. Jang, M. Kim, G. Harerimana, J. W. Kim, "Q-Learning algorithms: A comprehensive classification and applications," IEEE Access, 7: 133653-133667, 2019.

[9] M. Abdoos, N. Mozayani, A. C. Bazzan, "Hierarchical control of traffic signals using Q-learning with tile coding," Appl. Intell., 40: 201-213, 2014.

[10] H.C. Hsieh, J. L. Chen, "Distributed multi-agent scheme support for service continuity in IMS-4G-Cloud networks," Comput. Electr. Eng., 42: 49-59, 2015.

[11] D. Pereira, R. Oliveira, H. S. Kim, "A machine learning approach for prediction of signaling sip dialogs," IEEE Access, 9: 44094-44106, 2021.

[12] F. B. Mismar, J. Hoydis, "Unsupervised learning in next-generation networks: Real-time performance self-diagnosis," IEEE Commun. Lett.,  25: 3330-3334, 2021.

[13] S. Chen, Z. Yao, X. Jiang, J. Yang, L. Hanzo, "Multi-agent deep reinforcement learning-based cooperative edge caching for ultra-dense next-generation networks," IEEE Trans. Commun., 69: 2441-2456, 2021.

[14] M. Abdoos, N. Mozayani, A. L. C. Bazzan, "Holonic multi-agent system for traffic signals control," Eng. Appl. Artif. Intell., 26: 1575-1587, 2013.

[15] M. Abdoos, N. Mozayani, A. L. C. Bazzan, "Traffic light control in non-stationary environments based on multi agent Q-learning," in Proc. 14th International IEEE Conference in Intelligent Transportation Systems (ITSC): 1580-1585, 2011.

[16] Y. Yao, V. Hilaire, A. Koukam, W. Cai, "A holonic model in wireless sensor networks," in Proc. International Conference in Intelligent Information Hiding and Multimedia Signal Processing: 491-495, 2008.

[17] S.M. Hosseini, N. Mozayeni, "An intelligent method for resource management in wireless networks," in Proc. 5th Conference in Information and Knowledge Technology (IKT): 371-376, 2013.

[18] E. Pei, L. Zhou, B. Deng, X. Lu, Y. Li, Z. Zhang, "A Q-Learning based energy threshold optimization algorithm in LAA networks," IEEE Trans. Veh. Technol., 70: 7037-7049, 2021.

[19] M. Cossentino, N. Gaud, V. Hilaire, S. Galland, A. Koukam, "ASPECS: an agent-oriented software process for engineering complex systems," Auton. Agents Multi-Agent Syst., 20: 260-304, 2010.

[20] M. Poikselkä, The IMS: IP multimedia concepts and services: Newyoek, J. Wiley & Sons, 2006.

[21] N.M. Ahmed, N. E. Rikli, "QoS-Based data aggregation and resource allocation algorithm for machine type communication devices in next-generation networks," IEEE Access, 9: 119735-119754, 2021.

[22] V.K. Gurbani, R. Jain, "Transport protocol considerations for session initiation protocol networks," Bell Labs Tech. J., 9: 83-97, 2004.

[23] M. Ohta, "Overload control in a SIP signaling network," Int. J. Electr. Electron. Eng., 3: 87-92, 2009.

[24] E.N.V. Hilt, C. Shen, A. Abdelal, "Design considerations for session initiation protocol (SIP) overload control," Internet Engineering Task Force (IETF), Request for Comments: RFC6357, 2011.

[25] J. Liao, J. Wang, T. Li, J. Wang, J. Wang, X. Zhu, "A distributed end-to-end overload control mechanism for networks of SIP servers," Comput. Networks, 56: 2847-2868, 2012.

[26] H. Dong-Yeop, P. Ji Hong, Y. Seung-wha, K. Ki-Hyung, "A window-based overload control considering the number of confirmation massages for SIP server," in Proc. Fourth International Conference in Ubiquitous and Future Networks (ICUFN): 180-185, 2012.

[27] M. Homayouni, H. Nemati, V. Azhari, A. Akbari, "Controlling Overload in SIP Proxies: An Adaptive Window Based Approach Using No Explicit Feedback," in Proc. IEEE Global Telecommunications Conference: 1-5, 2010.

[28] S. V. Azhari, M. Homayouni, H. Nemati, J. Enayatizadeh, A. Akbari, "Overload control in SIP networks using no explicit feedback: A window based approach," Comput. Commun., 35: 1472-1483, 2012.

[29] A. Montazerolghaem, M. H. Yaghmaee Moghadam, "Improving efficiency of SIP protocol using window-based overload conditions," Soft Comput. J., 2: 16-25, 2021.

[30] M. Khazaei, N. Mozayani, "A dynamic distributed overload control mechanism in SIP networks with holonic multi-agent systems," Telecommun. Syst., 63: 437-455, 2016.

[31] Y. Hong, C. Huang, J. Yan, "Applying control theoretic approach to mitigate SIP overload," Telecommun. Systems, 54: 387-404, 2013.

[32] A. Abdelal, W. Matragi, "Signal-Based Overload Control for SIP Servers," in Proc. 7th IEEE Consumer Communications and Networking Conference: 1-7, 2010.

[33] R. G. Garroppo, S. Giordano, S. Niccolini, S. Spagna, "A Prediction-Based Overload Control Algorithm for SIP Servers," Network and Service Management, IEEE Transactions, 8: 39-51, 2011.

[34] S. Jing, T. Ruixiong, H. Jinfeng, Y. Bo, "Rate-based SIP flow management for SLA satisfaction," in IFIP/IEEE International Symposium on Integrated Network Management: 125-128, 2009.

[35] M. Khazaei, "Occupancy overload control by Q-learning," in Fundamental Research in Electrical Engineering, Singapore: 765-776, 2019.

[36] M. Khazaei, N. Mozayani, "Overload management with regard to fairness in session initiation protocol networks by holonic multiagent systems," Int. J. Network Manage., 27: e1969, 2017.

[37] A. Akbar, S. M. Basha, S. A. Sattar, "A cooperative overload control method for SIP servers," International Conference in Proc. Communications and Signal Processing (ICCSP): 1296-1300, 2015.

[38] A. Montazerolghaem, S. K. Shekofteh, M. H. Yaghmaee, M. Naghibzadeh, "A load scheduler for SIP proxy servers: design, implementation and evaluation of a history weighted window approach," Int. J. Commun. Sys., 2015.

[39] A. Montazerolghaem, M. H. Y. Moghaddam, A. Leon-Garcia, "OpenSIP: Toward software-defined SIP networking," IEEE Trans. Netw. Serv. Manage., 15: 184-199, 2018.

[40] R. Gandotra, L. Perigo, "SDVoIP—A software-defined VoIP framework for SIP and dynamic QoS," Comput. J., 64: 254-263, 2019.

[41] L. D. Cicco, G. Cofano, S. Mascolo, "Local SIP overload control: controller design and optimization by extremum seeking," IEEE Trans. Control Network Syst., 2: 267-277, 2015.

[42] Y. Hong, C. Huang, J. Yan, "Modelling chaotic behaviour of SIP retransmission mechanism," Int. J. Parallel Emerg. Distrib. Syst., 28: 95-122, 2013.

[43] J. Wang, J. Liao, T. Li, J. Wang, J. Wang, Q. Qi, "Probe-based end-to-end overload control for networks of SIP servers," J. Network Comput. Appl., 41: 114-125, 2014.

 

آمار
تعداد مشاهده مقاله: 452
تعداد دریافت فایل اصل مقاله: 381


سامانه مدیریت نشریات علمی. طراحی و پیاده سازی از سیناوب