in heavy traffic but slightly worse than GTDA. Therefore, a novel control approach DQNRR is proposed to improve the lane-changing efficiency. The proposed method DQNRR performs almost the same as GTDA. Still, the former method spends much less computational time than the latter, especially when there is a large traffic density. Thus, DQNRR has more advantages than the game theory-based methods in such a system with large-scale agents. Nevertheless, the limitations of the lane-changing model are the same as in the previous thesis mentioned above.
Bibliography
Abdulhai, B., Pringle, R., Karakoulas, G.J., 2003. Reinforcement learning for true adaptive traffic signal control. Journal of Transportation Engineering 129, 278–285.
Ali, Y., Bliemer, M.C., Zheng, Z., Haque, M.M., 2020a. Comparing the usefulness of real-time driving aids in a connected environment during mandatory and discre- tionary lane-changing manoeuvres. Transportation Research Part C: Emerging Tech- nologies 121, 102871. doi:10.1016/j.trc.2020.102871.
Ali, Y., Zheng, Z., Haque, M.M., Wang, M., 2019. A game theory-based ap- proach for modelling mandatory lane-changing behaviour in a connected environ- ment. Transportation Research Part C: Emerging Technologies 106, 220–242.
doi:10.1016/j.trc.2019.07.011.
Ali, Y., Zheng, Z., Haque, M.M., Yildirimoglu, M., Washington, S., 2020b. Understand- ing the discretionary lane-changing behaviour in the connected environment. Acci- dent Analysis & Prevention 137, 105463. doi:10.1016/j.aap.2020.105463.
Ali, Y., Zheng, Z., Haque, M.M., Yildirimoglu, M., Washington, S., 2021. CLACD:
A complete LAne-Changing decision modeling framework for the connected and traditional environments. Transportation Research Part C: Emerging Technologies 128, 103162. doi:10.1016/j.trc.2021.103162.
Alvarez, I., Poznyak, A., 2010. Game theory applied to urban traffic control problem, in: ICCAS 2010, IEEE. pp. 2164–2169. doi:10.1109/ICCAS.2010.5670234.
An, H., Jung, J.I., 2019. Decision-making system for lane change using deep re- inforcement learning in connected and automated driving. Electronics 8, 543.
doi:10.3390/electronics8050543.
Arai, K., Sentinuwo, S.R., 2012. Spontaneous-braking and lane-changing effect on traffic congestion using cellular automata model applied to the two-lane traffic.
International Journal of Advanced Computer Science and Applications 3, 39–47.
URL: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10. 1.1.258.6506&rep=rep1&type=pdf#page=50.
Arena, F., Pau, G., Ralescu, A., Severino, A., You, I., 2022. An innovative frame- work for dynamic traffic lights management based on the combined use of fuzzy logic and several network architectures. Journal of Advanced Transportation 2022.
doi:10.1155/2022/1383349.
Bajari, P., Hong, H., Ryan, S.P., 2010. Identification and estimation of a discrete game of complete information. Econometrica 78, 1529–1568. doi:10.3982/ECTA5434.
Ba¸sar, T., Olsder, G.J., 1998. Dynamic noncooperative game theory. Society for Indus- trial and Applied Mathematics. doi:10.1137/1.9781611971132.bm.
Bazzan, A.L., 2009. Opportunities for multiagent systems and multiagent reinforce- ment learning in traffic control. Autonomous Agents and Multi-Agent Systems 18, 342. doi:10.1007/s10458-008-9062-9.
Bui, K.H.N., Jung, J.J., 2018. Cooperative game-theoretic approach to traffic flow op- timization for multiple intersections. Computers & Electrical Engineering 71, 1012–
1024. doi:10.1016/j.compeleceng.2017.10.016.
Celtek, S.A., Durdu, A., Alı, M.E.M., 2020. Real-time traffic signal control with swarm optimization methods. Measurement 166, 108206.
doi:10.1016/j.measurement.2020.108206.
Chen, X., Deng, X., Teng, S.H., 2006. Sparse games are hard, in: Interna- tional Workshop on Internet and Network Economics, Springer. pp. 262–273.
doi:10.1007/11944874_24.
Cheng, S.F., Reeves, D.M., Vorobeychik, Y., Wellman, M.P., 2004. Notes on equi- libria in symmetric games, in: Proceedings of the 6th International Workshop on Game Theoretic and Decision Theoretic Agents, GTDT. pp. 71–78. URL:http://
citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.137.4019.
Daskalakis, C., Goldberg, P.W., Papadimitriou, C.H., 2009. The complexity of computing a Nash equilibrium. SIAM Journal on Computing 39, 195–259.
doi:10.1137/070699652.
Diakaki, C., Dinopoulou, V., Aboudolas, K., Papageorgiou, M., Ben-Shabat, E., Seider, E., Leibov, A., 2003. Extensions and new applications of the traffic-responsive ur- ban control strategy: Coordinated signal control for urban networks. Transportation Research Record 1856, 202–211. doi:10.3141/1856-22.
Diakaki, C., Papageorgiou, M., Papamichail, I., Nikolos, I., 2015. Overview and analy- sis of vehicle automation and communication systems from a motorway traffic man- agement perspective. Transportation Research Part A: Policy and Practice 75, 147–
165. doi:10.1016/j.tra.2015.03.015.
Dong, J., Chen, S., Li, Y., Du, R., Steinfeld, A., Labi, S., 2021. Space- weighted information fusion using deep reinforcement learning: The context of tactical control of lane-changing autonomous vehicles and connectivity range as- sessment. Transportation Research Part C: Emerging Technologies 128, 103192.
doi:10.1016/j.trc.2021.103192.
El-Tantawy, S., Abdulhai, B., Abdelgawad, H., 2013. Multiagent rein- forcement learning for integrated network of adaptive traffic signal con- trollers (MARLIN-ATSC): Methodology and large-scale application on downtown Toronto. IEEE Transactions on Intelligent Transportation Systems 14, 1140–1150.
doi:10.1109/TITS.2013.2255286.
El-Tantawy, S., Abdulhai, B., Abdelgawad, H., 2014. Design of reinforce- ment learning parameters for seamless application of adaptive traffic sig- nal control. Journal of Intelligent Transportation Systems 18, 227–245.
doi:10.1080/15472450.2013.810991.
Elhenawy, M., Elbery, A.A., Hassan, A.A., Rakha, H.A., 2015. An intersection game- theory-based traffic control algorithm in a connected vehicle environment, in: 2015 IEEE 18th International Conference on Intelligent Transportation Systems, IEEE. pp.
343–347. doi:10.1109/ITSC.2015.65.
Fu, H., Tang, H., Hao, J., Lei, Z., Chen, Y., Fan, C., 2019. Deep multi-agent re- inforcement learning with discrete-continuous hybrid action spaces. ArXiv Preprint arXiv:1903.04959. URL:https://doi.org/10.48550/arXiv.1903.04959.
Gartner, N.H., Stamatiadis, C., Tarnoff, P.J., 1995. Development of advanced traf- fic signal control strategies for intelligent transportation systems: Multilevel design.
Transportation Research Record 1494, 98–105.
Genders, W., Razavi, S., 2016. Using a deep reinforcement learning agent for traffic signal control. ArXiv Preprint arXiv:1611.01142. URL:https://doi.org/10. 48550/arXiv.1611.01142.
Genders, W., Razavi, S., 2018. Evaluating reinforcement learning state representa- tions for adaptive traffic signal control. Procedia Computer Science 130, 26–33.
doi:10.1016/j.procs.2018.04.008.
Gilpin, A., Sandholm, T., 2007. Lossless abstraction of imperfect information games.
Journal of the ACM 54, 25–es. doi:10.1145/1284320.1284324.
Gipps, P.G., 1986. A model for the structure of lane-changing deci- sions. Transportation Research Part B: Methodological 20, 403–414.
doi:10.1016/0191-2615(86)90012-3.
Halati, A., Lieu, H., Walker, S., 1997. Corsim-corridor traffic simulation model, in:
Traffic Congestion and Traffic Safety in the 21st Century: Challenges, Innovations, and OpportunitiesUrban Transportation Division, ASCE; Highway Division, ASCE;
Federal Highway Administration, USDOT; and National Highway Traffic Safety Ad- ministration, USDOT. URL:http://worldcat.org/isbn/0784402434.
Hidas, P., 2005. Modelling vehicle interactions in microscopic simulation of merging and weaving. Transportation Research Part C: Emerging Technologies 13, 37–62.
doi:10.1016/j.trc.2004.12.003.
Hoel, C.J., Driggs-Campbell, K., Wolff, K., Laine, L., Kochenderfer, M.J., 2019.
Combining planning and deep reinforcement learning in tactical decision making for autonomous driving. IEEE Transactions on Intelligent Vehicles 5, 294–305.
doi:10.1109/TIV.2019.2955905.
Hou, Y., Graf, P., 2021. Decentralized cooperative lane changing at freeway weaving areas using multi-agent deep reinforcement learning. ArXiv Preprint arXiv:2110.08124. URL:https://doi.org/10.48550/arXiv.2110.08124.
Howard, R.A., 1960. Dynamic programming and markov processes. American Psy- chological Association.
Howson Jr, J.T., 1972. Equilibria of polymatrix games. Management Science 18, 312–
318. doi:10.1287/mnsc.18.5.312.
Hu, J., Wellman, M.P., 2003. Nash Q-learning for general-sum stochastic games. Jour- nal of Machine Learning Research 4, 1039–1069. URL: https://www.jmlr. org/papers/volume4/temp/hu03a.pdf.
Huang, D.W., 2002. Lane-changing behavior on highways. Physical Review E 66, 026124. doi:10.1103/PhysRevE.66.026124.
Hyndman, R.J., Koehler, A.B., 2006. Another look at measures of forecast accuracy. International Journal of Forecasting 22, 679–688.
doi:10.1016/j.ijforecast.2006.03.001.
Iwase, T., Shiga, T., 2016. Linear game theory: Reduction of complexity by decom- posing large games into partial games. ArXiv Preprint arXiv:1609.00481. URL:
https://doi.org/10.48550/arXiv.1609.00481.
Jiang, C., Chen, T., Li, R., Li, L., Li, G., Xu, C., Li, S., 2020. Construction of extended ant colony labor division model for traffic signal timing and its application in mixed traffic flow model of single intersection. Concurrency and Computation: Practice and Experience 32, e5592. doi:10.1002/cpe.5592.
Jiang, S., Chen, J., Shen, M., 2019. An interactive lane change decision mak- ing model with deep reinforcement learning, in: 2019 7th International Confer- ence on Control, Mechatronics and Automation (ICCMA), IEEE. pp. 370–376.
doi:10.1109/ICCMA46720.2019.8988750.
Jin, C.J., Knoop, V.L., Li, D., Meng, L.Y., Wang, H., 2019. Discretionary lane-changing behavior: Empirical validation for one realistic rule-based model. Transportmetrica A: Transport Science 15, 244–262. doi:10.1080/23249935.2018.1464526.
Junchen, J., Xiaoliang, M., 2016. A learning-based adaptive signal con- trol system with function approximation. IFAC-PapersOnLine 49, 5–10.
doi:10.1016/j.ifacol.2016.07.002.
Kearns, M., Littman, M.L., Singh, S., 2013. Graphical models for game theory. ArXiv Preprint arXiv:1301.2281. URL:https://doi.org/10.48550/arXiv.1301. 2281.
Kita, H., 1999. A merging–giveway interaction model of cars in a merging section:
a game theoretic analysis. Transportation Research Part A: Policy and Practice 33, 305–312. doi:10.1016/S0965-8564(98)00039-1.
LeCun, Y., Bengio, Y., Hinton, G., 2015. Deep learning. Nature 521, 436–444. URL:
https://www.nature.com/articles/nature14539.
Lee, S., Younis, M., Murali, A., Lee, M., 2019. Dynamic local vehicular flow optimiza- tion using real-time traffic conditions at multiple road intersections. IEEE Access 7, 28137–28157. doi:10.1109/ACCESS.2019.2900360.
Lemke, C.E., Howson, Jr, J.T., 1964. Equilibrium points of bimatrix games.
Journal of the Society for Industrial and Applied Mathematics 12, 413–423.
doi:10.1137/0112033.
Li, H., Luo, Y., Wu, J., 2019. Collision-free path planning for intelli- gent vehicles based on Bézier curve. IEEE Access 7, 123334–123340.
doi:10.1109/ACCESS.2019.2938179.
Li, L., Lv, Y., Wang, F.Y., 2016. Traffic signal timing via deep rein- forcement learning. IEEE/CAA Journal of Automatica Sinica 3, 247–254.
doi:10.1109/JAS.2016.7508798.
Liang, X., Du, X., Wang, G., Han, Z., 2019. A deep reinforcement learning network for traffic light cycle control. IEEE Transactions on Vehicular Technology 68, 1243–
1253. doi:10.1109/TVT.2018.2890726.
Liu, B., Ding, Z., 2022. A distributed deep reinforcement learn- ing method for traffic light control. Neurocomputing 490, 390–399.
doi:10.1016/j.neucom.2021.11.106.
Liu, C., Lee, S., Varnhagen, S., Tseng, H.E., 2017. Path planning for autonomous vehi- cles using model predictive control, in: 2017 IEEE Intelligent Vehicles Symposium (IV), IEEE. pp. 174–179. doi:10.1109/IVS.2017.7995716.
Liu, H.X., Xin, W., Adam, Z., Ban, J., 2007. A game theoretical approach for modelling merging and yielding behaviour at freeway on-ramp sections. Transportation and Traffic Theory 3, 197–211.
Liu, Y., Wang, W., Hu, Y., Hao, J., Chen, X., Gao, Y., 2020. Multi-agent game abstrac- tion via graph attention neural network, in: Proceedings of the AAAI Conference on Artificial Intelligence, Association for the Advancement of Artificial Intelligence.
pp. 7211–7218. doi:10.1609/aaai.v34i05.6211.
Makantasis, K., Kontorinaki, M., Nikolos, I., 2019. A deep reinforcement learning driv- ing policy for autonomous road vehicles. ArXiv Preprint arXiv:1905.09046. URL:
https://doi.org/10.48550/arXiv.1905.09046.
Mao, H., Liu, W., Hao, J., Luo, J., Li, D., Zhang, Z., Wang, J., Xiao, Z., 2020. Neigh- borhood cognition consistent multi-agent reinforcement learning, in: Proceedings of the AAAI Conference on Artificial Intelligence, Association for the Advancement of Artificial Intelligence. pp. 7219–7226. doi:10.1609/aaai.v34i05.6212.
Marinescu, A., Dusparic, I., Taylor, A., Cahill, V., Clarke, S., 2014. Decentralised multi-agent reinforcement learning for dynamic and uncertain environments. ArXiv Preprint arXiv:1409.4561. URL:https://doi.org/10.48550/arXiv.1409. 4561.
Mukadam, M., Cosgun, A., Nakhaei, A., Fujimura, K., 2017. Tactical decision making for lane changing with deep reinforcement learning. URL: https://
openreview.net/pdf?id=B1G6uM0WG.
Pan, F., Zhang, L., Wang, J., Ma, C., Yang, J., Qi, J., 2020. Lane-changing risk anal- ysis in undersea tunnels based on fuzzy inference. IEEE Access 8, 19512–19520.
doi:10.1109/ACCESS.2020.2968584.
Van der Pol, E., 2016. Deep reinforcement learning for coordination in traffic light control. Master’s thesis, University of Amsterdam .
Rabby, M.K.M., Islam, M.M., Imon, S.M., 2019. A review of IoT applica- tion in a smart traffic management system, in: 2019 5th International Con- ference on Advances in Electrical Engineering (ICAEE), IEEE. pp. 280–285.
doi:10.1109/ICAEE48663.2019.8975582.
Sen, B., Smith, J.D., Najm, W.G., et al., 2003. Analysis of lane change crashes. Tech- nical Report. United States. National Highway Traffic Safety Administration. URL:
https://rosap.ntl.bts.gov/view/dot/6180.
Shalev-Shwartz, S., Shammah, S., Shashua, A., 2016. Safe, multi-agent, reinforce- ment learning for autonomous driving. ArXiv Preprint arXiv:1610.03295. URL:
https://arxiv.org/abs/1610.03295.
Sharifzadeh, S., Chiotellis, I., Triebel, R., Cremers, D., 2016. Learning to drive using inverse reinforcement learning and deep Q-networks. ArXiv Preprint arXiv:1612.03653. URL:https://doi.org/10.48550/arXiv.1612.03653.
Smirnov, N., Liu, Y., Validi, A., Morales-Alvarez, W., Olaverri-Monreal, C., 2021. A game theory-based approach for modeling autonomous vehicle behavior in congested, urban lane-changing scenarios. Sensors 21, 1523.
doi:10.3390/s21041523.
Steel, R.G.D., Torrie, J.H., et al., 1960. Principles and procedures of statistics. McGraw- Hill Book Company, Inc., New York, Toronto, London.
Steger-Vonmetz, D., 2005. Improving modal choice and transport efficiency with the virtual ridesharing agency, in: Proceedings. 2005 IEEE Intelligent Transportation Systems, 2005, IEEE. pp. 994–999. doi:10.1109/ITSC.2005.1520186.
Sutton, R.S., Barto, A.G., 2018. Reinforcement learning: An introduction. MIT Press.
Sutton, R.S., Barto, A.G., et al., 1998. Introduction to reinforcement learning. MIT Press Cambridge.
Szepesvári, C., Littman, M.L., 1999. A unified analysis of value-function- based reinforcement-learning algorithms. Neural Computation 11, 2017–2060.
doi:10.1162/089976699300016070.
Talebpour, A., Mahmassani, H.S., Hamdar, S.H., 2015. Modeling lane-changing behav- ior in a connected environment: A game theory approach. Transportation Research Procedia 7, 420–440. doi:10.1016/j.trpro.2015.06.022.
Tan, M., 1993. Multi-agent reinforcement learning: Independent vs. cooperative agents, in: Proceedings of the Tenth International Conference on Machine Learning, pp.
330–337.
Toledo, T., 2003. Integrating driving behavior modeling. Ph.D. thesis. Massachusetts Institute of Technology.
Toledo, T., Choudhury, C.F., Ben-Akiva, M.E., 2005. Lane-changing model with explicit target lane choice. Transportation Research Record 1934, 157–165.
doi:10.1177/0361198105193400117.
Touhbi, S., Babram, M.A., Nguyen-Huu, T., Marilleau, N., Hbid, M.L., Cambier, C., Stinckwich, S., 2017. Adaptive traffic signal control: Exploring reward def- inition for reinforcement learning. Procedia Computer Science 109, 513–520.
doi:10.1016/j.procs.2017.05.327.
Urbanik, T., Tanaka, A., Lozner, B., Lindstrom, E., Lee, K., Quayle, S., Beaird, S., Tsoi, S., Ryus, P., Gettman, D., Others, 2015. Signal timing manual. volume 1.
Transportation Research Board Washington, DC.
Vechione, M., Balal, E., Cheu, R.L., 2018. Comparisons of mandatory and discretionary lane changing behavior on freeways. International Journal of Transportation Science and Technology 7, 124–136. doi:10.1016/j.ijtst.2018.02.002.
Wall, G., Hounsell, N., 2005. Microscopic modelling of motorway diverges. European Journal of Transport and Infrastructure Research 5, 139–158. URL: https://
journals.open.tudelft.nl/ejtir/article/view/4397/4292.
Wang, H., Huang, Y., Khajepour, A., Zhang, Y., Rasekhipour, Y., Cao, D., 2019. Crash mitigation in motion planning for autonomous vehicles. IEEE Transactions on Intelli- gent Transportation Systems 20, 3313–3323. doi:10.1109/TITS.2018.2873921.
Wang, Y., Yang, X., Liang, H., Liu, Y., 2018. A review of the self-adaptive traffic signal control system based on future traffic environment. Journal of Advanced Transporta- tion 2018. doi:10.1155/2018/1096123.
Watkins, C.J., Dayan, P., 1992. Q-learning. Machine Learning 8, 279–
292. URL: https://link.springer.com/content/pdf/10.1007/
BF00992698.pdf.
Wu, Y.T., Ho, C.H., 2009. The development of taiwan arterial traffic-adaptive signal control system and its field test: A taiwan experience. Journal of Advanced Trans- portation 43, 455–480. doi:10.1002/atr.5670430404.
Xu, Y., Xi, Y., Li, D., Zhou, Z., 2016. Traffic signal control based on Markov decision process. IFAC-PapersOnLine 49, 67–72. doi:10.1016/j.ifacol.2016.07.012. Ye, F., Cheng, X., Wang, P., Chan, C.Y., Zhang, J., 2020. Automated lane change strategy using proximal policy optimization-based deep reinforcement learn- ing, in: 2020 IEEE Intelligent Vehicles Symposium (IV), IEEE. pp. 1746–1752.
doi:10.1109/IV47402.2020.9304668.
Yousef, K.M.A., Shatnawi, A., Latayfeh, M., 2019. Intelligent traffic light scheduling technique using calendar-based history information. Future Generation Computer Systems 91, 124–135. doi:10.1016/j.future.2018.08.037.
Yu, H., Tseng, H.E., Langari, R., 2018. A human-like game theory-based controller for automatic lane changing. Transportation Research Part C: Emerging Technologies 88, 140–158. doi:10.1016/j.trc.2018.01.016.
Zhang, L., Prieur, C., 2017. Stochastic stability of Markov jump hyper- bolic systems with application to traffic flow control. Automatica 86, 29–37.
doi:10.1016/j.automatica.2017.08.007.
Zhang, P., Zou, M., Li, X., Huang, Y., Ge, B., Yang, K., 2022. Control of traffic light timing using probability distribution genetic algorithm and finite state machine, in:
2nd International Conference on Applied Mathematics, Modelling, and Intelligent Computing (CAMMIC 2022), SPIE. pp. 167–176. doi:10.1117/12.2639045.
Zheng, Z., 2014. Recent developments and research needs in modeling lane changing. Transportation Research Part B: Methodological 60, 16–32.
doi:10.1016/j.trb.2013.11.009.
Zhu, F., Aziz, H.A., Qian, X., Ukkusuri, S.V., 2015. A junction-tree based learn- ing algorithm to optimize network wide traffic control: A coordinated multi-agent framework. Transportation Research Part C: Emerging Technologies 58, 487–501.
doi:10.1016/j.trc.2014.12.009.
Related Own Publications
Guo, J., Harmati, I., 2019a. Optimization of traffic signal control based on game theoretical framework, in: 2019 24th International Conference on Meth- ods and Models in Automation and Robotics (MMAR), IEEE. pp. 354–359.
doi:10.1109/MMAR.2019.8864629.
Guo, J., Harmati, I., 2019b. Optimization of traffic signal control based on game theo- retical framework, in: 2019 Proceedings of the Workshop on the Advances of Infor- mation Technology (WAIT), BME-IIT. pp. 105–110.
Guo, J., Harmati, I., 2019c. Optimization of traffic signal control with dif- ferent game theoretical strategies, in: 2019 23rd International Conference on System Theory, Control and Computing (ICSTCC), IEEE. pp. 750–755.
doi:10.1109/ICSTCC.2019.8885458.
Guo, J., Harmati, I., 2019d. Traffic signal control based on game theoretical framework, in: 2019 22nd International Conference on Process Control (PC19), IEEE. pp. 286–
291. doi:10.1109/PC.2019.8815096.
Guo, J., Harmati, I., 2020a. Comparison of game theoretical strategy and reinforcement learning in traffic light control. Periodica Polytechnica Transportation Engineering 48, 313–319. URL:https://pp.bme.hu/tr/article/view/15923.
Guo, J., Harmati, I., 2020b. Evaluating semi-cooperative Nash/Stackelberg Q-learning for traffic routes plan in a single intersection. Control Engineering Practice 102, 104525. doi:10.1016/j.conengprac.2020.104525.
Guo, J., Harmati, I., 2020c. Reinforcement learning for traffic signal control in decision combination, in: 2020 Proceedings of the Workshop on the Advances of Information Technology (WAIT), BME-IIT. pp. 13–20.
Guo, J., Harmati, I., 2021. Traffic lane-changing modeling and scheduling with game theoretical strategy, in: 2021 25th International Conference on Meth- ods and Models in Automation and Robotics (MMAR), IEEE. pp. 197–202.
doi:10.1109/MMAR49549.2021.9528432.
Guo, J., Harmati, I., 2022a. Lane-changing decision modelling in congested traffic with a game theory-based decomposition algorithm. Engineering Applications of Artificial Intelligence 107, 104530. doi:10.1016/j.engappai.2021.104530.
Guo, J., Harmati, I., 2022b. Lane-changing system based on deep Q-learning with a request-respond mechanism. Expert systems with applications , (Under re- view).doi:10.2139/ssrn.4139569.
He, H., Guo, J., Illés, B., Géczy, A., Istók, B., Hliva, V., Török, D., Kovács, J.G., Harmati, I., Molnár, K., 2021a. Monitoring multi-respiratory indices via a smart nanofibrous mask filter based on a triboelectric nanogenerator. Nano Energy 89, 106418. doi:10.1016/j.nanoen.2021.106418.
He, H., Guo, J., Molnár, K., 2021b. Triboelectric respiration monitoring sensor and a face mask comprising such as a triboelectric respiration monitoring sensor. Hungar- ian Patent, P2100102.
Taik, S., Guo, J., Kiss, B., Harmati, I., 2021. Demand response of multiple households with coordinated distributed energy resources, in: 2021 25th International Confer- ence on Methods and Models in Automation and Robotics (MMAR), IEEE. pp. 203–
208. doi:10.1109/MMAR49549.2021.9528490.
Appendix A
Application of Game Theory
In the field of smart grid, game theory can be utilized for consumers to reasonably trans- fer unconsumed electric energy to other households who need the energy (Taik et al., 2021). As shown in Fig.A.1, there are three households in the case, and each house- hold has two switches to transfer the electric energy to others. The switches control the energy flowing directions, i.e., transferring and receiving. Thus, each switch has two states (i.e., open and close), which can be represented by a binary code (1 and 0).
In a game theoretical framework, the households are considered the players, and the switches’ states (i.e., open-1 and close-0) are regarded as the decisions. There are two switches for each household, so the switch state can be defined as swi,j, where i represents the ith household and j represents jth switch of the household. The switch states swi,j of a household corresponding to the binary code can be converted into a decimal code dicorresponding to the decision vector, which can be shown in TableA.1.
The switches of households can be controlled to distribute the energy efficiently.
Households with more unconsumed electric energy could sell it to other households that lack electric energy instead of purchasing from the grid. Therefore, each household has to maximize the profits from selling the energy, and a utility function Uiindicating how much profit the household can get is defined as follows:
Ui(di) =
(Umin, Collision (A.1a)
|Powi| − |Powi(di)|, Otherwise (A.1b) where Powi and Powi(di)are the power value before and after exchanging the energy based on the decision di, respectively. A positive value of Powi means the household has extra energy to be sold and an energy shortage for a negative value. A negative and small value Umin would be assigned to the utility function if a collision happens
(e.g., household 1 is transmitting electric energy to household 2 while household 2 is transmitting electric energy to household 1).
Each household wants to maximize the profit to utilize the Nash equilibrium strategy for the most reasonable and balanced distribution of electric energy. None of the players can receive higher profits by changing their strategy while the others’ strategies are kept the same. The Nash equilibrium solution corresponding to the optimal strategy can be solved by:
U1(d1∗,d2∗,d3∗)≥U1(d1,d2∗,d3∗) (A.2a) U2(d1∗,d2∗,d3∗)≥U2(d1∗,d2,d3∗) (A.2b) U3(d1∗,d2∗,d3∗)≥U3(d1∗,d2∗,d3) (A.2c) where (d1∗,d2∗,d∗3) is the optimal decisions for each player, and each player tries to maximize the utility value.
Fig. A.1: Community-level framework of energy distribution.
Table A.1: The states of household corresponding to the binary code and decimal code
swi,1 1 1 0 0
swi,0 1 0 1 0
di 3 2 1 0
Appendix B
Application of Deep Learning
With the coronavirus disease 2019 outbreak, wearing a face mask is an effective man- ner to stop virus transmission and offer protection against the virus during the current pandemic. A changeable, self-powered, smart mask filter with high filtration efficiency and excellent breathability for monitoring respiratory state and protecting against par- ticulates is developed using nanofibers (He et al.,2021a,b). As shown in Fig.B.1, the working mechanism of the sensor is based on the coupling effect of contact electrifi- cation and electrostatic induction. There are no charges transported from two friction layers at the original state. During the breath cycles, the voltage output can be generated from the triboelectric nanogenerator by exhalation and inhalation. When exhalation oc- curs, the Polyacrylonitrile (PAN) nanofiber layer is blown up to contact the Polyvinyli- dene Fluoride (PVDF) layer. In this contact process, the PAN nanofiber layer tended to lose electrons and become positive, while the PVDF layer tended to gain electrons to generate negative charges. When inhalation starts, the PAN layer is sucked back to the original position; the two layers are separated. Due to the electric potential difference between the two electrodes, the transported electrons flow back from the PVDF layer to the PAN layer. The generated voltage signals from the sensor can be used to reflect the respiratory state of humans.
In the study of (He et al.,2021a), the smart mask filter can obtain real-time signals while wearing it. Nowadays, face recognition, fingerprint recognition, and acoustics recognition have been developed with the DL method, and it is evident that DL has so many advantages for identification. Thus, DL methods can also be applied further in this smart sensor for respiration recognition. The personal respiratory signals can be collected, analyzed and identified by the sensor with DL method, and features such as frequency, amplitude, and interval of peaks are extracted from the single waveform.