Journal of Electrical and Computer Engineering Innovations (JECEI)
مقاله 3 ، دوره 10، شماره 2 ، مهر 2022، صفحه 287-298 783.52 K )
نوع مقاله: Original Research Paper
شناسه دیجیتال (DOI): 10.22061/jecei.2021.8051.475 نویسندگان
S.V. Moravvej* 1 M.J. Maleki Kahaki 2 M. Salimi Sartakhti 3 M. Joodaki 1 1 Department of Electrical and Computer Engineering, Isfahan University of Technology, Isfahan, Iran.2 Department of Electrical and Computer Engineering, University of Kashan, Kashan, Iran.3 Department of Electrical and Computer Engineering, Amirkabir University of Technology, Tehran, Iran.
تاریخ دریافت : 24 تیر 1400 ،
تاریخ بازنگری : 10 آبان 1400 ،
تاریخ پذیرش : 17 آبان 1400
چکیده Background and Objectives: Text summarization plays an essential role in reducing time and cost in many domains such as medicine, engineering, etc. On the other hand, manual summarization requires much time. So, we need an automated system for summarizing. How to select sentences is critical in summarizing. Summarization techniques that have been introduced in recent years are usually greedy in the choice of sentences, which leads to a decrease in the quality of the summary. In this paper, a non-greedily method for selecting essential sentences from a text is presented.Methods: The present paper presents a method based on a generative adversarial network and attention mechanism called GAN-AM for extractive summarization. Generative adversarial networks have two generator and discriminator networks whose parameters are independent of each other. First, the features of the sentences are extracted by two traditional and embedded methods. We extract 12 traditional features. Some of these features are extracted from sentence words and others from the sentence. In addition, we use the well-known Skip-Gram model for embedding. Then, the features are entered into the generator as a condition, and the generator calculates the probability of each sentence in summary. A discriminator is used to check the generated summary of the generator and to strengthen its performance. We introduce a new loss function for discriminator training that includes generator output, real and fake summaries of each document. During training and testing, each document enters the generator with different noises. It allows the generator to see many combinations of sentences that are suitable for quality summaries.Results: We evaluate our results on CNN/Daily Mail and Medical datasets. Summaries produced by the generator show that our model performs better than other methods compared based on the ROUGE metric. We apply different sizes of noise to the generator to check the effect of noise on our model. The results indicate that the noise-free model has poor results.Conclusion: Unlike recent works, in our method, the generator selects sentences non-greedily. Experimental results show that the generator with noise can produce summaries that are related to the main subject.کلیدواژهها
Text Summarization ؛ Non-greedily ؛ Generative Adversarial Network ؛ Attention Mechanism ؛ Extractive Summarization
مراجع
[1] S. Samiei, M. Joodaki, N. Ghadiri, "A scalable pattern mining method using apache spark platform," in Proc. 7th International Conference on Web Research (ICWR): 114-118, 2021.
[2] S.V. Moravvej, A. Mirzaei, M. Safayani, "Biomedical text summarization using Conditional Generative Adversarial Network (CGAN)," arXiv preprint arXiv:2110.11870, 2021.
[3] R. Mitkov, The Oxford handbook of computational linguistics. Oxford University Press, 2004.
[4] I. Mani, M. Maybury, "Automatic summarization John Benjamin’s publishing Co," 2001.
[5] M. Joodaki, N. Ghadiri, Z. Maleki, M.L. Shahreza, "A scalable random walk with restart on heterogeneous networks with Apache Spark for ranking disease-related genes through type-II fuzzy data fusion," J. Biomed. Inf., 115: 103688, 2021.
[6] M. Kågebäck, O. Mogren, N. Tahmasebi, D. Dubhashi, "Extractive summarization using continuous vector space models," in Proc. 2nd Workshop on Continuous Vector Space Models and their Compositionality (CVSC): 31-39, 2014.
[7] M. Jang, P. Kang, "Learning-free unsupervised extractive summarization model," IEEE Access, 9: 14358-14368, 2021.
[8] Y. Liu, P. Liu, "SimCLS: A simple framework for contrastive learning of abstractive summarization," arXiv preprint arXiv:2106.01890, 2021.
[9] Z. Cao, F. Wei, W. Li, S. Li, "Faithful to the original: Fact aware neural abstractive summarization," in Proc. the AAAI Conference on Artificial Intelligence, 32(1), 2018.
[10] Y. Wu, B. Hu, "Learning to extract coherent summary via deep reinforcement learning," in Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
[11] A. Abdi, S. Hasan, S.M. Shamsuddin, N. Idris, J. Piran, "A hybrid deep learning architecture for opinion-oriented multi-document summarization based on multi-feature fusion," Knowledge-Based Syst., 213: 106658, 2021.
[12] S. Lamsiyah, A. El Mahdaouy, B. Espinasse, S.E.A. Ouatik, "An unsupervised method for extractive multi-document summarization based on centroid approach and sentence embeddings," Expert Syst. Appl., 167: 114152, 2021.
[13] S. Murarka, A. Singhal, "Query-based single document summarization using hybrid semantic and graph-based approach," in Proc. 2020 International Conference on Advances in Computing, Communication & Materials (ICACCM): 330-335, 2020.
[14] S. Akter, A.S. Asa, M.P. Uddin, M.D. Hossain, S.K. Roy, M.I. Afjal, "An extractive text summarization technique for Bengali document (s) using K-means clustering algorithm," in Proc. 2017 IEEE International Conference on Imaging, Vision & Pattern Recognition (icIVPR): 1-6 , 2017.
[15] M. Joodaki, N. Ghadiri, A.H. Atashkar, "Protein complex detection from PPI networks on Apache Spark," in Proc. 2017 9th International Conference on Information and Knowledge Technology (IKT): 111-115, 2017.
[16] T. Hirao, H. Isozaki, E. Maeda, Y. Matsumoto, "Extracting important sentences with support vector machines," in Proc. COLING 2002: The 19th International Conference on Computational Linguistics, 2002.
[17] S.Y. Ashkoofaraz, S.N.H. Izadi, M. Tajmirriahi, M. Roshanzamir, M.A. Soureshjani, S.V. Moravvej, M. Palhang, AIUT3D 2018 Soccer Simulation 3D League Team Description Paper.
[18] S. Vakilian, S.V. Moravvej, A. Fanian, "Using the Artificial Bee Colony (ABC) algorithm in collaboration with the fog nodes in the internet of things three-layer architecture," in Proc. 29th Iranian Conference on Electrical Engineering (ICEE): 509-513, 2021.
[19] S. H. Mirshojaei, B. Masoomi, "Text summarization using cuckoo search optimization algorithm," J. Comp. Rob., 8(2): 19-24, 2015.
[20] S. Vakilian, S. V. Moravvej, A. Fanian, "Using the cuckoo algorithm to optimizing the response time and energy consumption cost of fog nodes by considering collaboration in the fog layer," in Proc. 2021 5th International Conference on Internet of Things and Applications (IoT): 1-5, 2021.
[21] F.B. Goularte, S.M. Nassar, R. Fileto, H. Saggion, "A text summarization method based on fuzzy rules and applicable to automated assessment," Expert Syst. Appl., 115: 264-275, 2019.
[22] W.S. El-Kassas, C.R. Salama, A.A. Rafea, H.K. Mohamed, "EdgeSumm: Graph-based framework for automatic text summarization," Inf. Proc. Manage., 57(6): 102264, 2020.
[23] S.V. Moravvej, M. Joodaki, M.J.M. Kahaki, M.S. Sartakhti, "A method based on an attention mechanism to measure the similarity of two sentences," in Proc. 2021 7th International Conference on Web Research (ICWR): 238-242,2021.
[24] M.S. Sartakhti, M.J.M. Kahaki, S.V. Moravvej, M. javadi Joortani, A. Bagheri, "Persian language model based on BiLSTM model on COVID-19 corpus," in Proc. 2021 5th International Conference on Pattern Recognition and Image Analysis (IPRIA): 1-5, 2021.
[25] S.V. Moravvej, M.J.M. Kahaki, M.S. Sartakhti, A. Mirzaei, "A method based on attention mechanism using Bidirectional Long-Short Term Memory (BLSTM) for Question Answering," in 2021 29th Iranian Conference on Electrical Engineering (ICEE): 460-464, 2021.
[26] H. Liang, X. Sun, Y. Sun, Y. Gao, "Text feature extraction based on deep learning: a review," EURASIP J. Wireless Commun. Networking, 2017(1): 1-12, 2017.
[27] Z. Cao, F. Wei, L. Dong, S. Li, M. Zhou, "Ranking with recursive neural networks and its application to multi-document summarization," in Proc. the AAAI Conference on Artificial Intelligence, 29(1):2015.
[28] R. Nallapati, F. Zhai, B. Zhou, "Summarunner: A recurrent neural network based sequence model for extractive summarization of documents," in Proc. Thirty-First AAAI Conference on Artificial Intelligence: 3075-3081, 2017.
[29] J.Á. González, E. Segarra, F. García-Granada, E. Sanchis, L.ı.F. Hurtado, "Siamese hierarchical attention networks for extractive summarization," J. Intell. Fuzzy Syst., 36(5): 4599-4607, 2019.
[30] K. Al-Sabahi, Z. Zuping, M. Nadher, "A hierarchical structured self-attentive model for extractive document summarization (HSSAS)," IEEE Access, 6: 24205-24212, 2018.
[31] W. Nie, N. Narodytska, A. Patel, "Relgan: Relational generative adversarial networks for text generation," in Proc. International conference on learning representations, 2018.
[32] M. Lewis, A. Fan, "Generative question answering: Learning to answer the whole question," in Proc. International Conference on Learning Representations, 2018.
[33] T. Vo, "SGAN4AbSum: A Semantic-Enhanced Generative Adversarial Network for Abstractive Text Summarization," 2021.
[34] R. Paulus, C. Xiong, R. Socher, "A deep reinforced model for abstractive summarization," arXiv preprint arXiv:1705.04304, 2017.
[35] M. Moradi, "Frequent itemsets as meaningful events in graphs for summarizing biomedical texts," in Proc. 2018 8th International Conference on Computer and Knowledge Engineering (ICCKE): 135-140, 2018.
[36] Z. Wu, R. Koncel-Kedziorski, M. Ostendorf, H. Hajishirzi, "Extracting summary knowledge graphs from long documents," arXiv preprint arXiv:2009.09162, 2020.
[37] T. Safavi, C. Belth, L. Faber, D. Mottin, E. Müller, D. Koutra, "Personalized knowledge graph summarization: From the cloud to your pocket," in Proc. 2019 IEEE International Conference on Data Mining (ICDM): 528-537, 2019.
[38] R. Mihalcea, P. Tarau, "Textrank: Bringing order into text," in Proc. 2004 Conference on Empirical Methods in Natural Language Processing: 404-411, 2004.
[39] S. H. Zhong, Y. Liu, B. Li, J. Long, "Query-oriented unsupervised multi-document summarization via deep learning model," Expert Syst. Appl., 42(1): 8146-8155, 2015.
[40] M. Yousefi-Azar, L. Hamey, "Text summarization using unsupervised deep learning," Expert Syst.Appl., 68: 93-105, 2017.
[41] S. V. Moravvej, S.J. Mousavirad, M.H. Moghadam, M. Saadatmand, "An LSTM-based plagiarism detection via attention mechanism and a population-based approach for pre-training parameters with imbalanced classes," arXiv preprint arXiv:2110.08771, 2021.
[42] Q. Dou, Y. Lu, P. Manakul, X. Wu, M.J. Gales, "Attention forcing for machine translation," arXiv preprint arXiv:2104.01264, 2021.
[43] Y. Zhang, Y. Peng, "Research on answer selection based on LSTM," in Proc. 2018 International Conference on Asian Language Processing (IALP): 357-361, 2018.
[44] J. Devlin, M.W. Chang, K. Lee, K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018.
[45] Y. Liu, "Fine-tune BERT for extractive summarization," arXiv preprint arXiv:1903.10318, 2019.
[46] Y. Liu and M. Lapata, "Text summarization with pretrained encoders," arXiv preprint arXiv:1908.08345, 2019.
[47] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, "Generative adversarial nets," Advances in Neural Information Processing Systems, 27, 2014.
[48] M. Mirza, S. Osindero, "Conditional generative adversarial nets," arXiv preprint arXiv:1411.1784, 2014.
[49] T. Mikolov, K. Chen, G. Corrado, J. Dean, "Efficient estimation of word representations in vector space," arXiv preprint arXiv:1301.3781, 2013.
[50] K. M. Hermann, T. Kocisky, E. Grefenstette, L. Espeholt, W. Kay, M. Suleyman, P. Blunsom, "Teaching machines to read and comprehend," Advances in Neural Information Processing Systems, 28: 1693-1701, 2015.
[51] C.Y. Lin, "Rouge: A package for automatic evaluation of summaries," in Text summarization branches out: 74-81,2004.
آمار
تعداد مشاهده مقاله: 1,418
تعداد دریافت فایل اصل مقاله: 673
