Evaluation of GRU with Attention and DistilBERT in Text Classification Tasks
DOI:
https://doi.org/10.61467/2007.1558.2026.v17i2.1268Keywords:
Cyberbullying, Machine Learning, Deep LearningAbstract
In this paper compares two deep learning models for multiclass text classification in the cyberbullying domain. The corpus is written in English and includes six classes representing different types of harassment. The trained models are a GRU network with an attention mechanism and the DistilBERT transformer. The aim applies to a 5-fold cross-validation scheme. And uses accuracy, precision, recall, and F1 as evaluation metrics. The GRU model achieves significantly higher performance (accuracy = 0.83, F1 = 0.45) than DistilBERT (accuracy = 0.018, F1 = 0.004). The results obtained indicate that the GRU architecture with word embedding performs more effectively on this task. This is due to the dataset’s linguistic characteristics, as well as the limited fine-tuning capacity of the lightweight transformer. The findings emphasize the need to select model architectures that align with the corpus properties and the classification task’s complexity. Future work will involve fine-tuning strategies and evaluating additional transformer-based models.
Smart citations: https://scite.ai/reports/10.61467/2007.1558.2026.v17i2.1268
Dimensions.
Open Alex.
References
Aldhyani, T. H., Al-Adhaileh, M. H., & Alsubari, S. N. (2022). Cyberbullying identification system based on deep learning algorithms. Electronics, 11(20), 3273. https://doi.org/10.3390/electronics11203273
Almufareh, M. F., Jhanjhi, N., Humayun, M., Alwakid, G. N., Javed, D., & Almuayqil, S. N. (2025). Integrating sentiment analysis with machine learning for cyberbullying detection on social media. IEEE Access.
Anand, M., Sahay, K. B., Ahmed, M. A., Sultan, D., Chandan, R. R., & Singh, B. (2023). Deep learning and natural language processing in computation for offensive language detection in online social networks by feature selection and ensemble classification techniques. Theoretical Computer Science, 943, 203–218. https://doi.org/10.1016/j.tcs.2022.11.047
Barlett, C. P. (2023). Cyberbullying as a learned behavior: Theoretical and applied implications. Children, 10(2), 325. https://doi.org/10.3390/children10020325
Devi, K. N., Jayanthi, P., Rajasekar, V., Rathinasamy, R., Dhandapani, P., & Risam, R. (2025, February). Cyberbullying detection using deep learning algorithms. In 2025 3rd International Conference on Intelligent Data Communication Technologies and Internet of Things (IDCIoT) (pp. 1298–1306). IEEE.
Ghosh, R., Malhotra, M., & Kumar, N. (2025). Cyber bullying in the digital age: Challenges, impact, and strategies for prevention. In Combating cyberbullying with generative AI (pp. 151–180). IGI Global.
Hasan, M. T., Hossain, M. A. E., Mukta, M. S. H., Akter, A., Ahmed, M., & Islam, S. (2023). A review on deep-learning-based cyberbullying detection. Future Internet, 15(5), 179. https://doi.org/10.3390/fi15050179
Kazan, H. (2022). Cyber bullying and violence literacy in the context of digitalization. In Research anthology on combating cyber-aggression and online negativity (pp. 496–519). IGI Global.
Nghiem, H., Gupta, U., & Morstatter, F. (2024). “Define your terms”: Enhancing efficient offensive speech classification with definition (arXiv:2402.03221). https://arxiv.org/abs/2402.03221
Sultan, D., Toktarova, A., Zhumadillayeva, A., Aldeshov, S., Mussiraliyeva, S., Beissenova, G., & Imanbayeva, A. (2023). Cyberbullying-related hate speech detection using shallow-to-deep learning. Computers, Materials & Continua, 75(1), 2115–2131. https://doi.org/10.32604/cmc.2023.032993
Wang, J., Fu, K., & Lu, C.-T. (2020). SosNet: A graph convolutional network approach to fine-grained cyberbullying detection. En 2020 IEEE International Conference on Big Data (pp. 1699–1708). IEEE.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 International Journal of Combinatorial Optimization Problems and Informatics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
