Systematic Review of Hyperparameter Adjustment and Evaluation Metrics in Bert-Based Sentiment Analysis
Abstract
The development of sentiment analysis towards Aspect-Based Sentiment Analysis (ABSA) has made significant progress thanks to deep learning technology, especially through the Bidirectional Encoder Representations from Transformers (BERT) architecture. Despite its increasing popularity, a comprehensive synthesis of global research patterns and optimal model configurations is still urgently needed. This study presents a Systematic Literature Review (SLR) combined with bibliometric analysis to examine BERT-based ABSA research indexed in Scopus. Using the PRISMA and VOSviewer frameworks for visualization, a total of 62 eligible articles up to mid-2025 were analyzed. The results of the study show a strong upward trend of publications with a peak in 2024, where China, India, and Indonesia emerged as the major contributors in this domain. Further, the review identified a critical technical standard for effective model training: the Adam optimizer was the most dominant choice, typically paired with a learning rate between 1e-5 to 2e-5 and a batch size of 16. Regarding performance evaluation, Accuracy and F1-Score are set as de facto standard metrics. These findings provide strategic guidance for researchers to optimize BERT implementation and identify future directions in more in-depth sentiment analysis tasks.
References
Ahmad, W., Khan, H. U., Alarfaj, F. K., & Alreshoodi, M. (2025). Aspect-Base Sentiment Analysis: A Comprehensive Review and Open Research Challenges. IEEE Access. https://doi.org/10.1109/ACCESS.2025.3555744
Alshaikh, K. A., Almatrafi, O. A., & Abushark, Y. B. (2024). BERT-based model for aspect-based sentiment analysis for analyzing Arabic open-ended survey responses: A case study. IEEE Access, 12, 2288–2302. https://doi.org/10.1109/ACCESS.2023.3348342
Alshattnawi, S., Shatnawi, A., AlSobeh, A. M., & Magableh, A. A. (2024). Beyond word-based model embeddings: Contextualized representations for enhanced social media spam detection. Applied Sciences, 14(6), 2254. https://doi.org/10.3390/app14062254
An, Z., Wan, H., Xiong, T., & Wang, B. (2025). LePB-SA4RE: A lexicon-enhanced and prompt-tuning BERT model for evolving requirements elicitation from app reviews. Applied Sciences, 15(5), Article 2282. https://doi.org/10.3390/app15052282
Aziz, K., Ji, D., Chakrabarti, P., Chakrabarti, T., Iqbal, M. S., & Abbasi, R. (2024). Unifying aspect-based sentiment analysis BERT and multi-layered graph convolutional networks for comprehensive sentiment dissection. Scientific reports, 14(1), 14646. https://doi.org/10.1038/s41598-024-61886-7
Bai, P., Xia, Y., & Xia, Y. (2020). Fusing knowledge and aspect sentiment for explainable recommendation. IEEE Access, 8, 137150–137160. https://doi.org/10.1109/ACCESS.2020.3012347
Chao, K., Sarker, M. N. I., Ali, I., Firdaus, R. R., Azman, A., & Shaed, M. M. (2023). Big data-driven public health policy making: Potential for the healthcare industry. Heliyon, 9(9). https://doi.org/10.1016/j.heliyon.2023.e19681
Chauhan, A., & Mohana, R. (2023). Aspect-level sentiment modeling using transformer architectures. Scalable Computing: Practice and Experience, 24(3), 277–286. https://doi.org/10.12694/scpe.v24i3.2444
Chauhan, A., Sharma, A., & Mohana, R. (2025). An enhanced aspect-based sentiment analysis model based on RoBERTa. Computer Systems Science and Engineering, 49, 193–202.
Deb, S., & Chanda, A. K. (2022). Comparative analysis of contextual and context-free embeddings in disaster prediction from Twitter data. Machine Learning with Applications, 7, 100253. https://doi.org/10.1016/j.mlwa.2022.100253
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Cornell University. https://arxiv.org/abs/1810.04805
Dhanal, R. J., & Ghorpade, V. R. (2024). Aspect-based sentiment analysis using topic modelling and machine learning. International Journal of Electrical and Computer Engineering, 14(6), 6689–6698. https://doi.org/10.11591/ijece.v14i6.pp6689-6698
Feng, A., Liu, T., Li, X., Jia, K., & Gao, Z. (2024). Dual syntax-aware graph attention networks with prompt for aspect-based sentiment analysis. Scientific Reports, 14(1), Article 74668. https://doi.org/10.1038/s41598-024-74668-y
Feng, J., Cai, S., Li, K., Chen, Y., Cai, Q., & Zhao, H. (2023). Fusing syntax- and semantics-based graph convolutional network for aspect-based sentiment analysis. International Journal of Data Warehousing and Mining, 19(1). https://doi.org/10.4018/IJDWM.319803
Geetha, M. P., & Renuka, D. K. (2021). Improving the performance of aspect-based sentiment analysis using fine-tuned BERT base uncased model. International Journal of Intelligent Networks, 2, 64–69. https://doi.org/10.1016/j.ijin.2021.06.005
Hossain, Q., Yasmin, F., Biswas, T. R., & Asha, N. B. (2024). Data-Driven Business Strategies: A Comparative Analysis of Data Science Techniques in Decision-Making. Sch J Econ Bus Manag, 9, 257-263. https://doi.org/10.36347/sjebm.2024.v11i09.002
Hua, Y. C., Denny, P., Wicker, J., & Taskova, K. (2024). A systematic review of aspect-based sentiment analysis: domains, methods, and trends. Artificial Intelligence Review, 57(11), 296. https://doi.org/10.1007/s10462-024-10906-z
Jazuli, A., & Kusumaningrum, R. (2025). Optimizing aspect-based sentiment analysis using BERT for comprehensive analysis of Indonesian student feedback. Applied Sciences, 15(1), Article 172. https://doi.org/10.3390/app15010172
Karakolias, S. (2024). Mapping data-driven strategies in improving health care and patient satisfaction. World Journal of Advanced Engineering Technology and Sciences. https://doi.org/10.30574/wjaets.2024.13.1.0444
Kotei, E., & Thirunavukarasu, R. (2023). A systematic review of transformer-based pre-trained language models through self-supervised learning. Information, 14(3), 187.
Marutho, D., & Rustad, S. (2024). Optimizing aspect-based sentiment analysis using sentence embedding transformer, Bayesian search clustering, and sparse attention mechanism. Journal of Open Innovation: Technology, Market, and Complexity, 10(1), Article 100211. https://doi.org/10.1016/j.joitmc.2024.100211
Nusantara, U. M. (2022). Comparison of IndoBERT-Lite and RoBERTa in text mining for Indonesian language question answering application. International Journal of Innovative Computing, Information and Control, 18(6), 1719–1734. https://doi.org/10.24507/ijicic.18.06.1719
Onesi-Ozigagun, O., Ololade, Y. J., Eyo-Udo, N. L., & Oluwaseun, D. (2024). Data-driven decision making: Shaping the future of business efficiency and customer engagement. International Journal of Multidisciplinary Research Updates, 7(2), 19-29. https://doi.org/10.53430/ijmru.2024.7.2.0031
Patil, R., Boit, S., Gudivada, V., & Nandigam, J. (2023). A survey of text representation and embedding techniques in nlp. IEEe Access, 11, 36120-36146. https://doi.org/10.1109/ACCESS.2023.3266377
Pranatawijaya, V. H., Sari, N. N. K., Rahman, R. A., Christian, E., & Geges, S. (2024). Unveiling user sentiment: Aspect-based analysis and topic modeling of ride-hailing and Google Play app reviews. Journal of Information Systems Engineering and Business Intelligence, 10(3), 328–339. https://doi.org/10.20473/jisebi.10.3.328-339
Qiu, X., Sun, T., Xu, Y., & Huang, X. (2020). Pre-trained models for natural language processing: A survey. Science China Technological Sciences, 63(10), 1872–1897.
Rajan, A., & Manur, M. (2024). Aspect-based sentiment analysis using a novel ensemble deep network. IAES International Journal of Artificial Intelligence, 13(2), 1668–1678. https://doi.org/10.11591/ijai.v13.i2.pp1668-1678
Samant, R. M., Bachute, M. R., Gite, S., & Kotecha, K. (2022). Framework for deep learning-based language models using multi-task learning in natural language understanding: A systematic literature review and future directions. IEEE Access, 10, 17078–17097. https://doi.org/10.1109/ACCESS.2022.3149798
Santiago, M., & Ludeña, V. (2024). Aspect-based sentiment analysis using social media data. Advances in Hospitality and Tourism Research. https://doi.org/10.30519/ahtr.1436175
Shaik, F. A., & Oussalah, M. (2024). Mining mobile emergency communication applications in Nordic countries. International Journal of Disaster Risk Reduction, 108, Article 104566. https://doi.org/10.1016/j.ijdrr.2024.104566
Sirisha, U., & Chandana, B. S. (2022). Aspect-based sentiment and emotion analysis. International Journal of Engineering Research and Technology, 13(11).
Wang, Y., Yang, N., Miao, D., & Chen, Q. (2024). Aspect-guided multi-graph convolutional networks for aspect-based sentiment analysis. Data Intelligence, 6(3), 771–791. https://doi.org/10.3724/2096-7004.di.2024.0052
Yang, R., Ju, X., Guo, C., Ding, L., Li, M., & Zhang, B. (2025). A unified review of aspect sentiment triplet extraction methods in aspect-based sentiment analysis: R. Yang et al. Knowledge and Information Systems, 67(11), 9779-9820. https://doi.org/10.1007/s10115-025-02519-x
Yulianti, E., & Nissa, N. K. (2024). Aspect-based sentiment analysis of Indonesian customer reviews using IndoBERT: Single-sentence and sentence-pair classification approaches. Bulletin of Electrical Engineering and Informatics, 13(5), 3579–3589. https://doi.org/10.11591/eei.v13i5.8032
Copyright (c) 2026 Journal La Multiapp
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
