ChatGPT vs DeepL: Comparing the English Translation Quality of Digital Business and Information Technology Texts Using BLEU Metric
ChatGPT vs DeepL: Membandingkan Kualitas Terjemahan Bahasa Inggris Teks Bisnis Digital dan Teknologi Informasi Menggunakan Metrik BLEU
DOI:
https://doi.org/10.23971/jobit.v1i2.297Keywords:
AI, ChatGPT, DeepL, Translation, BLEU MetricAbstract
In the era of digital globalization, the demand for accurate and efficient translation tools has grown significantly. Advances in artificial intelligence (AI) have given rise to various automated translation platforms that facilitate cross-lingual communication. Among the most notable are ChatGPT and DeepL. This study focusses on translating digital business and information technology texts in English using both AI. All text is analyzed by determining their quality grammar and context-preserved, in addition, the accuracy assessment is being analyzed by using BLEU metric to ensure the translation quality threshold score. This study results that both have their own minuses and pluses. Both AI-translation tools have their strengths, with DeepL being slightly better for consistency in grammar rules and structure, while ChatGPT excels in clarity, accuracy, and adapting to the intended meaning. Meanwhile, the evaluation using the BLEU metric showed that ChatGPT scores ranged between 0.56-0.78, while DeepL scores ranged between 0.44-0.91.
References
Brown, T. B., et al. (2020). Language Models are Few-Shot Learners. ArXivLabs, 1(1), 1–63. https://doi.org/10.48550/arXiv.2005.14165
Dalayli, F. (2023). Use of NLP Techniques n Translation by ChatGPT: Case Study. Proceedings of the Workshop on Computational Terminology in NLP and Translation Studies (ConTeNTS) Incorporating the 16th Workshop on Building and Using Comparable Corpora (BUCC), 19–25. https://doi.org/10.26615/978-954-452-090-8_003
He, P., Meister, C., & Su, Z. (2021). Testing Machine Translation via Referential Transparency. Proceeding of 43rd International Conference on Software Engineering (ICSE), 410–422. https://doi.org/10.1109/ICSE43902.2021.00047
Himmah, E. F., & Kaestria, R. (2024). Pengaruh Durasi Penggunaan Gadget dan Durasi Belajar Mahasiswa Informatika terhadap Kemampuan Logika dan Hasil Belajar Matematika. Journal of Digital Business and Information Technology, 1(1), 9–21. https://doi.org/10.23971/jobit.v1i1.207
Kaestria, R., Himmah, E. F., & Irawan, R. (2024). Penerapan Matplotlib dalam Visualisasi Data untuk Analisis Hubungan Penggunaan Gadget dan Hasil Belajar. Journal of Digital Business and Information Technology, 1(1), 29–39. https://doi.org/10.23971/jobit.v1i1.204
Linlin, L. (2024). Artificial Intelligence Translator DeepL Translation Quality Control. Procedia Computer Science, 247(1), 710–717. https://doi.org/10.1016/j.procs.2024.10.086
Munasinghe, B., Bell, T., & Robins, A. (2021). Teachers’ Understanding of Technical Terms in a Computational Thinking Curriculum. Proceedings of the 23rd Australasian Computing Education Conference, 106–114. https://doi.org/10.1145/3441636.3442311
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2002). BLEU: A Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, 311–318. https://doi.org/10.3115/1073083.1073135
Popel, M., Tomkova, M., Tomek, J., Kaiser, Ł., Uszkoreit, J., Bojar, O., & Žabokrtský, Z. (2020). Transforming Machine Translation: A Deep Learning System Reaches News Translation Quality Comparable to Human Professionals. Nature Communications, 11(1), 4381–4395. https://doi.org/10.1038/s41467-020-18073-9
Popović, M. (2021). Agree to Disagree: Analysis of Inter-Annotator Disagreements in Human Evaluation of Machine Translation Output. 25th Conference on Computational Natural Language Learning (CoNLL), 234–243. https://github.com/m-popovic/
Saad, M. I., & Pratiwi, H. (2024). Development of a Web-Based Expert System for Diagnosing Cocoa Diseases Using Forward Chaining and Certainty Factor Methods. Journal of Digital Business and Information Technology, 1(1), 40–49. https://doi.org/10.23971/jobit.v1i1.223
Soysal, F. (2023). Enhancing Translation Studies with Artificial Intelligence (AI): Challenges, Opportunities, and Proposals. International Journal of Philology and Translation Studies, 5(2), 177–191. https://doi.org/10.55036/ufced.1402649
Surahmanto, M., Aras, S., Rifki, I. A. M., & Ussalama, P. (2024). Deteksi Jalan Berlubang Menggunakan Algoritma Yolov5. Journal of Digital Business and Information Technology, 1(1), 1–8. https://doi.org/10.23971/jobit.v1i1.198
Tamam, M. B., & Rofiuddin. (2024). Sistem Pendukung Keputusan Penentuan Lokasi Budidaya Ikan Tawar yang Cocok Dikabupaten Pamekasan dengan Menggunakan Metode Fuzzy Topsis. Journal of Digital Business and Information Technology, 1(1), 22–28. https://doi.org/10.23971/jobit.v1i1.203
Tursunovich, R. I. (2022). Linguistic and Cultural Aspects of Literary Translation and Translation Skills. British Journal of Global Ecology and Sustainable Development, 10(1), 168–173.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Haji Abdul Karim
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
