Abstract
Abstract
Objective: This study aimed to to find out the impact of chat bot use on researcher’s cognitive load and work performance: a study of chat bot user’s experience as a mediator
Study Design: A cross-sectional study was conducted.
Place and duration of study: The present research was carried out in different research settings such as universities, research labs, and research institutions of Rawalpindi and Islamabad, Pakistan.
Material and Methods: The Cross-sectional research design was used to find out the impact of chat bot use researcher’s cognitive load and work performance with the mediating effect of chat bot user’s experience. The sample consisted of 300 participants, including both male and female researchers, taken from different research settings of Rawalpindi and Islamabad, through purposive sampling technique. The age range of the participants was 25-44 years. Bot Usability Scale, (1) Cognitive Load Questionnaire, (2) Individual Work Performance Questionnaire, (3) and User Engagement Scale (4) were used to collect data. Pearson correlation, regression analysis and mediation analysis were done by using SPSS.
Results: The result showed that chat bot use had a weak but significant negative correlation with cognitive load (r = - .13*; p <.05) but had no meaningful correlation with work performance (r = .03; p>.05). Furthermore, indirect impact via chat bot user’s experience was found to be significant (β=-.20, SE=.10, 95% CI [-.42, -.03]), which means that chat bot user’s experience completely mediates the relation between chat bot use and cognitive load but not between chat bot usability and work performance.
Conclusion: The outcomes of this study may give insights into the creation and refinement of chat bot technologies targeted to research workflows, thereby increasing their utility and efficiency.
Keywords: Chat bot use, Cognitive Load, Work Performance, User Experience, Researchers
References
1. Amiot, C. (2020). Trustworthy chatbots assisting large-scale collaboration. https://scholar.archive.org/work/p5zzdvwjprdwzl3hakziwshjam/access/wayback/https://dl.eusset.eu/bitstream/20.500.12015/4048/1/ecscw2020_dc03.pdf
2. Arshad, N., Baber, M. U., & Ullah, A. (2024). Assessing the Transformative Influence of ChatGPT on Research Practices among Scholars in Pakistan. Mesopotamian Journal of Big Data, 2024, 1–10. https://doi.org/10.58496/MJBD/2024/001
3. Ashraf, M., Hafeez, R., & Sajid, A. N. (2022). Factors affecting the adoption of Fin-tech in Pakistan based on the Unified Theory of Acceptance and Use of Technology Model: An empirical study on financial inclusion in Pakistan. Journal of Financial Technologies (Fintech), Inclusion and Sustainability, 1(1), 9–26.
4. Behrooz, M., Ngan, W., Lane, J., Morse, G., Babcock, B., Shuster, K., Komeili, M., Chen, M., Kambadur, M., Boureau, Y.-L., & Weston, J. (2023). The HCI Aspects of Public Deployment of Research Chatbots: A User Study, Design Recommendations, and Open Challenges (arXiv:2306.04765). arXiv. https://doi.org/10.48550/arXiv.2306.04765
5. Bohomolova, I., Kushnir, N., & Moshkovska, S. (2021). Using Chatbots to Approach Individual Learning Trajectories in Physics for Foreign Students. ICTERI, 253–263. https://icteri.org/icteri-2021/proceedings/volume-1/20210253.pdf
6. Borsci, S., Malizia, A., Schmettow, M., van der Velde, F., Tariverdiyeva, G., Balaji, D., & Chamberlain, A. (2022). The Chatbot Usability Scale: The Design and Pilot of a Usability Scale for Interaction with AI-Based Conversational Agents. Personal and Ubiquitous Computing, 26(1), 95–119. https://doi.org/10.1007/s00779-021-01582-9
7. D’Haro, L. F., Kim, S., Yeo, K. H., Jiang, R., Niculescu, A. I., Banchs, R. E., & Li, H. (2015). CLARA: A Multifunctional Virtual Agent for Conference Support and Touristic Information. In G. G. Lee, H. K. Kim, M. Jeong, & J.-H. Kim (Eds.), Natural Language Dialog Systems and Intelligent Assistants (pp. 233–239). Springer International Publishing. https://doi.org/10.1007/978-3-319-19291-8_22
8. Dokukina, I., & Gumanova, J. (2020). The rise of chatbots–new personal assistants in foreign language learning. Procedia Computer Science, 169, 542–546.
9. Emon, M. M. H. (2023). Predicting Adoption Intention of ChatGPT-A Study on Business Professionals of Bangladesh. https://www.researchsquare.com/article/rs-3749611/latest
10. Følstad, A., & Brandtzaeg, P. B. (2020). Users’ experiences with chatbots: Findings from a questionnaire study. Quality and User Experience, 5(1), 3. https://doi.org/10.1007/s41233-020-00033-2
11. Følstad, A., Araujo, T., Law, E. L.-C., Brandtzaeg, P. B., Papadopoulos, S., Reis, L., Baez, M., Laban, G., McAllister, P., Ischen, C., Wald, R., Catania, F., Meyer von Wolff, R., Hobert, S., & Luger, E. (2021a). Future directions for chatbot research: An interdisciplinary research agenda. Computing, 103(12), 2915–2942. https://doi.org/10.1007/s00607-021-01016-7
12. Hauptmann, A., Magalhaes, J., Sousa, R. G., & Costeira, J. P. (2020). MuCAI’20: 1st International Workshop on Multimodal Conversational AI. Proceedings of the 28th ACM International Conference on Multimedia, 4767–4768. https://doi.org/10.1145/3394171.3421900
13. Jo, H., & Park, D.-H. (2024). AI in the Workplace: Examining the Effects of ChatGPT on Information Support and Knowledge Acquisition. International Journal of Human–Computer Interaction, 40(23), 8091–8106. https://doi.org/10.1080/10447318.2023.2278283
14. Kc, D. S., Staats, B. R., Kouchaki, M., & Gino, F. (2020). Task Selection and Workload: A Focus on Completing Easy Tasks Hurts Performance. Management Science, 66(10), 4397–4416. https://doi.org/10.1287/mnsc.2019.3419
15. King, J. L., & Ehrenberg, A. J. (2020). The Productivity Vampires. Information Systems Frontiers, 22(1), 11–15. https://doi.org/10.1007/s10796-019-09943-9
16. Kobiella, C., López, Y. S. F., Draxler, F., & Schmidt, A. (2024). “If the Machine Is As Good As Me, Then What Use Am I?”—How the Use of ChatGPT Changes Young Professionals’ Perception of Productivity and Accomplishment (arXiv:2404.12549). arXiv. https://doi.org/10.48550/arXiv.2404.12549
17. Kooli, C. (2023). Chatbots in Education and Research: A Critical Examination of Ethical Implications and Solutions. Sustainability, 15(7), 5614. https://doi.org/10.3390/su15075614
18. Krieglstein, F., Beege, M., Rey, G. D., Sanchez-Stockhammer, C., & Schneider, S. (2023b). Development and Validation of a Theory-Based Questionnaire to Measure Different Types of Cognitive Load. Educational Psychology Review, 35(1), 9. https://doi.org/10.1007/s10648-023-09738-0
19. Lee, P. Y., Salim, H., Abdullah, A., & Teo, C. H. (2023). Use of ChatGPT in medical research and scientific writing. Malaysian Family Physician, 18, 58. https://doi.org/10.51866/cm0006
20. Mamani, J. R. C., Álamo, Y. J. R., Aguirre, J. A. A., & Toledo, E. E. G. (2019). Cognitive services to improve user experience in searching for academic information based on chatbot. 2019 IEEE XXVI International Conference on Electronics, Electrical Engineering and Computing (INTERCON), 1–4. https://doi.org/10.1109/INTERCON.2019.8853572
21. Masoomi, H. A. S., Dharejo, N., Mahar, Z. A., Nazeer, M. I., Khoso, I. A., & Shah, A. (2024). Significant Predictors Influencing the Adoption of ChatGPT Usage in the Academia in Sindh, Pakistan: Extension of UTAUT Model. Pegem Journal of Education and Instruction, 14(4), 135–145. https://doi.org/10.47750/pegegog.14.04.14
22. Medhi Thies, I., Menon, N., Magapu, S., Subramony, M., & O’Neill, J. (2017). How Do You Want Your Chatbot? An Exploratory Wizard-of-Oz Study with Young, Urban Indians. In R. Bernhaupt, G. Dalvi, A. Joshi, D. K. Balkrishan, J. O’Neill, & M. Winckler (Eds.), Human-Computer Interaction—INTERACT 2017 (Vol. 10513, pp. 441–459). Springer International Publishing. https://doi.org/10.1007/978-3-319-67744-6_28
23. Montrieux, H., & Schellens, T. (2018). The impact of tablet devices on high school students’ cognitive load and learning. 12th International Technology, Education and Development Conference (INTED), 1591–1596. https://biblio.ugent.be/publication/8545807/file/8547411
24. Nakavachara, V., Potipiti, T., & Chaiwat, T. (2024). Experimenting with Generative AI: Does ChatGPT Really Increase Everyone’s Productivity? SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4746770
25. O’Brien, H. L., Cairns, P., & Hall, M. (2018b). A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. International Journal of Human-Computer Studies, 112, 28–39. https://doi.org/10.1016/j.ijhcs.2018.01.004
26. Okonkwo, C. W., & Ade-Ibijola, A. (2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2, 100033. https://doi.org/10.1016/j.caeai.2021.100033
27. Paas, F., & Van Merriënboer, J. J. G. (2020). Cognitive-Load Theory: Methods to Manage Working Memory Load in the Learning of Complex Tasks. Current Directions in Psychological Science, 29(4), 394–398. https://doi.org/10.1177/0963721420922183
28. Ramos-Villagrasa, P. J., Barrada, J. R., Fernández-del-Río, E., & Koopmans, L. (2019). Assessing job performance using brief self-report scales: The case of the individual work performance questionnaire. Revista de Psicología Del Trabajo y de Las Organizaciones, 35(3), 195–205.
29. Rostami, M., & Abadi, P. M. (2023). The Impact of Doing Assignments with Chatbots on The Students’ Working Memory. Health Nexus, 1(1), Article 1. https://doi.org/10.61838/kman.hn.1.1.10
30. Schmidhuber, J., Schlögl, S., & Ploder, C. (2021). Cognitive Load and Productivity Implications in Human-Chatbot Interaction. 2021 IEEE 2nd International Conference on Human-Machine Systems (ICHMS), 1–6. https://doi.org/10.1109/ICHMS53169.2021.9582445
31. Shaban, A., & Pearson, E. (2020). Evaluation of user experience and cognitive load of a gamified cognitive training application for children with learning disabilities. Proceedings of the 17th International Web for All Conference, 1–10. https://doi.org/10.1145/3371300.3383341
32. Spranger, M. D., Krishnan, A. C., Levy, P. D., O’Leary, D. S., & Smith, S. A. (2015). Blood flow restriction training and the exercise pressor reflex: A call for concern. American Journal of Physiology-Heart and Circulatory Physiology, 309(9), H1440–H1452. https://doi.org/10.1152/ajpheart.00208.2015
33. Sweller, J. (2011). CHAPTER TWO - Cognitive Load Theory. In J. P. Mestre & B. H. Ross (Eds.), Psychology of Learning and Motivation (Vol. 55, pp. 37–76). Academic Press. https://doi.org/10.1016/B978-0-12-387691-1.00002-8
34. Sweller, J. (2022). The Role of Evolutionary Psychology in Our Understanding of Human Cognition: Consequences for Cognitive Load Theory and Instructional Procedures. Educational Psychology Review, 34(4), 2229–2241. https://doi.org/10.1007/s10648-021-09647-0
35. Sweller, J., Van Merriënboer, J. J. G., & Paas, F. (2019). Cognitive Architecture and Instructional Design: 20 Years Later. Educational Psychology Review, 31(2), 261–292. https://doi.org/10.1007/s10648-019-09465-5
36. Talan, T., & Kalinkara, Y. (2023). The role of artificial intelligence in higher education: ChatGPT assessment for anatomy course. Uluslararası Yönetim Bilişim Sistemleri ve Bilgisayar Bilimleri Dergisi, 7(1), 33–40.
37. Ubah, A. E., Onakpojeruo, E. P., Ajamu, J., Mangai, T. R., Isa, A. M., Ayansina, N. B., & Al-Turjman, F. (2022). A Review of Artificial Intelligence in Education. 2022 International Conference on Artificial Intelligence of Things and Crowdsensing (AIoTCs), 38–45. https://ieeexplore.ieee.org/abstract/document/10102207/
38. Van Dis, E. A., Bollen, J., Zuidema, W., Van Rooij, R., & Bockting, C. L. (2023). ChatGPT: Five priorities for research. Nature, 614(7947), 224–226.
39. Wu, R., & Yu, Z. (2024). Do AI chatbots improve students learning outcomes? Evidence from a meta‐analysis. British Journal of Educational Technology, 55(1), 10–33. https://doi.org/10.1111/bjet.13334
40. Zhai, C., Wibowo, S., & Cowling, M. (2022). Work-in-Progress—Embedding cross-cultural humorous and empathetic functions to facilitate language acquisition. 2022 8th International Conference of the Immersive Learning Research Network (iLRN), 1–4. https://ieeexplore.ieee.org/abstract/document/9815992/
41. Zhang, H., Fan, L., Chen, M., & Qiu, C. (2022). The impact of SIPOC on process reengineering and sustainability of enterprise procurement management in e-commerce environments using deep learning. Journal of Organizational and End User Computing (JOEUC), 34(8), 1–17.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Copyright (c) 2026 Journal of Nursing and Allied Health