Explainable AI Supporting Responsible Human AI Interaction in Intelligent Decision Support Systems

Cut Amalia Saffiera; Takumi Sase; Qurotul Aini; Danny Manongga; Harry Agustian

Authors

Cut Amalia Saffiera Universitas Islam Kebangsaan Indonesia https://orcid.org/0009-0006-1693-5814
Takumi Sase International Islamic University Malaysia https://orcid.org/0000-0003-0240-946X
Qurotul Aini Satya Wacana Christian University https://orcid.org/0000-0002-7546-5721
Danny Manongga Satya Wacana Christian University https://orcid.org/0000-0002-7430-8740
Harry Agustian Sesindo Group https://orcid.org/0009-0006-8012-2260

Keywords:

Explainable Artificial Intelligence, Responsible AI, Human-AI Interaction, Intelligent Decision Support Systems, Decision Quality

Abstract

The rapid adoption of Artificial Intelligence (AI)-enabled Intelligent Decision Support Systems (IDSS) has transformed decision-making across multiple sectors. However, increasing AI complexity often limits users’ understanding of recommendation processes, creating challenges for responsible human-AI interaction. Existing studies mainly emphasize explainability and transparency from technical perspectives while providing limited evidence regarding their influence on responsible interaction and decision quality. This study investigates the effects of Perceived Explainability and Perceived Transparency on Responsible Human-AI Interaction and Decision Quality, including the mediating role of Responsible Human-AI Interaction. Grounded in the Human-AI Teaming perspective and the Responsible AI Framework, this quantitative study employs a survey of 180 respondents with experience using AI-enabled Intelligent Decision Support Systems. Data will be analyzed using Partial Least Squares Structural Equation Modeling (PLS-SEM). The study is expected to demonstrate that explainability and transparency strengthen responsible human-AI interaction, which subsequently enhances decision quality. These findings are expected to enrich responsible AI literature and provide practical guidance for designing transparent, human-centered Intelligent Decision Support Systems that promote trustworthy decision-making and reinforce the humanistic values underpinning intelligent technologies

References

[1] A. Qazi and S. Ullah, “Human–ai collaboration in intelligent decision support systems,” Multidisciplinary Research in Computing Information Systems, vol. 5, no. 9, pp. 691–702, 2025.

[2] H. Nurhaeni, S. Kosasi, M. B. Thalia, E. A. Natalia, and J. Edwards, “Sentiment aware chatbots as companions for reducing loneliness through positive computing,” Journal of Orange Technology, vol. 1, no. 1, pp. 39–50, 2024.

[3] S. Baker and W. Xiang, “Explainable ai is responsible ai: How explainability creates trustworthy and socially responsible artificial intelligence,” arXiv preprint arXiv:2312.01555, 2023.

[4] M. Fahrurrozi, S. R. Djalilah, D. Yunitasari, M. Mohzana, and H. S. Butt, “Development of technopreneurship based e-modules on entrepreneurial learning,” Aptisi Transactions on Technopreneurship (ATT), vol. 8, no. 3, pp. 795–808, 2026.

[5] K. Baum, S. Mantel, E. Schmidt, and T.Speith,“From responsibility to reason-giving explainable artificial intelligence,” Philosophy & Technology, vol. 35, no. 1, p. 12, 2022.

[6] V. Chamola, V. Hassija, A. R. Sulthana, D. Ghosh, D. Dhingra, and B. Sikdar, “A review of trustworthy and explainable artificial intelligence (xai),” IEEe Access, vol. 11, pp. 78994–79015, 2023.

[7] H.Handoko, S. B. Waluya, W. Wardono, S. Sugiman, and L. W. Ming, “Geometry learning pathways for creative thinking via edupreneurial approaches,” Aptisi Transactions on Technopreneurship (ATT), vol. 8, no. 2, pp. 669–681, 2026.

[8] U. Ehsan, P. Wintersberger, Q. V. Liao, E. A. Watkins, C. Manger, H. Daum´ e III, A. Riener, and M. O. Riedl, “Human-centered explainable ai (hcxai): beyond opening the black-box of ai,” in CHI conference on human factors in computing systems extended abstracts, 2022, pp. 1–7.

[9] J. Qadir, M. Q. Islam, and A. Al-Fuqaha, “Toward accountable human-centered ai: rationale and promising directions,” Journal of Information, Communication and Ethics in Society, vol. 20, no. 2, pp. 329–342, 2022.

[10] K. Kalasampath, K. Spoorthi, S. Sajeev, S. S. Kuppa, K. Ajay, and A. Maruthamuthu, “A literature review on applications of explainable artificial intelligence (xai),” IEEE access, vol. 13, pp. 41111–41140, 2025.

[11] S. L. Sitorus, A. Hermawan, A. W. Widjaja, and M. P. Berlianto, “Enhancing digital fashionpreneurship through innovation capability and market sensing,” Aptisi Transactions on Technopreneurship (ATT), vol. 8, no. 2, pp. 645–658, 2026.

[12] D. E. Mathew, D. U. Ebem, A. C. Ikegwu, P. E. Ukeoma, and N. F. Dibiaezue, “Recent emerging techniques in explainable artificial intelligence to enhance the interpretable and understanding of ai models for human: De mathew et al.” Neural processing letters, vol. 57, no. 1, p. 16, 2025.

[13] C. Meske, B. Abedin, M. Klier, and F. Rabhi, “Explainable and responsible artificial intelligence: Explainable and responsible artificial intelligence,” Electronic Markets, vol. 32, no. 4, pp. 2103–2106, 2022.

[14] N. Azizah, P. A. Sunarya, U. Rahardja, A. B. Mutiara, P. Prihandoko, and C. Pasha, “Improving smear negative tuberculosis detection using data augmentation and faster r-cnn,” International Journal of Cyber and IT Service Management (IJCITSM), vol. 6, no. 1, pp. 65–77, 2026.

[15] P. N. Mahalle, R. V. Patil, N. Dey, R. G. Crespo, R. S. Sherratt et al., “Explainable ai for human-centric ethical iot systems,” IEEE Transactions on Computational Social Systems, vol. 11, no. 3, pp. 3407–3419, 2023.

[16] M. Ren, N. Chen, and H. Qiu,“Human-machine collaborative decision-making: Anevolutionary roadmap based on cognitive intelligence,” International Journal of Social Robotics, vol. 15, no. 7, pp. 1101–1114, 2023.

[17] N. I. Susanthi, M. Ali, and A. H. Hernawan, “Digital learning platforms as facilitator for university business collaboration in logistics management curriculum design,” International Journal of Cyber and IT Service Management (IJCITSM), vol. 6, no. 1, pp. 37–50, 2026.

[18] M.A.RahmanandA.Rahman,“Asystematic review of intelligent support systems for strategic decision making using human-ai interaction in enterprise platforms,” Available at SSRN 5269458, 2025.

[19] M. Hardini, M. Siahaan, I. N. Hikam, T. Nurhaeni, and K. Vaher, “Examining the effects of innovation access safety and sustainability practices on global health outcomes,” Journal of Orange Technology, vol. 1, no. 2, pp. 103–114, 2025.

[20] M. F. Hilmi, H. Hamdan, A. Faturahman, B. N. Henry, and J. Wilson, “Ai in industry forecasting: The use of ai in predicting industry trends and demands,” Health, Empathy, and AI Learning (HEAL), vol. 1, no. 1, pp. 78–83, 2025.

[21] G. Kostopoulos, G. Davrazos, and S. Kotsiantis, “Explainable artificial intelligence-based decision support systems: A recent review,” Electronics, vol. 13, no. 14, p. 2842, 2024.

[22] S. Gupta, S. Modgil, S. Bhattacharyya, and I. Bose, “Artificial intelligence for decision support systems in the field of operations research: review and future scope of research,” Annals of Operations Research, vol. 308, no. 1, pp. 215–274, 2022.

[23] R. T. H. Safariningsih, U. Rahardja, M. T. D. S. Putri, N. Azizah, E. Harris et al., “Advanced big data analytics for proactive cyber threat mitigation in large scale computer networks,” Journal of Computer Science and Technology Application, vol. 3, no. 2, pp. 199–212, 2026.

[24] M. Elhaddad and S. Hamam, “Ai-driven clinical decision support systems: an ongoing pursuit of potential,” Cureus, vol. 16, no. 4, p. e57728, 2024.

[25] Q. Xu, W. Xie, B. Liao, C. Hu, L. Qin, Z. Yang, H. Xiong, Y. Lyu, Y. Zhou, and A. Luo, “Interpretability of clinical decision support systems based on artificial intelligence from technological and medical perspective: a systematic review,” Journal of healthcare engineering, vol. 2023, no. 1, p. 9919269, 2023.

[26] L. Meria, M. S. Gunawan, S. Solahudin, U. Rahardja, and A. Patel, “Enhancing digital business students competence through ui/ux design training for digital product development,” ADI Pengabdian Kepada Masyarakat, vol. 6, no. 2, pp. 133–144, 2026.

[27] S. Bayer, H. Gimpel, and M. Markgraf, “The role of domain expertise in trusting and following explainable ai decision support systems,” Journal of Decision Systems, vol. 32, no. 1, pp. 110–138, 2022.

[28] J. Amann, D.Vetter, S. N.Blomberg, H.C.Christensen, M. Coffee, S. Gerke, T. K. Gilbert, T. Hagendorff, S. Holm, M. Livne et al., “To explain or not to explain?—artificial intelligence explainability in clinical decision support systems,” PLOS Digital Health, vol. 1, no. 2, p. e0000016, 2022.

[29] F. Shwedeh and H. M. Alzoubi, “Artificial intelligence (ai) integration into the decision support systems of health care centers,” in International Scientific Conference Management and Engineering. Springer, 2024, pp. 165–176.

[30] A. Hussain and K. Ullah, “An intelligent decision support system for spherical fuzzy sugeno-weber aggregation operators and real-life applications,” Spectrum of mechanical engineering and operational research, vol. 1, no. 1, pp. 177–188, 2024.

[31] S. Alon-Barkat and M. Busuioc, “Human–ai interactions in public sector decision making: “automation bias” and “selective adherence” to algorithmic advice,” Journal of Public Administration Research and Theory, vol. 33, no. 1, pp. 153–169, 2023.

[32] I. J. Akpan, Y. M. Kobara, J. Owolabi, A. A. Akpan, and O. F. Offodile, “Conversational and generative artificial intelligence and human–chatbot interaction in education and research,” International Transactions in Operational Research, vol. 32, no. 3, pp. 1251–1281, 2025.

[33] G. Lorenzini, L. Arbelaez Ossa, D. M. Shaw, and B. S. Elger, “Artificial intelligence and the doctor patient relationship expanding the paradigm of shared decision making,” Bioethics, vol. 37, no. 5, pp. 424–429, 2023.

[34] “Explainable ai in clinical decision support systems: A meta-analysis of methods, applications, and usability challenges,” Healthcare, 2025. [Online]. Available: https://pmc.ncbi.nlm.nih.gov/articles/PMC12427955/