Designing Inclusive Companion Robots to Mitigate Bias and Enhance Empathy in AI-Driven Care Systems

Karnawati Karnawati; Dwi Nur Ramadhan; Tiurida Lily Anita; Risma Nurmala; Lily Maria

Authors

Karnawati Karnawati State University of Semarang https://orcid.org/0000-0001-9744-4617
Dwi Nur Ramadhan Pandawan Group https://orcid.org/0009-0004-2941-2364
Tiurida Lily Anita Bina Nusantara University https://orcid.org/0000-0003-0167-1204
Risma Nurmala Bank BRI
Lily Maria Pandawan Incorporation https://orcid.org/0009-0005-9759-710X

Keywords:

Ethical Artificial Intelligence, Inclusive Design, AI Care Agents, Humanistic Well-Being, Disabilities

Abstract

This study adopts a qualitative research approach using a semi-structured interview design. Data were collected from individuals with diverse physical, sensory, and cognitive disabilities who have prior experience interacting with AI-based care technologies. The data were analyzed using thematic analysis to identify recurring patterns and key experiential dimensions related to ethical perception, inclusivity, trust, emotional support, and well-being. The findings reveal that ethical AI design, particularly in terms of transparency, fairness, and privacy, plays a crucial role in fostering user trust and a sense of security. Inclusive design features, such as accessibility, adaptability, and personalization, were found to enhance user comfort, independence, and emotional engagement. Furthermore, emotional support emerged as a central theme, indicating that users perceive AI care agents not only as assistive tools but also as sources of psychological reassurance and companionship. This study contributes to the growing body of human-centered AI research by providing qualitative insights into how ethical and inclusive design principles shape meaningful user experiences and well-being outcomes among people with disabilities. The findings highlight the importance of integrating humanistic values into AI care development to promote equitable, responsible, and sustainable digital care solutions.

References

[1] D. B. Olawade, O. A. Bolarinwa, Y. A. Adebisi, and S. Shongwe, “The role of artificial intelligence in enhancing healthcare for people with disabilities,” Social Science & Medicine, vol. 364, p. 117560, 2025.

[2] M. F. Almufareh, S. Kausar, M. Humayun, and S. Tehsin, “A conceptual model for inclusive technology: advancing disability inclusion through artificial intelligence,” Journal of Disability Research, vol. 3, no. 1, p. 20230060, 2024.

[3] C. El Morr, D. Singh, V. Sawhney, S. Fernandes, Y. El-Lahib, and R. Gorman, “Exploring the intersection of ai and inclusive design for people with disabilities,” Digital Health and Informatics Innovations for Sustainable Health Care Systems, pp. 556–559, 2024.

[4] C. Lukita, M. Hardini, S. Pranata, D. Julianingsih, and N. P. L. Santoso, “Transformation of entrepreneurship and digital technology students in the era of revolution 4.0,” Aptisi Transactions on Technopreneurship (ATT), vol. 5, no. 3, pp. 291–304, 2023.

[5] E. Umucu, “Artificial intelligence and health equity for people with disabilities: An integrated framework for disability-inclusive ai design,” INQUIRY: The Journal of Health Care Organization, Provision, and Financing, vol. 62, p. 00469580251365472, 2025.

[6] B. Irfan, S. Kuoppam¨aki, and G. Skantze, “Recommendations for designing conversational companion robots with older adults through foundation models,” Frontiers in Robotics and AI, vol. 11, p. 1363713, 2024.

[7] J. J. White, “Artificial intelligence and people with disabilities: a reflection on human–ai partnerships,” in Humanity Driven AI: Productivity, Well-being, Sustainability and Partnership. Springer, 2021, pp. 279–310.

[8] M. Kchaou, Y. Munusamy, K. A. Alharthi, and A. F. Al-Mahmodi, “Industry 5.0 adaptation for disability-inclusive healthcare: A review of emergent and ai technologies for assistive digital health,” Digital Health,vol. 11, p. 20552076251395558, 2025.

[9] S. Sukardi, S. Wahyuni, and R. Rachmawati, “Entrepreneurship capability by triple series innovations in building competitive resilience within the airline industry,” Aptisi Transactions on Technopreneurship (ATT), vol. 7, no. 3, pp. 957–972, 2025.

[10] S. Ferebee, “Ai and accessibility: Breaking barriers for people with disabilities,” Premier Journal of Artificial Intelligence, vol. 2, p. 100012, 2025.

[11] J. Bricout, J. Greer, N. Fields, L. Xu, P. Tamplain, K. Doelling, and B. Sharma, “The “humane in the loop”: Inclusive research design and policy approaches to foster capacity building assistive technologies in the covid-19 era,” Assistive Technology, vol. 34, no. 6, pp. 644–652, 2022.

[12] . H. Lee, F. Y. Hsu, and A. S.-Y. Lien, “Health care professionals’ perspectives of socially assistive robots in health care settings: systematic review,” Journal of Medical Internet Research, vol. 27, p. e79634, 2025.

[13] . Ramadhan, Q. Aini, S. Santoso, A. Badrianto, and R. Supriati, “Analysis of the potential context of blockchain on the usability of gamification with game-based learning,” International Journal of Cyber and IT Service Management (IJCITSM), vol. 1, no. 1, pp. 84–100, 2021.

[14] G. Ricchezze, L. Tomassini, and M. Bacci, “Ethical reflections on integrating artificial intelligence in care practices,” Community Health Equity Research & Policy, p. 2752535X251370928, 2025.

[15] R. Cersosimo and V. Pennazio, “Learning to design, designing to care: High school students co-designing ai for disability inclusion,” Journal of Inclusive Methodology and Technology in Learning and Teaching, vol. 5, no. 3, 2025.

[16] J. Shuford, “Contribution of artificial intelligence in improving accessibility for individuals with disabilities,” Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (Online), vol. 2, no. 2, pp. 421–433, 2023.

[17] D. Newman-Griffis, J. S. Rauchberg, R. Alharbi, L. Hickman, and H. Hochheiser, “Definition drives design: Disability models and mechanisms of bias in ai technologies,” arXiv preprint arXiv:2206.08287, 2022.

[18] A. Lillywhite and G. Wolbring, “Coverage of ethics within the artificial intelligence and machine learning academic literature: The case of disabled people,” Assistive Technology, 2021.

[19] U. Rahardja, L. Sulistyo, D. Safarina, M. R. Kusuma, N. Silawati, and Z. Nanle, “Hibahqu education monitoring platform based on human-centric orange technology laravel 12 vue. js,” ADI Bisnis Digital Interdisiplin Jurnal, vol. 6, no. 2, pp. 203–218, 2025.

[20] G. C. K. Yew, “Trust in and ethical design of carebots: the case for ethics of care,” International Journal of Social Robotics, vol. 13, no. 4, pp. 629–645, 2021.

[21] T. Zidaru, E. M. Morrow, and R. Stockley, “Ensuring patient and public involvement in the transition to ai-assisted mental health care: A systematic scoping review and agenda for design justice,” Health Expectations, vol. 24, no. 4, pp. 1072–1124, 2021.

[22] C. Lukita, A. W. A. Rahman, I. N. Hikam, and U. Rahardja, “Integrating strategic management with sdg 10 for sustainable development and equity,” Aptisi Transactions on Technopreneurship (ATT), vol. 7, no. 2, pp. 638–649, 2025.

[23] V.-A. Melo-L´opez, A. Basantes-Andrade, C.-B. Gudi˜no-Mej´ıa, and E. Hern´andez-Mart´ınez, “The impact of artificial intelligence on inclusive education: A systematic review,” Education Sciences, vol. 15, no. 5, p. 539, 2025.

[24] G. Pisoni, N. D´ıaz-Rodr´ıguez, H. Gijlers, and L. Tonolli, “Human-centered artificial intelligence for designing accessible cultural heritage,” Applied Sciences, vol. 11, no. 2, p. 870, 2021.

[25] S. Ha, S. H. Ho, Y.-H. Bae, M. Lee, J. H. Kim, J. H. Kim, and J. Lee, “Digital health equity and tailored health care service for people with disability: user-centered design and usability study,” Journal of medical Internet research, vol. 25, p. e50029, 2023.

[26] S. Umbrello, M. Capasso, M. Balistreri, A. Pirni, and F. Merenda, “Value sensitive design to achieve the un sdgs with ai: A case of elderly care robots,” Minds and Machines, vol. 31, no. 3, pp. 395–419, 2021.

[27] T. Nadarzynski, N. Knights, D. Husbands, C. A. Graham, C. D. Llewellyn, T. Buchanan, I. Montgomery, and D. Ridge, “Achieving health equity through conversational ai: A roadmap for design and implementation of inclusive chatbots in healthcare,” PLOS Digital Health, vol. 3, no. 5, p. e0000492, 2024.

[28] S. Pancholi, J. P. Wachs, and B. S. Duerstock, “Use of artificial intelligence techniques to assist individuals with physical disabilities,” Annual Review of Biomedical Engineering, vol. 26, 2024.

[29] A. Chalkiadakis, A. Seremetaki, A. Kanellou, M. Kallishi, A. Morfopoulou, M. Moraitaki, and S. Mastrokoukou, “Impact of artificial intelligence and virtual reality on educational inclusion: A systematic review of technologies supporting students with disabilities,” Education Sciences, vol. 14, no. 11, p.1223, 2024.

[30] H. Habib, S. A. K. Jelani, and S. Najla, “Revolutionizing inclusion: Ai in adaptive learning for students with disabilities,” Multidisciplinary Science Journal, vol. 1, no. 01, pp. 1–11, 2022.

[31] N. Tilmes, “Disability, fairness, and algorithmic bias in ai recruitment,” Ethics and Information Technology, vol. 24, no. 2, p. 21, 2022.

[32] N. Lutfiani, D. A. Astrieta, V. Wildan, H. Sulistyaningrum, M. R. Anwar, and E. D. Astuti, “Emotional well-being and psychological support in infertility a multi-modal ai approach,” International Journal of Cyber and IT Service Management (IJCITSM), vol. 5, no. 1, pp. 81–92, 2025.

[33] K. Joamets and A. Chochia, “Access to artificial intelligence for persons with disabilities: Legal and ethical questions concerning the application of trustworthy ai.” Acta Baltica Historiae et Philosophiae Scientiarum, vol. 9, no. 1, 2021.

[34] M. Schaur and K. Matausch-Mahr, “Assistive technology using artificial intelligence in the long-term care sector for persons with disabilities: A systematic literature review,” Technology and Disability, vol. 37, no. 3, pp. 259–269, 2025.

[35] C. El Morr, B. Kundi, F. Mobeen, S. Taleghani, Y. El-Lahib, and R. Gorman, “Ai and disability: A systematic scoping review,” Health Informatics Journal, vol. 30, no. 3, p. 14604582241285743, 2024.

[36] W. Usino, M. M. Sari, F. P. Oganda, O. P. M. Daeli, and E. Smith, “Artificial intelligence integration for sustainable business model innovation insights from global startups,” Sundara Advanced Research on Artificial Intelligence, vol. 1, no. 2, pp. 82–89, 2025.

[37] R. Nacheva and M. Czaplewski, “Artificial intelligence in helping people with disabilities: Opportunities and challenges,” HR and Technologies, vol. 1, pp. 102–124, 2024.

[38] R. Binns and R. Kirkham, “How could equality and data protection law shape ai fairness for people with disabilities?” ACM Transactions on Accessible Computing (TACCESS), vol. 14, no. 3, pp. 1–32, 2021.

[39] A. Iannone and D. Giansanti, “Breaking barriers—the intersection of ai and assistive technology in autism care: A narrative review,” Journal of Personalized Medicine, vol. 14, no. 1, p. 41, 2023.

[40] J. Sin, R. L. Franz, C. Munteanu, and B. Barbosa Neves, “Digital design marginalization: New perspectives on designing inclusive interfaces,” in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021, pp. 1–11.

[41] G. Rubeis, M. L. Fang, and A. Sixsmith, “Equity in agetech for ageing well in technology-driven places: The role of social determinants in designing ai-based assistive technologies,” Science and Engineering Ethics, vol. 28, no. 6, p. 49, 2022.

[42] S. Ahmed, M. S. Rahman, M. S. Kaiser, and A. S. Hosen, “Advancing personalized and inclusive education for students with disability through artificial intelligence: perspectives, challenges, and opportuni- ties,” Digital, vol. 5, no. 2, p. 11, 2025.

[43] R. Fitas, “Inclusive education with ai: Supporting special needs and tackling language barriers,” AI and Ethics, vol. 5, no. 6, pp. 5729–5757, 2025.

[44] J. Donia and J. A. Shaw, “Co-design and ethical artificial intelligence for health: An agenda for critical research and practice,” Big Data & Society, vol. 8, no. 2, p. 20539517211065248, 2021.

[45] E. Grassini, M. Buzzi, B. Leporini, and A. Vozna, “A systematic review of chatbots in inclusive healthcare: insights from the last 5 years,” Universal Access in the Information Society, vol. 24, no. 1, pp. 195–203, 2025.