“What is my position on artificial intelligence (AI)? Am I an AI-supporter? AI-curious? AI-resistant? Even to take on any of these positions, do I understand AI enough? Are we even left with a choice to have a position on AI given the pace at which it is tightening its grip in all sectors including health?” 

I struggle to answer these questions myself. I usually avoid them. That was so far the most comfortable stance that I was enjoying until recently when an invitation came my way for an online discussion by one of the big tech companies which is shaping the present and the future of AI. 

As most of us are already aware, AI companies are building clinical health care models where they are using the existing medical records, available clinical guidelines for various diseases, and the data available from diagnostic tests, etc., to train the machines to learn patterns and by developing large language models (LLMs) that can provide information on medical decision-making. I was superficially aware of this information, as I recently had a strange experience of receiving an AI-generated discharge summary for my father’s hospitilisation from a leading hospital in Maharashtra, which had made me wonder about many ethical questions including health data privacy. As like any other caregiver, I had no time and energy to follow-up on this with the hospital, so with a sigh, I decided to ignore it. 

The topic and the rationale for the planned discussion, however, was something surprising and in a pleasant way. First, it wasn’t about clinical health care but about community-based health care and second, it was put together by a group of techies from that company’s global team who were consciously thinking about concerns around equity, human rights and sociocultural realities of patients and they were interested in discussing how AI can be more responsible in its approach. The team also mentioned that while they had been able to build AI models for clinical health care, they were finding it challenging to develop AI models for community-based care. 

Their team sent me two questions to address during the online discussion: 

Q1: ​“How do patients from various identities or backgrounds, such as socioeconomic status, age, gender, differ in how they define, tolerate, and report symptoms, and do you see a correlation in use of the types of health management strategies chosen across these backgrounds?”

and

Q2: ​“What should we expect health AI models and tools to know when it comes to providing responses that are grounded in cultural and local context?

Though these questions seem a bit loaded, I found it extremely interesting to have these come from a team deeply engaging with tech and AI. 

Their team specifically requested me to discuss these questions by bringing nuances from my work around mental health and sexual-reproductive health with communities. They also had another request- not to present quantitative data from publicly available datasets to make my points, as that is something they can easily check, they said, ​“we want your thoughts from the work you do, especially as both mental health and sexual health are topics of stigma”. 

So, I decided to present two community-based scenarios- both are real cases that I have come across– 

The first case was of a 45-year-old woman from Rajasthan belonging to a socioeconomically marginalised background, suffering from a triple burden of silicosis (a deadly but preventable occupational health hazard due to exposure to silica dust), tuberculosis (an infectious disease) and severe depression. As soon as we reached her house, she ensured to keep distance from us due to internalised stigma. She had high levels of breathlessness. So, she could barely do some household work. Still, she cared for her goats, cooked for her daughter as her two sons had already migrated to work in the mines, getting exposed to the same silica dust that caused her silicosis. Her prognosis for silicosis was poor. Her only question to the community health worker was, ​“मेरी साँस  कब ठीक हो जाएगी?” ​“When will I be able to breathe properly?” to which the worker responded by taking her hand in her hands, nodding with a hopeful smile without really giving any answer verbally. I also spoke about her remotely located house, inability to walk due to breathlessness and lack of money at hand which made her completely dependent on the medications that were given to her by the community health worker, developing a bond of trust between them. 

The second case that I chose was that of a 40-year-old man living in Mumbai suburbs, belonging to a lower middle class religious minority background who was suffering from the symptoms of auditory hallucinations. Initially for 2 years, the family thought that it was because of an evil eye due to some family dispute. They tried doing rituals only realising that they perhaps needed to see a doctor. They went to a hospital where the doctor did not spend even three minutes with him. The man was given some medications and a label which he could not make sense of as it was called ​“schizophrenia”- which he had never heard of before. Medications helped but they did not like the doctor as the doctor could not make sense of the auditory hallucinations (which had a clear cultural connotation). So, they went from one faith healer to another. Spending lakhs of rupees over the last 8 – 10 years, finally to reach a place where they met a psychiatrist working with a local organisation who gave them time, did not dismiss their expressions of mental illness, and provided treatment at an affordable price- with regular friendly reminders for the follow-up. 

While I was presenting these cases, Q1 was displayed on screen, which helped me bring the audience’s attention to the two words that were used in that question– ​‘chosen’ and ​‘use’. The first word is- ​‘chosen’- which necessarily implies that seeking or not seeking health care is a choice even among the most marginalised, and the second word is- ​‘use’ — which implies something being done actively. It is important not to assume that health care seeking is done actively, as a lot of times, patients may not have the privilege to actively go to a doctor as in the case of a lady who would perhaps live with breathless and symptoms of depression, in the absence of a community health worker providing her the medications by visiting her as she did not have the bodily, familial and social privilege to walk to a nearby health centre. I attempted to bring their attention to such bias that humans building such models may hold and how that may shape the way AI reads and assimilates the information provided to it. 

The second case was to stimulate thoughts on — ​‘What counts as evidence in a community setting?’ The person suffering from hallucinations may easily be labelled by an allopathic system of medicine as someone suffering from ​‘schizophrenia’ and someone who is ​‘non-compliant’ as a response to how he discontinued medicines given by the first psychiatrist. The point to note here is about the inability of a system and healthcare professionals who spoke to both his health and cultural needs. Therefore, the ​‘non-compliance’ was more of a reflection of the system rather than the patient’s behaviour. Hallucinations had cultural connotations and meanings which were getting missed by the psychiatrists, and while the faith-healers could provide him the cultural meanings, could not help him recover. 

The next question (Q2) requires deeper discussion on where exactly the AI world is trying to intervene by building models. Who is it for? How are these models being built? What are these models trying to ​‘solve’? More importantly, how is it going to gauge and ensure that the intervention is ​‘responsible’? As I did not get such an opportunity to discuss these questions with the team, it was only fair to stimulate some thoughts around ethical dilemmas that the team can take forward. 

I have heard from an engineer friend of mine working in this sector, how his colleagues building the AI models have been wondering about the dependence and reliance that they see among adolescents for seeking information about sexual health and mental health concerns. He shared how they are figuring out possible referral pathways in such delicate situations. This struck me as I remembered that, a year ago, in one of the student group projects for the course (Health Communication) that I teach at our university, some students mentioned ​‘communicating with chatGPT’ to be one of the trusted communication platforms for mental wellbeing. So, this indeed was a real experience in the Indian context, at least among adolescents and youth, who have access to basic English language skills, mobile phones and internet. 

While the woman suffering from Rajasthan asked, ​“When will I be able to breathe better?”, she did not necessarily seek a chatGPT-type answer. The community health worker’s gesture, thoughtful pause, and nodding (along with the efforts of bringing her the medicines every few weeks) might have done better than a long answer loaded with medical information. Especially given the stigma that the patient had internalised, holding hands itself was perhaps a ​‘medicine’ for her.

Many adolescents type questions in the AI chat window asking, ​‘When will I feel better?’, or ​‘why am I stressed?’ around sexual health and mental health, where AI shows its smartness to collate the available information to present bullet-point answers in a seemingly pleasing yet neutral language. Isn’t it important for the humans building these AI models to appreciate that health is a physical, sensory and a social experience needing much more than a verbal/​textual response that is loaded with information!

Can community-based health AI models suggest the adolescent boy at the right time not to be emotionally dependent on it for seeking answers on sensitive topics? What will it take to train the AI model to be able to gauge such ​‘right time’ in the chat window? Is it possible to acknowledge that building an AI model rooted in community care needs to have an ethical radar (!) and a plural understanding of health before it gets implemented in a community setting? 

As sensitive topics such as sexual reproductive health, mental health are associated with stigma, AI may not be able to understand the ​“silences”, it won’t be able to ​“feel” the internalised stigma, but is it possible to build an ecosystem where AI is, at least able to identify the ​“triggers”, and the possibilities of ​“silences” to create a contextually relevant local referral pathway? Can AI be used in empathetic liaison with local stakeholders from grassroots organisations and the government, collaborating with actual context-experts; rather than AI trying to be the context-expert itself? Perhaps the easiest way to engage with that difficult question (Q2) in a limited time, was by asking such difficult questions (.. also as I cannot pretend to have answers to these questions!) 

The discussion left me with a thought. Our Public Health team in Bengaluru recently had several rounds of discussions to design a certificate programme on ​‘Public Health Fundamentals: Perspectives and Practice’ which we will be offering for health and development practitioners. Did we think of people working with tech and AI to be the participants? Honestly, no! However, the tech team’s questions seemed to be seeking a lens to be able to understand what responsible AI might mean while building models to support community-based health. Public health practitioners, researchers and academicians speak and write about the need of having an equity lens in the technology, but are we engaging enough with those who are building it? We have a few good examples of the use of appropriate technology interventions developed by the grassroots-level organisations. Is it possible to bring together a few case studies from different parts of the world, where community-based AI models are guided by public health values? What capacities do we, as faculty, need to build in ourselves to be able to speak to this tech world, beyond checking student assignments in Turnitin? This experience is pushing me towards becoming an ​‘AI-curious’.  While my position is shifting, I also ensured to share the details of the ​‘Public Health Fundamentals’ certificate programme with this well-intentioned tech-team, as the position needs to shift from both the ends!

Mukta Gundi is a faculty at Azim Premji University, Bengaluru. She is trained in Public Health. Her work focuses on adolescent health and community-based mental health approaches. She teaches courses on Research Methods, Health Communication and Mental Health. 

Acknowledgements: The author is thankful to Sagar A. and Ketan G. for their helpful inputs while thinking through this topic. Thanks to Adithya Pradyumna and Arima Mishra for reviewing the draft of this article. .