'Generative AI is an ambivalent technology — it must be bounded as an aid, not a replacement'
"AI conducts the initial consultation, and the doctor makes a diagnosis based on that?" The use of AI in psychiatric care settings has been growing. KAIST and Gangnam Severance Hospital recently announced the development of an "AI interviewer system." Unlike other medical specialties that rely on objective data such as blood tests or MRI scans, psychiatry depends on patients directly sharing their symptoms and lifestyle context with their doctors. Yet it is difficult to fully assess a patient's condition during a brief first appointment, and patients themselves often need time before they can open up. Preliminary AI interviews have emerged as a form of supplementary care designed to overcome these limitations.
As generative AI moves beyond that model and begins to be used more broadly in mental health care, a research team has published findings analyzing the technology's potential and risks — and the priorities needed for its safe adoption — based on the real clinical experiences of South Korean psychiatrists.
A joint research team led by Prof. Cho Chul-hyun of the Department of Psychiatry at Korea University Anam Hospital and Prof. Chung Doo-young of the School of Digital Humanities and Computational Social Sciences at KAIST published the study in a top international journal in the field of digital health. The paper examines the experiences and interpretations of Korean psychiatrists regarding generative AI in clinical practice, and identifies the tasks required for its safe integration into mental health care.
The study was conducted as part of a project by the Future Strategy Committee of the Korean Neuropsychiatric Association. Kim Myung-sung, a doctoral candidate at the UNIST Graduate School of Medicine, and Prof. Ahn Yu-seok of Seoul National University College of Medicine and the National Traffic Injury Rehabilitation Hospital served as co-first authors. Prof. Cho and Prof. Chung served as co-corresponding authors.
The research team surveyed members of the Korean Neuropsychiatric Association from Oct. 27 to Dec. 26, 2025. A total of 408 respondents participated, including 326 board-certified psychiatrists and 82 residents. Of these, the 311 who provided substantive open-ended responses were included in the qualitative analysis.
The survey comprised three questions: clinical experiences with generative AI tools such as chatbots and diagnostic aids; the advantages and limitations of generative AI compared with human therapists; and the most urgent priorities for safely introducing generative AI into mental health care.
The team applied the concept of "horizon scanning" — a futures-forecasting method — to qualitative research, analyzing physicians' responses through a three-stage framework: field signals (experience), interpretation, and implementation priorities. A methodological hallmark of the study was its use of clinical signals emerging from everyday practice, rather than external sources such as papers or patents, as the starting point for anticipating future developments.
The study's central finding is that generative AI is a clinically ambivalent technology. The same function can be either helpful or harmful depending on the context of use, the intensity of use, and the vulnerability of the patient.
Physicians reported positive cases in which patients used generative AI as a low-threshold tool for emotional processing, self-management, and entering treatment. One doctor described a patient who used generative AI not merely to generate comforting phrases but to organize their own situation and emotions — articulating feelings that had previously been difficult to express and experiencing an improvement in symptoms as a result.
When use became excessive or occurred in high-risk situations, however, adverse effects were pronounced. Cases were reported involving the reinforcement of delusional beliefs, social withdrawal, overdependence, and links to suicide and self-harm risk, including medication overdose. Physicians also observed a new pattern in which patients compared generative AI outputs with their doctor's diagnosis, undermining the therapeutic relationship itself, treatment adherence, and trust in the diagnosis.
Respondents viewed generative AI as a standardized tool unaffected by fatigue, while also finding it "relationally thin" in areas requiring deep therapeutic relationships. They noted its limited ability to reflect nonverbal cues and emotional nuance, and warned that its tendency to accept users' expressions repeatedly without critique or verification could reassure patients while also carrying the risk of excessively validating distorted beliefs. Physicians broadly agreed that generative AI is useful as a supplement to human therapists but cannot be accepted as a replacement.
On implementation priorities, physicians emphasized not the expansion of generative AI but the preconditions for its safe adoption. Four key priorities emerged: governance and accountability; safety infrastructure for crisis situations and vulnerable populations; technical reliability and clinical validation before wider rollout; and education, supervision, and structural support.
The research team explained these findings through two concepts. The first is "vulnerability amplification" — the way generative AI's output, which may function like emotional support in low-risk situations, can instead worsen the condition of patients whose grip on reality is unstable or who are in severe psychological distress. The second is "access-protection tension." Generative AI can lower the threshold for mental health care by being available anonymously and instantly, even before a hospital visit — but high accessibility alone does not guarantee safe care.
In a social environment where digital engagement is high yet people may hesitate to seek psychiatric care due to concerns about medical records, social stigma, or potential disadvantages, the anonymity and immediacy of AI use can be attractive while simultaneously heightening the risks of unsupervised use.
The research team noted that physicians' attitudes reflected not simple rejection of generative AI but "conditional acceptance." Doctors acknowledged the technology's usefulness for administrative tasks, information organization, patient education, and pre- and post-consultation support. However, they drew a clear line against replacing human therapists in areas involving direct patient treatment, crisis assessment, and the formation of therapeutic relationships.
The team suggested that psychiatrists may increasingly be called upon to go beyond simply providing treatment — interpreting patients' generative AI usage, guiding appropriate boundaries of use, and supervising for risk.
"This study is significant in that it goes beyond surveying abstract attitudes to identify, based on real clinical experience, how psychiatrists are actually encountering generative AI in the consulting room," Prof. Cho said. "It shows that AI in mental health requires more layered oversight and diagnosis-sensitive safeguards than AI in other areas of medicine."
"The central question is not whether psychiatry will encounter generative AI, but how its use will be bounded, supervised, and governed — taking into account patient vulnerability, relational needs, and the safety risks specific to psychiatry," Prof. Chung said. "Limited supplementary use and strengthened governance and validation must come before rapid replacement."
The study was supported by the Future Strategy Committee of the Korean Neuropsychiatric Association and the National Research Foundation of Korea. The paper, titled "Mapping Practice-Based Signals of Generative AI in Psychiatric Care: Qualitative Study of Korean Psychiatrists' Experiences, Interpretations, and Implementation Priorities," was published in the Journal of Medical Internet Research on June 2.
kty@heraldcorp.com
