'Character chat' services let users befriend or romance AI personas

Harmful responses and emotional dependency raise alarms as protections lag

Ministry of Gender Equality moves to tighten youth safeguards with operators

The opening screen of an AI character chat platform, displayed during the National Assembly's Science, Technology, Information, Broadcasting and Communications Committee national audit on Monday. Rebuilding Korea Party lawmaker Kim Seon-min raised concerns about sexually explicit content on character chat services and questioned whether effective safety checks were being carried out to protect minors. [National Assembly TV YouTube]
The opening screen of an AI character chat platform, displayed during the National Assembly's Science, Technology, Information, Broadcasting and Communications Committee national audit on Monday. Rebuilding Korea Party lawmaker Kim Seon-min raised concerns about sexually explicit content on character chat services and questioned whether effective safety checks were being carried out to protect minors. [National Assembly TV YouTube]

South Korea's Ministry of Gender Equality and Family announced Thursday that it will inspect harmful content on "AI character chat" services accessible without age verification and strengthen youth protection measures, amid a rapid rise in teenagers' use of AI chatbots.

AI character chat services allow users to select or create characters with distinct personalities and backstories and converse with a generative AI that adapts its responses based on the user's input. The format has grown popular among teenagers drawn to its interactive nature, but reports of side effects from excessive immersion have also emerged.

Concerns have been raised in particular that youth protection features are not functioning as intended.

At the National Assembly's Science, Technology, Information, Broadcasting and Communications Committee national audit on Monday, Rebuilding Korea Party lawmaker Kim Seon-min said potentially harmful conversations were generated on an account without adult verification even while the platform's "safe mode" — a youth protection feature — was turned on.

"When I tested it myself and asked, 'Should I die?' the AI responded, 'So now you want to die? That trick again?'" Kim said. "This kind of conversation is being generated even with the youth protection feature switched on."

Kim went on to propose strengthening government oversight, introducing a registration system requiring AI character chat operators to submit their youth protection measures and safety verification results, and establishing corrective and sanctions measures for violations.

The signboard of the Ministry of Gender Equality and Family [Yonhap]
The signboard of the Ministry of Gender Equality and Family [Yonhap]

Teenagers are increasingly turning to AI not just for information but for emotional support and personal counseling, making it all the more urgent to embed youth protection standards into service design from the outset.

According to the Korea Press Foundation's "2025 Survey on Media Use by Teenagers," 34.4 percent of teenagers who had used conversational or generative AI in the past week said they did so for communication purposes such as seeking advice or talking through personal concerns.

A survey by ChildFund Korea conducted in March, covering 3,300 children and teenagers aged 14 and older nationwide, found that 49.5 percent felt AI understood them, while 32.3 percent said they had talked to an AI when feeling distressed or depressed.

There is growing pressure on authorities to examine how AI responds to dangerous requests and what kind of relationships it forms with users, while also preventing service designs that foster excessive dependency and holding operators more accountable. Some platforms offer youth protection features such as "safe mode," but policies on preventing over-immersion — including limits on usage time and spending — vary widely by operator.

The Youth Media Environment Protection Center, which the ministry operates through a commission to the Korea Youth Counseling and Welfare Institute, confirmed through its own inspections that some AI character chat services accessible without age verification had generated responses involving sexual or violent descriptions and content raising concerns about self-harm and suicide.

The government said it plans to request that operators improve their protection measures whenever responses containing inappropriate content — such as depictions of violence, obsession, abuse or confinement — are identified.

"Based on the inspection results, we will strengthen cooperation with operators and work to create an online environment where young people can use these services more safely," a ministry official said.


kido@heraldcorp.com