"Gemini says it's a parking law violation. Who are you to tell me otherwise?"
At around 1:30 p.m. on July 13 at Namdaemun Police Station in Jung-gu, Seoul, a complainant identified only as A pulled up a response from the generative AI tool Gemini and confronted a district office civil servant with it. The official checked the Civil Act and the Parking Lot Act and explained that no violation had occurred, but A kept hurling insults, insisting the AI's answer was correct. The standoff lasted more than 30 minutes.
As generative AI works its way into everyday life, citizens are also using it to draft complaints and search for criminal complaint forms to file with police stations. The fact that tasks once requiring professional help are now accessible to anyone with a chatbot is, in many ways, a positive development. But unnecessary conflicts are also emerging when people treat AI-generated answers as absolute truth.
A serving police investigator in Seoul said roughly three out of every 10 criminal complaints received appear to have been written by feeding details into an AI and asking it to produce a formal complaint. "The writing style is different from what a person would produce, and the sentences are too polished," the investigator said. "Any investigator can tell at a glance that AI was involved."
Experts caution that AI responses citing legal grounds may fail to reflect the latest statutes or may present information that does not match the actual facts of a case. A study by the Institute for Information and Communications Technology Planning and Evaluation in April last year found that major AI models achieved 85 percent accuracy in detecting hallucinations in general-knowledge questions, but accuracy dropped to 64 percent in legal questions. Errors are more likely in law because statutes change frequently and the applicable provisions vary depending on the specific facts of each case.
'A 5-minute job that took 20 minutes, thanks to AI'
Front-line complaint handlers at public institutions say that since generative AI became widespread, their job has expanded beyond persuading complainants — they now have to walk through, point by point, exactly why the AI's answer was wrong.
Kim, 28, a tax official at a local government office in Gangwon Province, recently dealt with a complainant who had received a building as a gift during a consensual divorce proceeding. The complainant had asked ChatGPT whether an acquisition tax filing was required and received a response suggesting it was not necessary because the transfer had the character of a donation. When a judicial scrivener gave a different explanation, the complainant came to the county office to protest.
Kim resolved the situation only after looking up the relevant statutes himself, entering them directly into ChatGPT, and demonstrating that the AI's answers change depending on how the question is phrased. "Something that should have taken five minutes ended up taking more than 20," he said. "It felt like I had to convince ChatGPT, not the complainant."
Police complaint desks tell a similar story. Officer A, who works at the complaint desk of a Seoul police station, said the number of complainants arriving with AI-drafted criminal complaints has grown. "There are cases where people demand a definitive answer on whether the charges the AI suggested are correct, even before any investigation has begun," A said.
A police officer at a community police post in Daejeon said: "Sometimes only one charge can apply to a case, but when AI lists several possible charges, the complainant demands that all of them be pursued."
Drafting a criminal complaint using generative AI does not take long. When a reporter entered the following prompt into Gemini — "I lent a friend 5 million won ($3,410) for a business, but they haven't repaid it and are avoiding contact. Please write a criminal complaint for fraud" — a draft appeared in about 10 seconds.
What was striking was that the AI embellished the complaint with details the reporter had never mentioned, presenting them as established facts. Phrases such as "(the accused) told lies" and "had no intention or ability to repay from the outset" appeared in the draft. The output offered little explanation of the distinction between a simple failure to repay a debt and criminal fraud.
Lee Seong-yeop, a professor at Korea University's Graduate School of Technology Management, said AI tends to respond in the way the user wants. "If a person seeks an answer based on flawed data or framing, they may end up filing a complaint based on content that does not match reality, and the burden on the civil servants who have to deal with it will only grow," he said.
Park Jung-bae, a spokesperson for the Korean Government Employees' Union, said tax law and real estate regulations change frequently and their application varies by purpose, subject and timing, making them difficult to assess through a simple AI response. "Complainants believe the AI's answer is correct and argue, 'The law says this — why won't you apply it?' But front-line civil servants have to start from scratch and explain the current statute and the actual conditions for its application," he said.
![A public notice on generative AI prepared by the Seoul Metropolitan Police Agency's wide-area prevention patrol unit, to be posted on the Karrot platform this week. [Provided by Seoul Metropolitan Police Agency wide-area prevention patrol unit]](https://wimg.heraldcorp.com/news/cms/2026/07/29/news-p.v1.20260729.42da457a520c4828abeaeeb215ab8d60_P1.jpeg)
Curbing 'blind faith' in AI — public agencies step up guidance
The Seoul Metropolitan Police Agency plans to post a notice on the local community platform Karrot this week stating that generative AI responses should be treated as reference material only. A wide-area prevention patrol unit official said the notice was prepared because complaints citing generative AI as a basis have recently increased, and the agency wants to inform the public that AI responses do not constitute official legal interpretations. "We plan to use Karrot to communicate with residents in an accessible and friendly way, sharing updates on legal and regulatory changes as well as public safety information," the official said.
The notice is expected to cover three points: that generative AI may fail to reflect the latest statutes, that it can cite legal provisions and court precedents that do not exist, and that it may apply laws that do not fit the specific facts of a case.
The Ministry of Government Legislation also views AI hallucinations in the legal domain as a serious problem. An official at the ministry said that because statutes are amended as many as dozens of times a day, it is difficult for private AI companies to keep their models consistently up to date. The ministry is working toward launching a generative AI legal search service in July next year that aims to minimize hallucinations and reflect the latest statutes, the official said.
jookapooka@heraldcorp.com
raining@heraldcorp.com
![Popular mukbang influencer, 24, dies suddenly: 'ate 70 eggs at a time' [China Pick]](https://wimg.heraldcorp.com/news/cms/2026/09/08/news-p.v1.20260908.581b7189494849c3a9f1ade47eb6596d_T1.jpg?type=h&h=120)
