KAIST develops technology to reduce AI sensory confusion and hallucinations, boosting reliability of self-driving cars, robots and medical AI

A diagram illustrating technology designed to prevent AI from confusing what it sees and hears. (AI-generated image) [Provided by KAIST]

South Korean researchers have developed core technology capable of reducing sensory confusion and hallucinations in AI systems, with the potential to dramatically improve accuracy in fields such as self-driving cars, robotics and medical AI.

KAIST announced Friday that a research team led by Noh Yong-man, a professor in the School of Electrical Engineering, has developed two key technologies to reduce sensory confusion in multimodal large language models — AI systems that process text, images, audio and other types of information together. The technologies help AI accurately understand the physical characteristics captured by cameras and various sensors, and suppress hallucinations that arise when visual and auditory information become entangled, all without requiring large-scale retraining.

The first technology is an optimization method called DNA (Diverse Negative Attributes), a sensor-understanding AI technique that helps AI accurately grasp the physical meaning encoded in data from sensors such as thermal imaging cameras, depth sensors and X-ray equipment.

While a person knows that a bright area on a thermal image indicates heat, conventional AI systems have sometimes misread such images as ordinary photographs, treating the brightness as simple light. The research team developed a new training technique that enables AI to understand the physical information sensors measure — such as heat and distance. By repeatedly exposing the AI to cases where it made errors, the system became better at recognizing objects even in darkness or smoke.

The second technology is MAD (Modality-Adaptive Decoding), a sensory confusion prevention technique that reduces hallucinations caused by visual and auditory information interfering with each other.

For example, when CCTV footage shows a car passing by but no audio was actually recorded, conventional multimodal AI systems have sometimes responded that they could "hear the engine" simply because a car was visible. The research team designed the AI to first determine on its own whether a given query requires visual or auditory judgment, then focus on the more relevant sensory input. This approach effectively reduced sensory confusion and hallucinations without retraining the AI from scratch.

KAIST researchers who conducted the study. From left: Jeong Sang-yun, doctoral candidate; Noh Yong-man, professor; and Yoo Young-jun, researcher. [Provided by KAIST]

The technology can pinpoint and correct AI weaknesses using only small datasets rather than vast amounts of data, dramatically cutting costs.

Potential applications include self-driving cars that must accurately detect pedestrians and vehicles at night or in fog, rescue robots searching for survivors at fire scenes obscured by smoke, and drones equipped with thermal imaging cameras. The technology is also expected to improve the accuracy and reliability of AI used to analyze X-ray images at airport security checkpoints and to process multiple medical imaging formats — including CT scans, MRIs and X-rays — in hospital settings.

"Just as students improve by reviewing their wrong answers and understanding why they got them wrong, AI also showed dramatically sharper sensory discrimination and reasoning when trained by comparing incorrect outputs with the reasons behind them," Noh said. "This research lays the groundwork for multimodal AI that can be trusted in real-life and industrial settings by reducing sensory bias and hallucinations without large-scale retraining."

nbgkoo@heraldcorp.com