A new multimodal smart speaker system designed to help users track their psychological state (Image courtesy of KAIST)
DAEJEON, Sept. 25 (Korea Bizwire) – The Korea Advanced Institute of Science and Technology (KAIST) has unveiled a new multimodal smart speaker system designed to help users track their psychological state, the institute announced on September 24.
This innovation comes at a time when concerns about mental health issues among single-person households are growing due to rapid demographic changes in South Korea.
As the country grapples with an aging population and declining birth rates, the number of people living alone has surged, bringing mental health challenges such as social isolation and depression to the forefront.
While smart speakers have been employed to diagnose these issues, previous random survey methods risked inducing stress or irritation in users, potentially skewing responses.
To address this, a research team led by Lee Uichin from KAIST’s School of Computing has developed a smart speaker equipped with multimodal sensors.
This advanced system can detect changes in the user’s environment to identify optimal moments for initiating mental health self-tracking surveys.
“By analyzing various sensor data including indoor movement, lighting, noise, and carbon dioxide levels, we can pinpoint the best times to pose questions,” Lee explained.
“This approach has achieved higher response rates compared to conventional random surveys.”
A research team led by Lee Uichin from KAIST’s School of Computing (Image courtesy of KAIST)
The system operates by recognizing activity transitions, such as when a user turns off a video or switches on a light, and by measuring carbon dioxide levels to determine if the user is alone. This contextual awareness allows for more timely and less intrusive interactions.
To validate their approach, the team installed self-tracking smart speakers in 20 single-person households for a month-long field study.
The experiment yielded a dataset of 2,201 mental health survey responses, providing insights into response times, patterns based on activity context, and preferences for voice or touch input in various situations.
Interestingly, users predominantly preferred touch interfaces over voice input, citing familiarity with interaction and concerns about low recognition rates for voice commands.
“We plan to evolve this smart speaker into a mental health management support system that functions similarly to a human counselor,” Lee stated.
He also revealed ongoing research to predict lifestyle patterns based on users’ mental health states by applying AI models to daily life data collected indoors.
This research, supported by the LG Electronics-KAIST Digital Healthcare Research Center, represents a significant step forward in leveraging technology for mental health support.
Kevin Lee (kevinlee@koreabizwire.com)
