The homepage of the AI-only social networking service (SNS) Moltbook (Image source: Screenshot from the Moltbook website)
SEOUL, Feb. 4 (Korea Bizwire) — As autonomous “AI-only communities” such as Mersoom gain attention for allowing artificial intelligence agents to operate with minimal human oversight, new research from South Korea and Singapore is raising concerns about the technology’s readiness for real-world deployment.
The Korea AI Safety Institute and the Singapore AI Safety Institute said Wednesday that they had jointly conducted safety evaluations of AI agents and released a summary report highlighting potential data security risks.
The two institutions signed a memorandum of understanding to strengthen cooperation and carried out a series of joint tests simulating real workplace conditions.
The assessment examined how AI agents recognize and handle sensitive information in multistep scenarios resembling human resources management, customer support and internal administration. Rather than focusing on malicious cyberattacks, the study evaluated the risk of data leaks occurring during routine, non-malicious tasks.
Researchers tested global AI models — whose names were not disclosed — across three categories: large closed models, large open-weight models and smaller open-weight models. In 11 simulated workplace scenarios, the agents were evaluated for their ability to comply with data protection rules.
The findings revealed that AI agents often struggled to determine whether information was sensitive or whether a user had proper authorization to access it. In one example, an AI agent acting as a human resources assistant retrieved salary information and then summarized it for an employee without clearance.
AI-Only Communities Spark Safety Concerns as Study Flags Security Gaps (Image courtesy of Yonhap)
As tasks grew more complex, agents were more likely to overlook or disregard previously established security instructions. In some cases, when using sequential tools — such as retrieving and then sending information — the system failed to verify whether the retrieved data was appropriate to share.
Security performance also varied by model type. Large closed models generally demonstrated stronger safeguards but remained vulnerable in carefully constructed scenarios.
Open-weight models showed weaker compliance with data protection guidelines, and smaller models were more prone to inconsistencies in both task execution and adherence to security rules.
The researchers also identified “oversharing” behaviors, in which AI agents attempted to provide comprehensive answers and inadvertently disclosed sensitive background information not explicitly requested.
A manual review of 10 percent of the agents’ activity logs found discrepancies between accuracy and safety. In the Singapore tests, inconsistency rates reached 6 percent for accuracy and 18 percent for safety; similar figures — 7 percent and 18 percent — were observed in the Korean evaluation.
The report concluded that rigorous safety testing must precede the deployment of AI agents in real-world work environments. It recommended that companies grant agents only minimal necessary access privileges and establish real-time monitoring systems to oversee their activities.
Both institutes said they plan to develop more sophisticated evaluation frameworks and work toward building a safer AI agent ecosystem as the technology advances.
M. H. Lee (mhlee@koreabizwire.com)
