[Edaily Reporter Kim Hyun-ah] #KT has set out to establish an artificial intelligence (AI) safety evaluation system that reflects the social and cultural context of South Korea. The company is accelerating the race to ensure AI reliability by unveiling the country’s largest multimodal benchmark, which can verify how safely global AI models operate in a Korean-language environment.

KT announced on the 16th that it has unveiled “KSAFE-MM,” a safety benchmark for multimodal large language models (MLLMs) developed in collaboration with Korea University.

KSAFE-MM is designed to evaluate the safety of multimodal AI models—which process various forms of data such as text, images, and audio simultaneously—by incorporating the cultural characteristics and social issues of Korean society.

The benchmark consists of ‘KSAFE-MM-G,’ which adapts global common risk factors to the Korean cultural context, and ‘KSAFE-MM-C,’ which reflects specific domestic issues such as lease fraud and the Dokdo territorial dispute. The entire evaluation dataset comprises 14,135 samples, making it the largest Korean-language multimodal safety evaluation dataset among those publicly available in Korea.

Using this dataset, researchers from KT and Korea University verified the safety of 12 major global and domestic multimodal AI models, including Google’s Gemma and Naver’s HyperCLOVA X.

The key innovation of this research is the automation of the benchmark construction process. Existing AI safety evaluation datasets required manual review by experts, resulting in high costs and significant time investment.

In contrast, KSAFE-MM applied a four-step pipeline that automates the entire process—from collecting sensitive issues based on local communities, to generating template-based queries, creating synthetic images, and even generating so-called “jailbreak prompts” designed to bypass AI safety mechanisms.

KT explained that this has laid the groundwork for rapidly building safety benchmarks that reflect local characteristics, even without experts from specific countries or cultural regions. In fact, the research team verified the system’s global scalability through a pilot project called “JSAFE-MM-C,” which applied the same method to a Japanese-language environment.

As generative AI evolves into a multimodal era capable of processing images, videos, and audio, the importance of safety evaluation systems capable of assessing risks such as cultural bias, misinformation, and the generation of harmful content is growing.

Accordingly, KT expects this benchmark to be utilized in various fields, including safety verification prior to the launch of AI services, as well as red team testing and guardrail model evaluation. The research results and dataset have been made public through the paper-sharing platform arXiv and the AI open-source platform Hugging Face.

Following the recent release of “XL-SafetyBench,” a multilingual AI safety evaluation framework, KT is further strengthening its AI safety research by introducing KSAFE-MM. Led by its dedicated Responsible AI (RAI) organization, the company is spearheading the design of risk classification systems and the development of evaluation metrics.

Park Jae-hyung, Head of the Frontier AI Lab at KT’s AX Future Technology Institute (Senior Vice President), stated, “The release of this AI safety benchmark goes beyond simply providing data; it lays the groundwork for the advancement of the entire research ecosystem.” He added, “We expect KSAFE-MM to establish itself as the leading standard for evaluating AI safety within the Korean language and cultural context.”