Deepfake

Deepfake refers to a technology that uses deep learning to realistically manipulate and synthesize a person's face, voice, and video, and is regarded as a cybersecurity threat in the form of phishing scams, impersonation attacks, and similar exploits.

Technical Mechanisms

At the core of deepfakes are deep learning architectures such as GANs (Generative Adversarial Networks) and autoencoders. GANs pit two models against each other — a "generator" and a "discriminator" — to produce fabricated video footage so convincing it is indistinguishable from reality. Beyond face swapping, the technology enables voice synthesis synchronized with lip movements and control over facial expressions and gaze. In recent years, the rapid advancement of Generative AI has created an environment in which even general users can produce high-quality content at low cost.

In the audio domain, a technique known as "voice cloning" has become widespread, making it technically possible to reproduce a specific individual's voice from just a few dozen seconds of audio samples. By combining video and audio, it has become possible to create footage that makes real executives or politicians appear to say things they never said, and actual fraud cases have been reported.

Impact on Cybersecurity

The reason deepfakes are considered particularly dangerous is that they fundamentally undermine conventional authentication and trust models.

• Sophistication of Business Email Compromise (BEC): Cases have been confirmed in which video calls impersonating the faces and voices of senior executives are used to instruct staff to make fraudulent transfers.
• Impersonation attacks: Exploited as a means of bypassing systems that use facial recognition for identity verification.
• Increased precision of phishing scams: Video messages disguised as trusted individuals are used to lure targets into downloading malware or entering credentials.
• Manipulation of public opinion and disinformation: Misuse in political contexts damages societal trust itself.

As the concept of Zero Trust Network Access (ZTNA) suggests, the principle of "never trust what you see" is becoming increasingly important. From an AI governance perspective, deepfakes are also positioned under the EU AI Act as subject to transparency obligations, with regulatory requirements such as mandatory disclosure and labeling of deepfake content now being introduced.

Current State of Detection Technology and Countermeasures

As a countermeasure against deepfakes, research into forensic detection models is advancing. The mainstream approach involves using machine learning to detect subtle artifacts such as blinking patterns, skin texture, and unnatural light reflections. However, generation technology and detection technology are engaged in a constant arms race, with the risk that improvements in generation quality will perpetually outpace detection capabilities.

The following organizational countermeasures are considered effective:

• Multi-factor authentication workflows, such as callback verification, for critical decisions (e.g., fund transfers, information disclosure)
• Advance validation of impersonation attack scenarios through AI Red Teaming
• Cultivating a culture of skepticism through AI Literacy training for employees
• Embedding digital watermarks and provenance information into video and audio content

Incorporating the concept of HITL (Human-in-the-Loop) into organizational processes — designing systems in which humans exercise judgment over AI-generated content — is also a practical approach to minimizing harm.

Future Outlook

As the quality of Video Generation AI continues to improve, the deepfake threat is expected to expand further. At the same time, international standardization of content authentication (such as C2PA) and stricter terms of use by generative AI providers are also progressing, and we are entering a phase in which countermeasures must be pursued across three layers: technology, regulation, and literacy.