Deepfakes & Synthetic Media

Synthetic media stopped being a novelty the moment the tooling got cheap and good. Three families of models do most of the work.

Generative Adversarial Networks (GANs) pit two networks against each other: a generator that fabricates images and a discriminator that tries to spot fakes. They train in competition until the generator's output fools the discriminator — and, increasingly, humans. GANs powered the first wave of eerily realistic fake faces.

Diffusion models — the engine behind today's leading image and video generators — learn to reverse a noising process, starting from static and denoising step by step into a coherent image guided by a text prompt. They produce higher fidelity and finer control than earlier GANs.

Voice cloning uses autoregressive and transformer-based models that can reproduce a person's voice — timbre, cadence, accent — from only a few seconds of reference audio. Combined with face-swap and lip-sync techniques for video, the result is a deepfake: synthetic audio, video, or imagery of a real person doing or saying things they never did. What once needed a studio now runs on a laptop, and the quality curve is still climbing.

This is not a future threat; it has already moved real money. In 2019, the CEO of a UK energy subsidiary received a call in what he believed was the voice of his German parent-company boss — same accent, same melody — instructing an urgent transfer of about €220,000 to a supplier. He sent it. The voice was an AI clone. It was one of the first publicly reported cases of deepfake audio used for fraud.

The scale escalated fast. In early 2024, a finance employee at the engineering firm Arup, in Hong Kong, joined a video conference with what appeared to be the company's CFO and several colleagues. Every participant except the victim was a deepfake. Convinced by the familiar faces and voices, the employee authorized transfers totaling roughly HK$200 million — about US$25 million. The fraud was only uncovered afterward.

The instinct is to build a deepfake detector — a classifier that spots the fakes. It's a losing position, structurally. Detectors learn the artifacts of the specific generators they trained on: unnatural blinking, boundary blur, spectral quirks. But generators improve monthly, and a detector tuned to last year's artifacts generalizes poorly to this year's. Attackers can also post-process outputs — recompress, add noise, downscale — to scrub the very tells detectors rely on. It's an arms race the detector tends to lose.

Worse, detection creates a second-order problem: the liar's dividend. Once the public knows convincing fakes exist, bad actors can dismiss authentic evidence as "just a deepfake." Genuine recordings of real events get plausibly denied. The mere possibility of fakery corrodes trust in all media, which may be more corrosive than any individual fake.

You cannot win by proving something is fake faster than fakes improve. The stable move is to prove what's real.

The strategic answer flips the problem: instead of detecting fakes, establish provenance — a verifiable record of where a piece of media came from and how it was edited. The leading standard is C2PA (Coalition for Content Provenance and Authenticity), the industry effort behind the Content Credentials you now see attached to images from major cameras, editing tools, and AI generators.

C2PA works by attaching a cryptographically signed manifest to media: who or what created it, when, with which device or tool, and what edits followed. Tamper with the content and the signature breaks. It doesn't judge whether an image is "true" — it establishes a trustworthy chain of custody, so you can decide based on the source.

Complementing provenance is watermarking: embedding a robust, often invisible signal into AI-generated content so it can be recognized later. Google DeepMind's SynthID, for instance, watermarks generated images, audio, and text in ways designed to survive common transformations.

Until provenance is universal, the frontline defense is human process. The single most effective control against deepfake fraud is out-of-band verification: when an urgent request to move money or change payment details arrives — no matter how convincing the voice or face — confirm it through a separate, pre-established channel. Call back on a number you already have, not one provided in the request. Use an agreed code word for high-value transactions. Enforce dual authorization so no single person can be socially engineered into a transfer.

For real-time interactions, liveness detection and challenge-response — asking the caller to perform an unpredictable action — raise the bar, though sophisticated real-time deepfakes are eroding even these.

The deeper implication is societal. As synthetic media becomes indistinguishable, trust migrates from the artifact to its provenance and the institutions that vouch for it. The specialist's job is to build systems and habits that don't depend on the naive belief that a face on a screen is real. Skepticism, verified channels, and cryptographic provenance are the new literacy.