Deepfakes in Cybercrime: Phishing, Biometric Fraud and Dark Web Services

For most people, deepfakes mean apps like Reface or Avatarify that paste the faces of friends or celebrities onto videos and GIFs. Rick McElroy, principal cybersecurity strategist at VMware, explains what sits behind them: audio or video produced entirely, or modified, by artificial intelligence (AI) or machine learning (ML) so that it convincingly shows a person doing or saying something they never did or said.

The harmless uses can be funny, but criminals are also turning the technology into a weapon. McElroy cites the widely reported case of a chief executive whose voice was imitated well enough to set off a US$243,000 wire transfer. While the public uses deepfakes to imitate people they know or admire, the prospect of attackers using them for anything from social engineering to disinformation has put the security community on alert.

Governments are paying attention

The increasing sophistication of deepfakes and their use in crime have become an international worry. Singapore's government launched a S$700,000 global competition under its national AI programme to build tools that detect and verify deepfakes. In the United States, the FBI published a report in March stating that malicious actors will almost certainly use "synthetic content" for cyber and foreign influence operations over the following 12 to 18 months.

McElroy says he spent several months talking with CISOs at major international companies about deepfakes appearing in security incidents. He groups their concerns into three areas.

Three concerns raised by CISOs

Phishing gets a new disguise

Social engineering still works on the same basic principle, but security teams are now seeing deepfakes added as an extra layer of deception in business communication compromise (BCC) and in phishing. These attacks exploit remote working, catching employees out with a well-timed voicemail or voice message faked to sound like someone they know.

According to VMware's Global Incident Response Threat Report, 32% of respondents had seen attackers use business communication platforms such as Microsoft Teams or Slack to help them move laterally during an attack. McElroy argues that phishing through email or these platforms is especially open to this kind of manipulation, because it trades on the trust employees and users place in a virtual workplace they know well.

Biometrics can be fooled

The spread of deepfakes raises a host of problems for identity authentication. Multifactor authentication is a foundation of good security practice, and biometrics have become an important factor because every person's are unique. Several governments now use biometrics to verify national digital identities. India's Aadhaar programme takes fingerprints and retinal scans during enrolment and is the largest identity scheme in the world, having issued 1.3 billion Aadhaar numbers as of 2021.

Keeping track of identities as they move is central to an organisation's internal security, and deepfakes able to defeat biometric factors sharply raise the risk of compromise. Experian has named synthetic identity fraud the fastest-growing form of financial crime. As criminals use deepfaked faces to trick biometric checks, McElroy expects businesses that depend on facial recognition to run into serious new obstacles in their identity and access management.

Deepfakes as a service

Ransomware turned into ransomware-as-a-service (RaaS), and McElroy sees deepfakes following the same path. Threat intelligence firm Recorded Future found that sellers on the dark web now offer made-to-order services and tutorials for getting past security measures using visual and audio deepfakes.

In McElroy's reading, this shows attackers going beyond the deepfake-driven influence operations the FBI warned of in March and using synthetic audio and video to slip past security controls. Criminals are also sharing deepfake tools and techniques for breaking into organisations on the dark web and through other channels such as internet forums and messaging services.

Criminals adopt first

Distributed workforces rely on video conferencing for meetings and everyday contact, which produces a large supply of audio and video that can be fed into machine learning software to generate convincing fakes. McElroy notes that modern attackers treat nothing as off-limits and are often early adopters of advanced technology, hoping to exploit defenders' unfamiliarity with it to get inside the perimeter.

The availability of deepfake tools on the dark web, and their likely role in future attacks, should warn every CISO and security professional that attackers are ushering in a new period of distrust and distortion, he writes. His advice is to "never trust, always verify", above all now that reality itself can be manipulated.