YouTube opens its doors to deepfakes

Today, let’s talk about platforms’ early moves to moderate the way people create and distribute media created with generative artificial intelligence. Announcements made by YouTube today about its own synthetic media policies suggest that the leverage in moderating deepfakes may not lie where we expected.

I’ll get to the YouTube announcements in a minute. But first, I think it’s helpful to frame how we’ve been approaching digital content moderation up until this point.

I.

In the first era of digital media — from the rise of Facebook onward — we put the primary responsibility for moderating content onto the user and the platform. US users carry the legal liability for most of what they post, thanks to Section 230; similar laws shield platforms from legal responsibility in other big markets. Platforms have some legal responsibilities — they often have to remove terrorist content, for example, or CSAM — but most of the moderation they do serves business interests. Most people don’t like spending time or money in places full of hate speech and other harms, and so platforms remove it.

The AI era of digital media has introduced a third character into the moderation stack: the tool. Generative AI tools can create realistic depictions of human beings, mimic their voices, and animate them on video. They can create erotica of various kinds. They can, if left unchecked, offer detailed instructions on how to build weapons.

We can debate how new this really is. Adobe Photoshop can also create realistic depictions of human beings, and erotica of various kinds. Talented actors can mimic voices and create convincing likenesses on video. You can get pretty far in building a weapon just by Googling.

Still, in the first era of digital media, we saw relatively little pressure on tools like these to perform content moderation during the act of creation. If you draw a naked human form using Photoshop, Adobe won’t interrupt you to ask you what you’re doing. We don’t generally prohibit legal prohibitions on actors from mimicking people. Google won’t delete your account based on your search activity alone.

There are some good reasons for this. One, historically we’ve mostly agreed that what you do on your computer is your own business, as long as it’s not hurting anyone. Two, and maybe more importantly, we’ve been able to count on platforms intervening to stop the spread of harmful material. A deepfake ...