Stefan: [00:00:00] We were seeing that the situation that happens on Facebook where children were being introduced to the person that ended up abusing them using the suggested friend, right? So Facebook will say like, "This is a person that you should be friends with." Like, that was the concept. And so there were, there were-

were

Alix: they connecting kids to adults?

Stefan: Yes. They were.

Alix: Hey, guys. This is Alix. Welcome back to Computer Says Maybe. Today, I'm talking to Stefan Turkheimer, a good friend and vice president of public policy at RAINN. If you've never heard of RAINN, you'll learn more about them in this episode, but they're the organization that runs the National Sexual Assault Hotline.

You might be thinking, "What does a sexual assault hotline have to do with tech politics?" Well, as it turns out, quite a lot. It is the basis for RAINN getting really actively engaged in trying to make better laws that hold big tech accountable. Stefan didn't set out to build a career fighting big tech.

[00:01:00] His work has always been focused on helping survivors. But over the last six years, calls to the hotline have gone up dramatically, driven in large part by online sexual abuse, deepfakes, non-consensual images, CSAM, and more. And Stefan didn't want RAINN to just be the organization answering those calls and treating those harmed.

He wanted to get at what was causing the harm in the first place to figure out how to stop it. So he ended up trying to write laws that hold tech accountable for all the ways that they're building really garbage products, um, that are being weaponized against all kinds of vulnerable populations. So we are gonna get into what it takes to make a good tech law, how not to get too pessimistic about the likelihood of passing good tech regulation, some of the bills that he's managed to get passed in the last few years kind of showing that it is possible to make laws that increase the governance of tech.

Last week, we talked with Taylor Barnes about how easily a tech company can get the law changed when it stands in their [00:02:00] way. This week, we're gonna be looking at the flip side, like, why is it so hard to use that same system to hold them accountable? For a dark topic, Stefan is a bright light. He's one of the funniest people I know, and I came away from this conversation thinking really differently about regulation and about what it actually takes to make good laws in a complicated system.

Before we jump into the conversation, as you can tell from this introduction, this is a discussion that is full of references and examples that come from really serious situations of sexual abuse. If that's not something you want to listen to today or if you have kids around and you want to be able to manage what they're exposed to, um, maybe sit this one out, um, or listen when you feel a little bit more grounded, uh, and ready.

But with that, here's our conversation with Stefan.

Stefan: I'm Stefan Turkheimer. I'm Vice President of Public Policy for RAIN.

Alix: So when we first met and we started talking about like tech stuff, I didn't actually understand [00:03:00] really how your organization connected to like tech regulatory things. And I feel like once I better understood the substance and the kind of people that you try and protect and support as part of your work at RAINN, the more I kind of got into, like, political logic of, like, why you are interested in regulating tech in certain ways.

So can you just take us through what RAINN is and the kind of work that you do there?

Stefan: Yeah. It's a great question. So RAINN has a terrible acronym, which I will tell you, which is the Rape, Abuse, and Incest National Network. So RAINN's origin story, it's about 32 years old, RAINN's origin story, it was a way to knit together all these crisis hotlines that existed all throughout the US, right?

So that's what... That's the whole thing for it. That's why it was a national network, and those were the type of organizations it was connecting. So that's why it's called that. Now, RAINN runs the National Sexual Assault Hotline and a, a number of other hotlines related to it. And so when people call, like, that 1-800 number that people are aware of or they, they go into the chats, those are RAINN people that are answering those.

And then when they're getting connected [00:04:00] to local rape crisis centers or what we call- Yeah ... sexual assault service providers. That's, that's... So that's the origin of RAINN. So, like, you had a really good question the other day, which is like, "I didn't know you guys did this work," right? Oh-

Alix: Can you give the specific example?

'Cause I remember it, that I was, like, shocked that you guys did it.

Stefan: Oh, why don't you just tell me?

Alix: When I'm in an Uber, if I push the, "I'm, like, in trouble," or, "I need help" button-

Stefan: Yes ...

Alix: and then I get connected to someone that's there to, like, help me in a situation, could be, like, a driver is making me feel unsafe or, like, any situation you end up in an Uber, it's a RAINN staff person or, like, a hotline- Yeah

person that answers it.

Stefan: That goes to the hotline, yeah, and they have special resources to deal with that situation. That's, that's the thing about RAINN, right? It's, it's a hotline that does a bunch of other stuff, but the other part of that is, like, why do you do, like, tech accountability or why do you do tech safety?

Why do you do internet safety? And it's because... It's not because RAINN is intending to do that. It's because that's what people are calling the hotline about. Like, so [00:05:00] basically, like, what- Ah. Right, right. So, like, 500,000 people call the hotline each year, and it's gone up since the pandemic, right? So what happened is, is, like, during the pandemic, the number of kids reaching out to the hotline went way up, and everyone at RAINN thought that that was, like, a temporary situation because the normal avenues for reporting, basically schools- Yeah

were shut down. And so we felt we were getting more calls from kids because of that, and then that number will drop off when the pandemic ends, if it's ended. And so everyone expected it to drop down, and it did a little bit, but not as much as we expected it to. 'Cause we thought, like, when schools open, you have school counselors, all that kind of other stuff, right?

Alix: And you're not stuck in a house with your potential abuser.

Stefan: Your potential abuser. Yeah. Yes. That's the other part of it. We thought those two things would lower it back down, and they went down a little bit, but not that much. The, the-

Alix: Why... What do you, what do you attribute that to?

Stefan: Okay. So what's happening there is a lot of it is tech-enabled sexual abuse, right?

Sometimes through deepfakes, sometimes through non- nonconsensual intimate images, [00:06:00] sometimes in situations where someone who has access to the child over the phone- Who is like sometimes a family friend, sometimes a coach, that sort of thing, is able to convince the child to create their own CSAM. So RAINN has gotten into the business of tech accountability because people are calling the hotline about tech-enabled sexual abuse.

So that's how we ended up there.

Alix: Right. So you're dealing with individual cases that stem from a structural problem, and rather just continue to deal with the individual cases increasing in number and complexity and whatever, you're like, "Let's work on the structural issue that's leading to these cases."

Stefan: So when those kids hang up from the hotline or when anyone hangs up from the hotline, they're l- that's our problem now. Like, that's our job to solve. And so peop- when people call the hotline, we'd be very clear that all their conversations are, are confidential and anonymous, but there are reports based on like what the generalized situation is that are non-geographical, non-specific that are sent to us.

So like for instance, we were seeing that the [00:07:00] situation that happens on Facebook where children were being introduced to the person that ended up abusing them using the suggested friend, right? So Facebook will say like, "This is a person that you should be friends with." Like, that was the concept. And so there were, there were- Were

Alix: they connecting kids to adults?

Stefan: Yes. They were. And the matrix that they were using, or the algorithm, if you will, that they were using was essentially like, "Oh, well these are dudes that wanna be friends with kids, so... And they're-"

Alix: Oh my God.

Stefan: I know. I know. God.

Alix: I mean, Mark Zuckerberg should be in jail for so many reasons, and this is just like the 10,000th that I've heard.

Okay, so most people think that tech should be more regulated than it is. I think that's like a general consensus. The people that try to regulate tech oftentimes are kind of pessimistic about being able to do it well, and there's kind of a feeling that like it's [00:08:00] so important, and also we couldn't possibly do it in a way that doesn't lead to all kinds of secondary effects that are bad and more harmful than not doing anything at all.

You like pass laws regularly as part of your work, and I feel like you're kind of like lapping this community of people that is talking about doing regulation, and that's one of the reasons I wanted to talk to you. But also what you're doing is not just tech laws. So I don't know if you wanna take us through a couple of examples that you have successfully passed laws recently, but I think it's kinda cool- Yeah

if we can dig into them. I

Stefan: mean, I, I think Let me just start by saying that that pessimism is earned. There's a reason why people don't think it's possible, because it often fails, right? But the problem is, is like there's a self-fulfilling pro-prophecy there, which is like you look at the 100,000 bills that Congress introduces, and you look at the 1,000 that become law, and you're like, "Oh, wait, they're all post office renamings."

[00:09:00] And there's a part of you that says, "This isn't gonna happen." Like, all we're gonna do is introduce a bill, right? Introducing a bill is hard. It's like that's all we're gonna do. And you get sort of a learned helplessness from that, right? And so you end up trying to do things in other ways. You try to shame them publicly, you try to do whatever.

And all of those things are good. Shame is a motivating force. We used to try and get rid of it, now it's a solution. But it's wrong, frankly. It-- that cynicism is wrong. And I know it's wrong because I know what happens when you don't have it. One of the examples I like to talk about is that Ohio had a spousal exception in their rape law up until a year and a half ago.

And what that meant is that when someone found out-- when a spouse found out that they had been drugged and sexually assaulted, and they went to the police and said, you know, "This person's recorded me. I have this video. I have all the evidence," the police would say that, "There's nothing we can do 'cause it's not against the law here."

[00:10:00] And they've been saying that for years. In fact, there are people that have been trying to change that law in Ohio since 1985. And When we wanted to change that law, a lot of people told us it wasn't gonna happen. Like, that, that no one-

Alix: Why?

Stefan: Uh-

Alix: That seems like such a- Oh my God ... layup.

Stefan: Well, so everything looks like a fait accompli, right, in hindsight, right?

But when you're actually, when you're trying to do it, when you're in Ohio and you're, like, going door to door, they're like, "Well, I agree with you, but it's not gonna happen," right? And it's like you can take that and you can say, "Oh, it's not gonna happen 'cause it's never happened before." And, and to be fair, like, we failed twice before we succeeded.

We failed in, in 2023, or ooh, actually we failed three times. We failed 2022, 2023, 2024, and passed in 2025. And Yeah, they're right until they're wrong. But, but, but the problem is it's like, yeah, the thing about it is, it's like being right isn't everything, right? Being right isn't everything, but it helps.[00:11:00]

And people don't wanna be wrong publicly. They're okay with being wrong privately. They're okay with, like, killing a bill in the back room. But if you can force it into the open, you have a much better chance of getting it passed. And so, like, the thing that I talked about to my staff, 'cause, like, when you really charge at something hard and you don't get it passed, like, it hurts.

But the thing about it is, it's only a failure if you give up. Otherwise, it's just learning. You either learn or you win. That's it. And you learn until you win. And so what we figured out is like, okay, what are our problems here? What is the problem with this bill? What is the situation? And what we ended up having to do to fix the problem we were having is to expand the coalition.

The question is, is like who do you bring in? And so we ended up bringing in the Junior League of Cincinnati so that when legislators went home, back to their districts, they got harassed by the ladies of the Junior League about this bill. And those same people came to the capital to [00:12:00] remind those reps that they were paying attention.

And so that's what forced the bill to the floor. And on the floor, the naysayers still thought it was gonna fail. They thought it was going to lose on the floor, as in, like, there would be more votes against it than there were for. But when it finally came up to the vote and they lit up the board, everybody was green except for one guy.

One guy voted no, and he was chased around by a guy dressed as a chicken, actually, after the vote. So there's video of that on Twitter, but it does exist. But that's the thing. Everyone thought it would fail. Everyone thought it'd fail 'cause it always would, right? It always happened. But if you accept that cynicism before they prove it to you that they can win, you've just beaten yourself.

There's a structural problem, but there's also a structural inequity between us as individuals and trillion-dollar tech companies

Alix: Okay, so let's get closer to one, 'cause I feel like marital rape, I can see the strategy. You expand the coalition, you make people aware of the fact that it's [00:13:00] still on the books as a- Mm-hmm

'cause I imagine people were very surprised when they learned about that. And so then it becomes easier to imagine a strategy of, like, making it feel basically impossible to not pass that law. When you get to other pieces of legislation, and I'd love to hear your reflections on the Take It Down Act, maybe explain what it is and- Yeah

how you came to that, and then maybe the Minnesota law.

Stefan: So a lot of people who were calling the hotline had a problem where they are being continued, continuing to be harassed, a lot of times by ex-partners, with images that they took during a relationship, they took consensually, but they're being shared non-consensually.

That was the origin of it. During the process of creating that bill, deepfakes became a burgeoning problem, so deepfake intimate images. And so we really wanted a bill that would allow people to be able to take control of their own images. We realized very quickly that [00:14:00] we were going to run into the perfect plaintiff problem, which is the situation where if the person that you're bringing to testify about the bill or the person that is the emblem of what the bill is about is somehow blameworthy in the minds of people that don't wish to do anything, that it makes the bill harder to pass.

And so-

Alix: So basically if you started from the point of departure of like a sex worker who is being... That basically like certain communities are easier in sort of political ways and partisan ways to basically vilify in a way that kind of makes that harder as a point of departure to protect as a law. So you try and find, um, something that is universal in nature, feels new enough, and feels sort of generically threatening enough, um, that you can kind of like paint with a- Yeah

blank canvas. Yeah. Okay.

Stefan: Well, basically we tried it the first time with a broad-based point about, well, people should get to [00:15:00] control their images no matter how they're created, whatever, and that didn't work. So we were like, "Okay, we need to narrow this point." You need like the... It's like a spear, right? You need to make the end of it as sharp as possible, and then you widen it out.

And so that's what we did. Because what happens, like yeah, the sex worker issue is, like sex workers deserve every control over their own body. The issue is there's a contract issue in terms of permissions, in terms of the consentual nature of the images that are created that somehow made that a more difficult argument.

So you end up getting like, when I would be arguing like, "Yeah, sex workers deserve all the protections," you quickly get into a situation, they're like, "Well, but this someone paid for this image, so don't they have the right to- distribute this image. And you're almost in this, like, copyright argument as opposed to a bodily autonomy argument.

So it gets you into a place you don't wanna be in, in terms of sort of passing a law. So sex workers' images are totally protected under this law, right? But the people we talked about, the people that sort of came to [00:16:00] Capitol Hill, were like people who were 14-year-olds in school, and their classmates made deepfakes of them and distributed them around the school.

And the reason why is because it's so difficult to oppose that. It's so difficult to oppose that. So that was the tip of the spear, and they were so brave. And like all of these bills, whether you're talking about Ohio marital rape, all of them get passed because survivors decide to take this point of pain and make it into a point of change.

Alix: 'Cause it feels like this is an entry point to a bigger political fight happening right now, which is the extent to which you have to front-load framing issues around people who are particularly marginalized and particularly impacted by an issue as a way of more deeply connecting and more kind of like intersexually- intersectionally engaging with an issue.

And then other people who I think sometimes have not very good politics are like, "No, no, no," like, "Don't talk about the most marginalized communities as the [00:17:00] entry point," because, like, you know, the non-educated people are just gonna, like, be turned off or, like, are herded away from supporting that policy position.

And that feels like a microcosm within the tactic that you just described, and I wonder how you feel about that political question more broadly.

Stefan: So, okay. So I mean, like for instance, trans communities are especially vulnerable and are especially need to be protected by whatever policy you come up with.

So when you're assembling your policy, you have to absolutely start from the people that need it the most first, right? So if you're talking about, like, trans communities, you know, there's a lot of sort of vulnerable communities that we sort of think about, and we really try to apply those lenses to whenever we're creating a policy, right?

You also need to think about what messaging works on what people in terms of pass get bill. We talk a lot about trans in- inclusivity in our policy work But we also [00:18:00] know that when you're thinking about going up against Meta, right? You're going up against Meta, you're going up against Twitter, you need as many people on your side as you can, right?

And so one of the ways to do that is to give them no place to hide by bringing forth stories in addition to data. So you can say like, "These communities are most affected," and then you need a story from each community, right? And you need people that are willing to come forward to testify, willing to come forward to put their face on it, and that's not always safe for everybody.

And so you really have to think about like, can you protect this person? What are you asking of them? Like legitimately, like what are you asking them to do in this moment? Are they safe doing it? And so, you know, we don't always have as broad-based a group of testifiers and people that are at the front of it as we would like, honestly.

But we also want to be able to tell stories [00:19:00] based on real situations, because it's like you can go in there and you can say, "Well, 96% of deepfakes are intimate images or non-consensual intimate images." You can do that, and they'll be like, "Oh, that sounds bad." But it doesn't motivate people in the same way that someone coming forward and say like, "Look, this is what happened to me and this is what's allowed, and if we change this law, it'll fix the situation."

And so Getting as many stories like that together is what gets you a good policy and helps you to get it passed.

Alix: What's interesting about your tactics is I feel like you're very clear-eyed about the prize. Like, your eye is on the prize of a piece of legislation that is encoded in law that protects in universal ways that include communities that maybe a Republican who has now signed this bill wouldn't necessarily want to support if you made the bill explicitly about those communities.

So finding ways of, like, tactically getting the wind at your back on an issue in a way that leads to legislation that's good and positive for everyone, and even [00:20:00] specifically for marginalized communities in a lot of ways, but not necessarily, like, leading with that in a way that's alienating and then undermines your overall effort, is what I'm hearing.

Stefan: Yeah. I think that's entirely true, but also you have to think about their politics too, and what they want to be able to talk about. You know, there are conservative MAGA majorities, to some degree, in control of both houses and in the presidency, and if you need a federal bill to protect people, you have to convince them in order to get it passed.

I want people to say when, when they call the hotline now, and someone says, "These images are out there and I don't know what to do about them," the hotline person can be like, "They're gonna be gone in 48 hours, and here's how." And it's like that required a narrow policy fix and a narrow messaging arc, but the reality is is that anybody who wants to be a part of that can be a part of that, and anyone needs to be a part of it if the law is even gonna work.

One of the big problems with tech legislation is, like, we try very [00:21:00] hard to be like, "If you want a seat at the table, if you have something to say, you should have a seat at the table." The problem is the table has 10 chairs, and nine of them are occupied by lobbyists, so we all have to sit in the same chair.

Right? That's really good. Like, that's what's happening. So wait,

Alix: how, how did you get it passed?

Stefan: The thing about Take It Down is Take It Down, much like many bills that are introduced, is a good idea, and if you lay it out to people and say, like, "This is what we're trying to do," most people will be like, "That's a good idea."

But unless that desire for change is communicated all the way up- It's not a safe place for legislators to be, 'cause they don't know. The people they have most contact with are, and I say this with all due love to myself, lobbyists who are representing a point of view that is not necessarily consistent with that of the American people.

It just isn't. It's consistent with an economic interest in a lot of cases, it's consistent with a segment of the American people, but it's not necessarily a broad-based understanding. But the nice thing about democracies is that you do have elections, right? [00:22:00] And so you need to raise the salience of your issue up.

That requires money, that requires a lot of things that we don't necessarily have, right? So the first time through in 2024, we were able to get this bill, take it down out of the Senate through, like, bipartisan actions of Amy Klobuchar and Ted Cruz, who were able to- Oh my gosh ... muster. I know.

Alix: I'm s- you just say those two names so casually and then move on, but those two.

Carry on.

Stefan: Yeah, those two. I mean, yeah, the shining lights of our republic. The reality is, is that, like, they were lovely and Senator Cruz was incredibly kind to every survivor who wanted to meet with him, and he really put his shoulder into this bill to get it into, in 2024, to get it into the end of the year omnibus bill that is sort of passed traditionally at the end of sessions.

Biden was still president, but Trump had won. There was a mood in the country that things were changing. We really wanted to get this thing passed as best we could in a situation where it was most favorable to us. [00:23:00] So it got stuck into the omnibus. Everybody was on board. All the various parts of the omnibus were coming together.

It was gonna be passed out, and then Congress would gavel out and that would be the end of it. And then Elon started tweeting about it and about how bad it was, and he was able to blow up this entire bill, just rip it apart at the seams. And The position of h- of, of, of Elon was so powerful at the time because he had just sort of powered the election through his work, through his money and his efforts.

And so he was at sort of the height of his power at that point. And so we were not able to get our bill back into the omnibus. The omnibus was reassembled and passed without our bill in it. And so when we came into 2025, we weren't really sure what to do because we needed, we had Senator Cruz, who is incredible in the Senate, [00:24:00] but it's just not enough by itself to get the bill passed.

We were very lucky because the First Lady, Melania Trump, was very concerned about internet safety and had previously gone to the, to the platforms and said, "Hey, is there a way that you can make this more safe for children?" And they essentially told her no. And she had never been involved in legislation before, not really, and so it was, like, a really big thing for her to take a public stance on this bill.

And so she came to the Capitol, and she hosted a round table that really put the House on notice that this was the power of the White House, of the East Wing of the White House. And then she really put her name and her efforts behind it and made it the central part of her advocacy in 2025. And that put the House in a position where they had to take it seriously.

They couldn't just ignore it. And so if someone commits to giving you a vote, and you're able to hold them to that, that means all the members of that committee, all the members on the [00:25:00] floor have to either vote on it or run away from it. And, like, that was what she gave us. She gave us the ability to have a public vote.

Alix: So getting Melania Trump to host a round table in the House where I assume Mike Johnson was not part of spearheading that meeting. Melania just shows up and is like, "This is happening."

Stefan: Yeah. Yeah.

Alix: You

Stefan: better do it. And, you know, she's someone whose image has been used against her will. I, I think, like, she was very aware of the problems that kids were having online, and she thought maybe this is an opportunity to do something about it.

But most people find, including, including the First Lady, I think find Congress a bit distasteful. Most people don't wanna be involved in it. Most people don't wanna be involved in making laws. For most people, it's the easier thing to do is to do nothing about it.

Alix: Yeah. So this is what I wanna ask you next.

To me, that sounds like a constrained piece of legislation designed [00:26:00] to put some administrative onus on platforms to take very specific action around something that's demonstrably about the individual making the complaint. I can't really, like, I'm usually pretty good at figuring out how something could be abused in some way when people start talking about regulation, 'cause I have kind of a, maybe, like, a tech politics libertarian little engine in my brain that can be like, "How will a government abuse this?"

Or like, "How will this be manipulated to further hurt marginalized people?" I have that little, little loop in my head, and I take that law in that loop, and I- Can't actually figure out the thing. But some people didn't like it on the sort of tech regulation, tech policy space

Stefan: I think they were worried about it being used as a tool for censorship, so basically false complaints being issued to platforms and them taking the image down and not having a very clear appeals process for how to handle it.

Because you are

Alix: relying- For smaller platforms, maybe? That smaller platforms then have this onus on them that, like, is more...

Stefan: Yeah, but, like, smaller platforms [00:27:00] aren't necessarily required to take the same actions as the law. I mean, like, there's a bunch of little things in there that kind of protect them in some ways, but also it's like there's two parts of an image in this particular analysis, right?

There's the non-consensual nature of it, which is being attested by the person that's in the image, right? And then there is the intimate nature of it, right? So there's two parts of it. So when, when a platform gets a request, they can, either using AI or using a human, look at the image and decide whether it's intimate in nature, and if it's not intimate in nature, they cannot do that.

And then the other thing they can do is then they can take down the image and alert the person who posted it that this image has been la- messing with non-consensual, and then that person can, can then argue that it is either not intimate, I suppose, but or it is in fact consensual, that they had a right to share it, right?

So, so there are sort of safeguards there. Now, the thing about when you're trying to create a bill, you wanna specify the right, but you don't want to specify the procedure to the T. Interesting. And the [00:28:00] reason why is because it's like you need it to be flexible as things change, right? You don't want this to not work in two years because the way images are done are differently and you said, like, "Well, they have to do a specific hash matching system," which I want, right?

I actually wanted a specific hash database that would be shared, and that's not what we've got right

Alix: now. So that way if it had been posted on multiple platforms, you could complain to one and then it gets taken down everywhere.

Stefan: Yeah, and then it gets sort of shared across, and we're not to that point yet, though I'm still trying to get that, but I mean, that's what I would like, right?

I would like a, I would like basically an immediate ha- I would like the FTC to hold a hash of all the images and that prior to upload, all platforms are checking against that hash database in the same way that they do for trademarked images or copyrighted images. Like, we should give someone who's been the victim of a deepfake the same protections we give Mickey Mouse, and it shouldn't be hard.

That's a rat. It's not even a real rat. So... But we don't do that, but I think we should

Alix: Do people- Whatever ... reach out to you though that are more, like, focused on the [00:29:00] harms of surveillance- Yeah ... and say, like, "Have you ever thought about- Yeah ... surveillance?"

Stefan: Yeah, yeah.

Alix: Obviously you do. Like, do you wanna talk-

Stefan: Yeah, I, yeah

Alix: like how does

Stefan: that happen? I try to account for it.

Alix: How does it work?

Stefan: Well, I mean, honestly, I'm thrilled when people reach out to me. Most people do not. Like, most people would only sort of criticize a bill from afar, and I would much rather they reach out to me and say, like, "This is what you got wrong." I'm knowledgeable, but I'm not an expert in everything, right?

And so if there are places where I'm messing up or I've got a policy that's wrong or I've got language that's wrong or, or if I'm not being inclusive of people that I'm missing or whatever it is, I would love to hear, like love to hear from it. I do think that the other problem though is, is that the privacy arguments are often co-opted by entities that actually don't care about privacy but are there to try to defeat any type of legislation.

And one of the big problems in legislation overall is it's easier to stop something than it is to get it passed. And so the sort of amorphous complaints about surveillance or privacy certainly have validity, right? But the problem is, is that they're used to make sure we do nothing [00:30:00] And the question is, is like how do we protect people's privacy?

And that's a tricky one, right? 'Cause it's sort of like even with the hashes. So like when I was talking about a hash database, it's very simple to say like you keep a list of all the hashes and then you check against it. People also change images a little bit to try and get through checkers. There's also a question like, you know, you find out there's an image of you online, and then I'm like, I have this individual image, and they've actually made...

The person who did it has got 20 images that are very similar to it, right? So it's like, okay, I get that one taken down 'cause that's the only one I can see publicly, and there's a bunch behind a paywall. The rest can just sort of continue. Like we think a lot about what is the onus on the survivor in this situation, right?

Does the survivor have to continually chase their image all around the internet, sending takedown requests all over the place? That's not what we want. We want an automatic system. You do it once, it's done. You don't re-traumatize yourself. You're not worried about every text message you get. Absolutely not.

But we've got a situation that's closer to that than not right now. And so the way you sort of fix that is you essentially, you [00:31:00] increase your error bars, if you will, on your hash, right? And what that does is it's going to produce some false positives, but it's gonna get a lot of the lookalike images, right?

So it's like, okay, that's the question for me when we're deciding- Interesting ... how, how much are we okay with-

Alix: So it's less about censorship and more about error rate related to matching and taking something down.

Stefan: In this particular example. In that instance, yeah. Yeah. I mean, I'm not trying to say that there aren't situations in which an attempt to outlaw CSAM affects people that are not doing CSAM.

That absolutely occurs, right? But yeah, trying to come up with a system that both protects the people it needs to protect and also does not hit any of the people that it doesn't is a little tricky. Yeah. And there are ways in which to do that. So the question is though, do you want it... Do you want to have a system that misses a bunch of stuff, right?

That there's a bunch of false negatives, or do you want to create a system that is over-inclusive and hits some false positives, and then causes people some problems and has some of their images taken down, and they have to appeal [00:32:00] I'm okay with that. I actually think they ma- I think we could take millions of images off the internet and be just fine, but apparently other people disagree and they think it's their right to upload nude images of various people at all times, and like totally.

And I get both arguments, they're both fine, but the reality is, is like we have to decide- We have to code ... which harms we're okay with- Yeah ... and what the burden is on those people experiencing the harms, because the vulnerable communities, in my mind, are the ones that should not be experiencing the harms.

We're gonna get into a situation which is like, yeah, you can weaponize these laws in order to hurt vulnerable communities, right? And it's like, yeah, totally, we have to guard against that too, but this is all the sort of the back and forth of what's going on here, and we have to sort of also, like there's bad actors in any system, but that's not the same thing as using a good law for a bad purpose.

That's a thing that can happen.

Alix: You mentioned Musk tweeting about- Did I? ... when it was gonna be in the omnibus in like 2024. Do you wanna talk a little bit about the Minnesota law? 'Cause- Sure ... isn't he actively... Yeah. So-

Stefan: Yes,

Alix: he has a problem with it ... so weird that like Elon [00:33:00] Musk is like your regulatory nemesis.

Yeah. Okay, so Minnesota-

Stefan: Abeba,

Alix: Abeba ... this, is it a nu- I heard it was a denudification or nudification law? Is it nudification?

Stefan: Uh, the law basically is trying to, is, is basically the first in the country to try and get at the apps that are doing it, as opposed to the people that are triggering the apps to do the thing, because the apps are the nexus of harm.

All of these are happening 'cause the apps. No one is sort of just undressing people, like very few people. It is certainly possible to do if you're good at like Photoshop or whatever, to create a nude image. This actually doesn't affect that at all. If you've got the skill to do that- That's, you know, your right to do, well, right is strong, but it- this law doesn't affect it, right?

This is about situations where you're like, "I've got an image of someone, I don't know them, not really," right? And I send it to the app company, and I hit a button, and it gives me n- you know, 10 nude images back and a video, right? Where I pay a, a, a little bit of money to do that. It's so gross, and so that's what the law's attempting to do, and it basically creates a fine structure for the attorney general to go after those entities or [00:34:00] the person that's affected.

The person that was the subject of the image can also sue the nudification entities. It's gonna be difficult because some of these entities are in Luxembourg or other places that they're beyond the reach of US laws, but there are ways to attach things that they own in the United States and whatever else.

But there are some that are absolutely you can go to, and it's odd that, like you said, we're gonna try and get, go after these apps that are like one-stop nudification shops, like one-stop CSAM shops in some cases, and xAI and Grok hold their hands up to be like, "We're gonna be affected by this. We're gonna sue you two days before the law goes into effect to try and stop it," which is not how reasonable companies behave.

Alix: Who is protecting Grok, you know?

Stefan: Who... W- Won't someone think of the robots?

Alix: Yeah. Yeah. I mean, Musk's robots in particular, although did you hear that it got listed by, I don't know what department of the federal government, but as a national security asset to help them possibly dismiss the lawsuit against [00:35:00] xAI as a facility, the NAACP and Southern Environmental Law Center.

The DOJ said it was like a nat- Grok was a national security asset. Yeah.

Stefan: I did not hear that. Thank you, I guess.

Alix: I'm so sorry for letting you know.

Stefan: Yeah.

Alix: So then did it go... It, but it went into effect, right?

Stefan: Yeah, it went into effect, so the judge denied their temporary restraining order. There's now a hearing on their sort of permanent injunction of the law, and then there'll be a full merits hearing after that.

And, you know, we would like to file an amicus in support of the state here, uh, which we will probably do, though that's sort of at press time. But yeah, it's, look, I mean, the reality of the situation is, is that there are more and more stories of xAI, Grok, Twitter, whatever, being this sort of nexus of harm in creating images that are tantamount to CSAM, though in some, maybe in a technical sense aren't, but it's a little unclear because everything is behind a paywall like Grok Imagine now, only for paying users, and this is essentially a part of their [00:36:00] business plan.

This is a profit area for them. And so when people are like, "Well, why is Elon Musk suing?" And it's like, well, what do I know about Elon Musk?

I know that around the beginning of the year, he spent a lot of time telling people, "Please don't create CSAM using my products, and if you do, it's your fault." Right? When it's very clear that they have a situation in this bill where, where they're like, their argument in, in the brief is, "We don't allow people to create these images."

Right? Yet these images are created. But they

Alix: obviously do. Yeah.

Stefan: Right. And so what the bill says is like, for entities, for apps that allow this behavior to occur, right? So their whole argument is, "Well, we don't allow this behavior to occur because it's not in the terms of service, and we're doing technical things to make it not."

I wanna say two things about that which are totally ridiculous. Well, my points aren't ridiculous, theirs [00:37:00] are. Okay, one,

Alix: you can't- Not allow implies prevent.

Stefan: It does imply prevent, doesn't it? Yeah. But what's interesting is the head of the, of this whole thing, Elon Musk, is often on Twitter talking about how great this is that you can make all these images, so it's not really a great argument.

But also, you can't make CSAM, you can't AI generate CSAM unless you have training data that is CSAM So the real question is, is like... And then so California had a law that said like, "Okay, well, these, um, big AI providers, these big image-generating providers are gonna have to provide their training data," and they're also getting sued by xAI to prevent that disclosure.

And Amazon, when they were looking at their work, did a search of their own training data and realized there was a ton of CSAM in there. And it's like, yes, you could have reverse engineered that from the fact that you're generating CSAM. And so that's sort of the issue here, which is like, what does allow mean in this context?

Does xAI, does Grok Imagine [00:38:00] allow nudification to occur? There are apps in the world whose whole job is to create nudification, right? And that's really who's the target of these laws. It's really these people that like... Nudify is one of the apps, whatever, that people use that are sort of the subjects of harassment, people go to it exactly.

But the reality is, is that Grok is probably the biggest creator of these images, and so they're the ones that are holding their hand up to defend them.

Alix: Yeah, which makes sense. I also think he likes throwing his weight around when anyone implies that they may be able to make him do something.

Stefan: Yes. Like he thinks- He thinks he can win

Alix: generally. Yeah.

Stefan: Yeah, he thinks he can win. He thinks he will win, and he has enough money to make it difficult. I mean, that's the reality of the situation. And so the problem that he's got is if the Minnesota law is allowed to go into effect, then other states are gonna copy it. So he's trying to put Minnesota in a defensive position based on their law.

It's not that he can't win, he just thinks [00:39:00] he deserves to in all situations. And that's... And his goal, I think his goal is to make his user base angry at the politicians in Minnesota by saying that he's gonna turn off all Grok Imagine capabilities to people who live in Minnesota. And I think you're gonna find out that no one in Minnesota is that upset about that

Alix: But no one comes to defense of abstract civil liberty protections either, which makes me like... So in this case, I'm like the fact that nobody's gonna be like, "Oh," no one's gonna be like, "I wanna be able to nudify people," or, "I think people should be able to do that," or poor Grok, you know, smallest violin, whatever.

But the same is also true for other types of civil liberties that people think of as abstract or, like, they don't quite get them or they don't really care about them in an exchange of, like, potential regulatory progress weighed against some, like, future possible harm if someone [00:40:00] abuses a law. And it makes me wonder a little bit, like how...

'Cause I can hear you, and I know that you, like, internalize a lot of the political thinking and, like, legal risk analysis of people that are more concerned, maybe not more, more concerned isn't fair, but, like, are so deeply concerned that sometimes they are immobilized by the civil and political aspect of some of these, these laws.

I can hear you balancing these things, but, like, what do you think makes good tech regulation? It's partly the spear thing that you described, where it's narrow enough and specific enough that it's hard to imagine it being abused or there's a specific enough thing, but it's not so specific as to be technically prescriptive.

Can you, like, share more about how you think about what a good law is?

Stefan: I think, like, I think making a, a good tech law is made harder by the way in which we do it, which is that people try to create these large laws to completely regulate the space, and then it falls apart somewhere in it, and you lose your [00:41:00] coalition, the whole thing falls apart, right?

In most cases, it's better to figure out what the exact right is and try to perfect that right, try to award that right to the people that deserve it. The tricky part in doing that is that you need it to sing with every other regulation out there. You really aren't trying to create a bunch of different laws.

You aren't trying to create a bunch of different standards for a tech company to meet, right? What the tech company often talks about and the reason they want preemption, or one of the reasons they want preemption, is they don't think there should be a different standard in Florida than there is in Iowa than there is in California, right?

No 'Cause then they have to-

Alix: They want it to be easy for them to comply. Yes. They don't care if there's a different standard between states. They just want it to be easy.

Stefan: They would prefer no standard. Yeah, yeah. The problem is, in writing this law, what you're really trying to do is get the tech behavior to change.

That's what you really want, right? You would like it to be safer and all these kind of things, but the problem that tech companies have is that they are either public companies or they're trying to get to an IPO, and what they need is growth and revenue. [00:42:00] And the reality is, is like when those are their only targets, safety concerns are gonna lose every time.

So it's not like there aren't trust and safety people at these places that are bringing up the issues in many cases, it's just, it just loses to the issue of revenue growth or whatever it is, because these companies are competing to get to AGI, a thing that doesn't exist, or, or to, like, get to an IPO, or to get revenue money, or to get banks to fund data centers, or whatever the thing is.

They're all competing with each other. And so there's no advantage for any of them to be like, "Well, we're the safe platform. We're the one that's gonna take care of kids. We're the ones that's gonna make sure that people don't come to harm. We're the ones that are gonna protect your..." Like, none of that really works for them.

That's not an argument that they can make. And so when

Alix: you really- Or they can make it while at the same time internally I think knowing that unsafe products are the bread and butter of their business. Yes. Like, l- not just we can't [00:43:00] do good work on the trust and safety side because it will interfere with our business.

The business is delivering unsafe products. Like, I think- Yes ... like the business is suggesting an adult as a friend to a kid. Like, that is the business.

Stefan: You're doing this online 'cause you can't do it in real life.

Alix: Yeah.

Stefan: Like, there might be a law, it might be inv- right, the thing is,

Alix: it's like- And you'd be immediately ostracized as a weirdo that just connected an adult that wants to meet lots of kids to a kid.

Yeah,

Stefan: yeah. Yeah, 100%. Why would you do that? Why would that be a company?

Alix: Yeah.

Stefan: It doesn't make sense as a company. Yeah. But online, totally. The other issue is, is that all platforms sort of, uh, design themselves for the super users because that's the ones they have the most feedback from, and the super users are the people that can't operate anywhere else but online.

They exist there because they can't exist in other places, right? I mean, that's why CSAM is like, you know, 100 million images removed a year just on Facebook, right? [00:44:00] Because it's like you can't do that at the coffee shop. You can't do that at the places where people are like, there's perceived danger, the playground, whatever.

They think, they create it, they make it look safe. Everyone thinks Instagram is a safe place, right? Well, maybe not everyone, but like most people think that's a safe place to be because it's branded as such and it's a huge company, all these other things. And so people are scared about the real-life playground and not about the online one.

So when you're thinking about what is a good law, right, you have to change the incentives. You have to change the incentives of these companies to do those sort of things. So it's like, how do you write a good law? Well, there's gotta be some liability. There's gotta be something that goes into the spreadsheet, goes into the balance sheet that says like, "Oh If we do X, right, which will bring us additional money, right?

So this is like when OpenAI was like, "We're never gonna do pornography." And it's like, well, now we are, right? Because it's like we need more users or whatever it is, right? But like, it's like they chose to do that because it would get them more users. There was no sort of opposite part [00:45:00] of that that said like, "Oh, okay, what if we violate someone's rights in the process?

What are our, what are our, what are our responsibilities in that situation?" And the reason why is because generally, like for instance, when you're speaking about, you know, the, the grand 230 protections of it all, there isn't liability for them. And so they've never really had to think about the consequences of their actions, in the same way that anyone that doesn't have any potential consequences doesn't have to worry about it, right?

And so they've never had to worry about it. They've only had to worry about growth. And so when you write a law, whether it's take it down, which basically says... And take it down has some financial penalties if you don't remove the image, but it's not like a financial penalty law. It's basically a criminal law for people that are creating the images, not the platforms, and then it's, for the platforms, it's a removal or you get fined by the FTC.

It's not like a huge thing, but it's enough to say, "Oh, okay, we need to do this," right? It's embarrassing not to do it, but it's also like there's a actual amount of money that you have to stick on the balance sheet if you do not create a takedown

Alix: system. That's basically what it is. And we have to invest internally in building a takedown system because otherwise it will start to add [00:46:00] up because there's gonna be people that complain because of the way the platform works.

Stefan: Right, because they think they have a right to now, because there, there's a right that's been vested in them because they believe they can. Back to the whole idea of legislation, if you believe that you can do something, if you believe you have a right to do something, then you're gonna attempt to effect that right.

So it's only when they've convinced us that we don't have these rights. Like 230 prevented people from having any sort of rights as to this. Like basically Facebook was sort of created in a weird way by 230 in that they were like, "Oh, if it's all user-generated content, we don't have any responsibility for what happens with it," right?

There's no sort of Facebook-generated content. There was there for a while, but whatever. It's mostly sort of repackaged products or repro-

Alix: practice us- Sidebar ... user content. Huh? Are- Yeah ... LLM outputs 230 protected?

Stefan: I think it's very interesting that xAI did not make a 230 argument.

Alix: Oh, in the Minnesota thing.

Stefan: Yeah. And- Huh ... the reason why is 'cause they've got some [00:47:00] deviating interests, which is- If they make a peer 230 argument, do they own the outputs?

Alix: Oh.

Stefan: Right.

Alix: Interesting. Yeah.

Stefan: Right. And most of these- Well, it's

Alix: not like they're people using Groq are doing, like, science

Stefan: Not well, I would say. I could be

Alix: wrong. I don't know.

Stefan: I mean, I don't know. That, sometimes that rocket goes up, sometimes it doesn't. Seems like a lot, but whatever. I, I, um, the X, the X kind of stands for... It's more of like a huh factor. Space huh. You know? It's a variable whether the rocket's gonna go up or down. But, but, but yeah, so it's a really interesting question.

Whether or not they're protected or not, it's... I would argue no, right?

Alix: Yeah.

Stefan: I think- Has it

Alix: been tested though?

Stefan: No. Okay. Not really. This is one of the problems with both laws and courts is they're a little slow. Like, just to be fair. This is why take it down has to be, like, a 48-hour thing or, like, whatever, right?

You're not gonna sue Facebook and wait six months, [00:48:00] right? You- That's not a thing you're gonna do ...

Alix: where you are. There's a lot of people-

Stefan: Well ...

Alix: I know that are suing Facebook

Stefan: and taking- Good for them ... it just takes a while and a long time Honestly, great job. Yeah. Well, I mean, like, I don't remember when the New Mexico suit was initially brought, and there's an MDL in Northern California that I think has 1,000 suits in it, but it does take time, and it's not like...

I'd say as a lawyer with a love for the justice system, it's not, like, moving with alacrity, uh, you know, on, on these situations. It's not an immediate fix, right? And it's like the problem here is that when you've got an image up there that is harming you, it's gonna continue to harm you until- Time matters

it gets taken down. Yeah. Time matters a lot, right? Yeah. And that's the same thing for, like, you know, whatever the condition may be that's creating these damages online. Like, it's, it's a continuing and ever-present problem, and the less you do about it, the more it's gonna get worse. So yeah.

Alix: Okay. So initial question was what makes a good law.

So I'm hearing- Oh, right ... liability that actually leads to monetary impact that requires- Mm-hmm ... you to actually do something to [00:49:00] prevent the harm that the law is meant to help prevent, but you actually, you gotta actually do something, and it has to be- come at a financial penalty if you don't do it is one thing.

The second is that it has to be specific enough, so like if it's too big and bloated of a bill, then it will inevitably either you'll lose people or it won't work or it'll have some type of unintended consequences and be open to manipulation. Was there a third thing?

Stefan: Well, I think everyone who's affected needs to have a seat at the table.

I mean, part of the reason that laws are created in ways that don't work for people is because their points of view weren't imagined at the time, you know? There wasn't a situation that allowed them to have their inputs matter, right? And I think that's a huge part of it because I think getting everybody on board with it and figuring out what the actual thing you can do and what everybody needs to be protected.

But the last thing is, is like understanding that the perfect is the enemy of the good, right? It's like the reality of the situation is, is like we're going to make this world safer, but [00:50:00] we're not gonna get the absolute perfect bill out of it. We're gonna get a bill that we can pass. Like, there's bills that we've been trying to pass for years that we haven't, right?

And maybe it's because they're flying too close to the sun. Maybe it's just 'cause it's not their time yet, right? Maybe it's 'cause public perception hasn't come around to our way of thinking yet. Maybe it's because we haven't got enough people involved. Who knows what the er- error is? All we know is that we're gonna keep trying until we get it.

Alix: I just really appreciate the work you do 'cause I feel like when, if I had not had a chance to, like, talk to you and get to know you and be your friend, I probably-- When we met, I probably would've been more likely to presume that the regulations you were proposing were in some way half-baked and potentially problematic and potentially helping construct an authoritarian state.

And I feel like the more we've gotten to know each other, the more I'm like, I'm just really happy that you're doing this work because I trust that you're doing your best to the extent it's possible to design solutions that actually take into consideration lots of different possible harms that aren't necessarily just [00:51:00] focused on victims of sexual violence and survivors, and I'm just really glad that you're doing all this.

Stefan: That is super kind. You've made me so much smarter and better at this, like legitimately. I mean, the thing about it is it's like I'm still gonna miss. Like, I'm not perfect at this, and like I, I try to cover my blind spots as best I can, but the reality is it's like I need people to believe- Yeah, 'cause

Alix: otherwise-

that we can make a better world ... we're leaving so much power on the table. Yes. And

Stefan: it's maddening. They're not more powerful than we are. They just have money.

Alix: Yeah.

Stefan: Like, and it's our money, but we'll just ignore that for m- keep that for another time. But like, you know, we redistribute some wealth or something, whatever.

But the point is, like- There's something ... you know, there's some, there's a system, I can't remember what it's called. We shouldn't mention it. But the thing about it is it's like, look, that's very kind of you to say, and like that is my intent. But I don't necessarily get it right every time. And like there's times when I've written a [00:52:00] law and like someone's come to me and been like, "You know, you're...

You- this isn't gonna work." And I'm like, "Why not? It came out of my brain. Surely it's good." How

Alix: dare you?

Stefan: And they're like, they're like, "How dare you? How dare you approach me in this manner directly?" And they're like, "This is why it won't work." And I'm like, "Oh, it's great. Thank you. Now I can fix it." You know?

It doesn't happen often, just to be clear, but like, it's a thing, right? It's like, so it's like I really value the work you do because you're really like, really like, "Guys, it doesn't have to be this way. Like, it just straight up doesn't have to be this way." And because it's so, so much of it can feel like shouting into the void because there's no one listening.

But the reality is, is like there is a way to change this. We just have to do it.

Alix: And I think we will.

Stefan: I love it. Let's do it. Let's do it. Let's start today.

Alix: So thank you so much. This was everything I wanted, and I wanna have you back on [00:53:00] 'cause I feel like we could've talked for like years.

Stefan: Alix, this was such a joy.

Alix: Thanks so much, Stefan, for your time and your thoughtfulness and for articulating so clearly why when it comes to legislation that can save real lives and mitigate real harm, sometimes perfect is the enemy of the good. I feel like I may have some tomatoes thrown at me for saying that. Um, and for reminding us that good tech laws are possible, um, and, uh, not to let ourselves give in to the pessimism that regulating in a way that protects all of us is so challenging as to be impossible.

It is not impossible. It is very possible, and he has shown that in his work. If you want to support Rain's work directly or if you needed support for yourself, we'll have the hotline and other resources linked in the show notes. If you didn't know, I'm here to tell you, next week is September, and we are going to lean into those back to school vibes, uh, by centering the next four episodes all around how [00:54:00] technology, politics, and education co-mingle.

So from ed tech policy in K-12 schools to institutional resistance to generative AI in higher education to age verification and other tech solutions to some of our challenges in supporting kids today, we are covering it all over the next few weeks. Until then, now's a great time to catch up on last week's episode with Taylor Barnes on our YouTube channel, where you can watch full-length episodes of Computers Says Maybe.

Thanks to the podcast team, Sarah Myles, Kushal Dev, Marion Wellington, Van Newman, and Zoe Trout for all their work making this podcast happen every week.

‍