Online Harassment in the Age of AI, Algorithms, and Viral Misinformation
It’s no exaggeration to say that the internet has changed the world. It’s revolutionized how we work, how we play, how we communicate with each other, and how we stay informed. And it’s also changed how we get manipulated. Social media algorithms don’t care if they show us things that are false or dangerous as long as they keep us scrolling. Scammers use AI to personalize their hooks, craft realistic messages, and change their voices and faces in real time. Misinformation and disinformation can easily go viral and lead to targeted online harassment campaigns. It’s never been more important to understand how these systems work and take steps to protect yourself.
See The Internet is Playing You with Nina Jankowicz for a complete transcript of the Easy Prey podcast episode.
Nina Jankowicz is an internationally-recognized expert on disinformation, online harassment, and AI-generated harms. TIME magazine named her one of the 100 Most Influential People in AI. She describes her work broadly as studying the bad things that happen on the internet. She is also the author of How to Lose the Information War, which follows the governments of five Western countries as they respond to Russian disinformation, and How to Be a Woman Online, which is a guidebook for mitigating the damage of online abuse and harassment.
Disinformation, Misinformation, and Online Harassment
“Disinformation” and “misinformation” are often used interchangeably. But there is a distinction, and a lot of media still gets it wrong. The difference is in the intent. Disinformation is false or misleading information spread with malicious intent. That includes nation-states like Russia or China, political campaigns, people selling fake or shoddy products, or people trying to stir up an online harassment campaign against someone else. Misinformation still spreads false or misleading information, but it doesn’t have that intent.
Nina describes misinformation to people as Aunt Sally at Thanksgiving dinner trying to convince you the moon is made out of Styrofoam. It’s definitely a conspiracy theory and not true. But she’s not trying to convince you because she wants to hurt you, affect who’s in power, or make money. She’s doing it because she believes it, thinks it’s interesting, or thinks people should investigate it more.
Disinformation and misinformation are not interchangeable, and understanding the difference is important to internet safety. Nina doesn’t think we need to fight misinformation. The people spreading it aren’t malicious. We need to get of the source of this false information and fight disinformation and the people putting it out.
When Disinformation Becomes Misinformation
Most of the disinformation we encounter on a regular basis is stuff that gets us worked up on social media. That’s the tip that Nina gives people. If you see something online and it makes you emotional, pause and think it through first. It might not be true. That’s where disinformation tends to cross the line into misinformation. It gets in front of people who have strong feelings about it and who then share it, often unintentionally distributing misinformation.
If you’re getting emotional about something you see online, take five seconds, think it through. You might be being manipulated.
Nina Jankowicz
Some people get concerned that even if fighting disinformation would be beneficial for internet safety, it would involve censorship or infringe on freedom of speech. They worry that everyone who shares the article about the moon being made of Styrofoam would be punished. And that’s not what any of the experts want.
Platforms’ algorithms reward emotional engagement. It doesn’t matter to the algorithm if the post is false, misleading, or online harassment – if it gets people to engage, the algorithm will show it to more people. Nina would like to see some controls on that. Nobody wants to see Aunt Sally get in trouble for being convinced the moon is made of Styrofoam. The better solution would be for platforms to keep some of these stories from going viral so easily, the originators of disinformation to face consequences like account suspension, and people like Aunt Sally get a bit more education about how today’s online environment can manipulate them.
Disinformation and Online Harassment
Nina sees online harassment and disinformation campaigns as two sides of the same coin. Where there’s one, there’s almost always the other. If someone is the target of a disinformation campaign, harassment tends to come with that, especially if that person is a woman or a person of color. Data shows that women of color or women with other marginalized identities are more likely to be targeted with online harassment or threats than white women or women without those identities.
Where there is online harassment, frequently there’s disinformation … where there’s disinformation campaigns, there’s almost always online harassment.
Nina Jankowicz
No matter where disinformation is coming from, the most memorable and persistent kind has a “main character” – someone who pulls the story together and gives it life. Sometimes that person is a political figure. Sometimes it’s an unwitting person roped into this whole thing. One example is Ray Epps, who was at the January 6 insurrection. He has since repented, but Tucker Carlson claimed he was a federal agent, which put him at the center of a lot of conspiracy theories and ended with him having to leave his home.
It’s one thing to talk about disinformation and misinformation in a general, abstract way. But the spread of false and misleading stories has real, material impacts on real people’s lives.
Telling Information from Disinformation
A key skill for using the internet is being able to tell the difference between misinformation, disinformation, and just plain information. The idea of content that makes you feel a strong emotion is still a good one here. It doesn’t even have to be a negative emotion – there’s a reason animal videos are so popular. But if you’re getting angry, anxious, scared, excited, or anything else while looking at something online, it’s time to take a step back. Especially when it’s some form of breaking news where there’s not a lot of facts yet, don’t react immediately.
Nina encourages the strategy of “lateral reading” – look at different news outlets or content creators covering the same content. Getting outside the things you usually read will not only help you get a fuller picture of the actual situation, but will give you clues into how each outlet or person posting wants you to see the situation.
Visual media can be much more challenging. AI experts agree that AI-generated content is so good that we can’t really trust our own eyes and ears anymore. Nobody should put pressure on themselves to know definitively. We need to slow down, wait for news outlets to fact-check, and see what deepfake experts say. Because it’s past the point where the average person can tell. Until you’re able to verify, slowing down is the best thing you can do.
No layman should put that pressure on themselves. What we need to do is slow down and … wait for the [experts] to say whether this is real or not, because it’s past the point where we can know.
Nina Jankowicz
Slowing Down Isn’t Easy
It’s all well and good to say to slow down and verify first. But when we’re emotional, that’s not what we’re wired to do. And platforms don’t want us to verify – they want us to get emotional, react, and keep scrolling. One of the things that makes internet safety so difficult is that platforms profit from disinformation, online harassment, and scams, so it’s actually in their best interest to promote these things.
The United States especially has not regulated online space. Many other countries have laws to stop platforms from monetizing our attention, but despite radicalization, disenfranchisement, and even suicides, the U.S. hasn’t done anything to make platforms even slow down harmful things. Reuters last year did a story showing that Facebook made billions of dollars and fraud and scams. They have the ability to identify these things on the back end, but they don’t stop them, because it’s profitable.
Online harassment is also profitable for platforms. Because they monetize attention, the more people are engaging on the platform, the better. If people are harassing someone online, that’s a lot of engagement. If the target responds, that benefits them, too. To platforms, there’s no downside to harassment.
One of the most interesting things about the mechanics of online abuse is that it’s so good for the platforms.
Nina Jankowicz
How to Respond to Online Harassment
Everyone hopes that they’re not going to be a target of online harassment. But a key part of internet safety is being prepared for the worst case scenario. Nina talks about strategies to deal with harassment in her book How to Be a Woman Online – which, despite its title, is useful for anyone engaging in potentially-controversial discourse on the internet.
Her biggest tip is to not engage. If someone is not engaging in good faith, Nina mutes them and moves on. If they’re saying something especially horrible, everyone should be able to use the block button early and often. Many people have a feeling that social media profiles or feeds should be some kind of utopia, but in reality, they’re more like the Wild West. There are no rules and no consequences. You shouldn’t force yourself to endure harassment out of some utopian ideal, because that isn’t the reality.
You should not force yourself to endure [harassment] just out of some kind of democratic ideal that doesn’t exist.
Nina Jankowicz
If you do want to try to engage with people and get them to see your side, Nina gives people one reply. She says something polite and measured and sees how they respond. If they double down, she mutes and moves on. Sometimes she has engaged for more than one reply, but she’s regretted it every time. And she loves the idea of them posting into the void and never getting a response. Sometimes she will also take a screenshot, redact the username, and share it. A lot of people don’t know that harassment happens or the extent of it, so that’s a way she can educate people without being responsible for more harassment. Some people think that harassers deserve to be harassed back. But having been a victim of online harassment, Nina doesn’t think anyone deserves that.
The Response is Inadequate
Nina’s friend Van Batem, an Austrailian columnist for The Guardian, likes to remind people that the problem of online harassment is something to be managed, not solved. Until the platforms give us better tools, we won’t be solving it any time soon. In the meantime, use the tools that you do have. Mute and block people. Some platforms, like Instagram and TikTok, let you mute keywords. Bluesky has a “nuclear block” where you can hide a reply from the rest of your followers, too. These tools are key to preserving your sanity online.
The sad truth is that not only are platforms not currently equipped to deal with online harassment (and are increasingly relying on AI, which is also woefully inadequate for identifying threats), law enforcement also aren’t equipped to deal with it. Nina can say that from experience. There are also tons of reports of people who have gone to law enforcement with reports of something that happened online and been laughed out of the room. One victim told a sociologist, “It was like I was drowning and law enforcement didn’t know what water was.”
It’s a bleak picture. But when you’re drowning, you have to use whatever life vest is available. Don’t worry about hurting someone’s feelings by muting or blocking them. Take whatever steps you need to make the platform usable for you. Harassment and abuse online is making a lot of people self-select out of online discourse. And that’s not good for anybody.
Effective Response Requires Systemic Change
If we want law enforcement to be better equipped to deal with online harassment, there will need to be systemic changes. California has some of the better laws in the United States for people experiencing different forms of online threats and harassments. But Nina has still heard stories about the ways law enforcement and judges are still ill-equipped. It’s going to take training and investment to bring cops, detectives, and judges up to speed.
One of the lawyers Nina works with said that the law in the U.S. is inherently looking backwards. And this can be a challenge in areas where tech is changing rapidly. If you’re trying to apply a telecommunications law from the 1980s to cyberstalking today, it’s not going to fit very well.
A lot of the work Nina does is trying to fill in those gaps and encourage that systemic change. She encourages legislators to invest in training and in acquiring talent that understands the internet. Cops do a lot of great work. But not everybody is cut out for cyber investigations. It’s a very different specialty than standard investigations and requires different knowledge and skills. Getting that talent on police forces across the country is something governments need to invest in.
Collecting Evidence and Getting Support
If you’re experiencing online harassment or abuse, it’s crucial to preserve evidence. It can be difficult for victims to not only have to see the messages, but screenshot them and save them somewhere. There are also software programs that can take and catalog these screenshots, too. Not all of them collect the necessary metadata and other information required to be admissible in court, though. So if you’re experiencing a serious level of abuse, that’s something to consider.
A lot of people experiencing online harassment suffer alone and in silence. Even Nina made that mistake early on. But there are places you can go for help. If it’s happening to you because of your work, Nina recommends asking your employer for support. Even before her own experience, Nina had worked with employers to put policies in place for employees.
A lot of people make a mistake of suffering in silence and suffering alone when they’re subject to [online harassment].
Nina Jankowicz
Ironically, most organizations have policies for what you can and can’t say online, because they view you as always repping the company in some way. But when things go wrong and someone starts harassing you online, they don’t do anything about it. That’s something Nina is trying to change, too. Putting an employee in a hotel for a few weeks if they get doxxed is a big investment for a company. But it’s really just a reality of doing business online these days.
Mitigating the Impact of Online Harassment
If you want to take steps to manage trolls, mitigate doxxing, and minimize the impact of online harassment, Nina’s book How to Be a Woman Online is a good place to start. She also has a Substack, The Wayfinder, that includes an online safety tuneup. It’s basic steps that you can take so that if the worst happens, you’re one step ahead. They’re all broken down into easy, manageable steps that you can tackle over the course of days or weeks. These steps aren’t just a nice-to-have anymore. If the worst happens, doing them will put you in a less-bad situation.
These steps include simple things like having a complex password and using a password manager. When Nina tells this to audiences, they groan. But then she asks how many of them still have their passwords in an unprotected note on their phone, or use the same two or three passwords for everything. Turning on two-factor authentication is also important. These steps may be boring and basic, but they’re crucial.
Another thing people don’t think about as much is how much about their location they might be sharing accidentally. When you’re posting a photo of your kid or puppy online, you’re probably giving away more than you think. If you put your address on your puppy’s tag, that could be visible. If your child was at your local park, that could help someone determine the rough vicinity of your house – especially if you included a caption about walking or biking there. When you’re putting yourself out there online, it’s important to think through these things.
Making it Less Bad
Nina created this toolkit because of her own experience. She was doxxed and had a nationwide online harassment campaign against her. She hired a security consultant to help her deal with it. The Russian government had been trying to get into Nina’s email for over a decade at that point, so she was already taking a lot of security steps. The consultant told her that if she hadn’t already been doing that, the consequences would have been much worse. Her private photos and documents would have already been leaked.
It doesn’t take long to follow these steps and protect yourself. The toolkit also helps with preventing doxxing. Nina personally would prefer to throw money at the problem than do the work herself, so she pays for two separate tools to keep herself and her family off the internet. She has a toddler and a puppy, she doesn’t have time. But some people would rather do the work than pay somebody, so she includes those steps, too.
Keeping yourself safe shouldn’t be scary. It can sound intimidating, but Nina breaks it down. It’s just a couple of hours of work, and it’s easy to do in just a little bit of time each day or week. People think that you have to do everything at once, but you don’t. Just do what you can today. Tomorrow or next week, come back and do the next thing. That’s the approach Nina takes. Doing it slowly is better than not doing it at all.
Find Nina Jankowicz on her Substack, The Wayfinder, at thewayfinder.net. She is also on LinkedIn, Bluesky, and Instagram. Find her books, How to Be a Woman Online and How to Lose the Information War, wherever books are sold.