The Hidden Downsides of Predicting the Future
Humans have always wanted to know what the future has in store. And with AI and machine learning exploding recently, algorithmic prediction has never been easier. But making these predictions doesn’t just forecast the future – it shapes it. As algorithms continue to play bigger roles in decisions, the line between prediction and influence keeps getting blurrier. It’s essential to understand what’s really going on when we try to predict the future.
See The Power of Prediction with Carissa Véliz for a complete transcript of the Easy Prey podcast episode.
Carissa Véliz is a philosopher, associate professor at Oxford where she teaches philosophy and ethics, and researcher at the Oxford Internet Institute. Her work focuses on the ethics of technology, privacy, and AI, and on the societal impact of digital surveillance. She’s also the author of the books Privacy is Power and recently Prophecy: Prediction, Power, and the Fight for the Future, from Ancient Oracles to AI.
Carissa has always been an unusual philosopher because she cares a lot about the practical applications. She advises companies around the world, from small startups to Fortune 500s, as well as governments an policymaking. Her family is in business, so she’s thought a lot about how businesses use data and prediction tools, probably more than the average philosopher.
The History of Prophecy and Prediction
Humans have been using prediction as long as we’ve been humans. In some ways, it’s what gives us our competitive advantage. If you can predict animal behavior, it’s easier to hunt them; if you can predict the location of the stars, you can use them as a map or clock; if you can predict next season’s weather, it’s easier to farm. But there’s a fine line between reasonable prediction – guessing what the future holds based on the past and your confidence that the future will resemble the past – and the mystical or supernatural.
Humans also have a lot of anxiety about wanting to know what the future holds. That deep desire to know the future makes us vulnerable to charlatans. Prophecy has always been a big business. Prophets are just merchants of prediction.
Our anxiety to know what the future holds is so prominent that it makes us very vulnerable to charlatans of all kinds.
Carissa Véliz
One of the most important figures in the history of prediction is the Oracle of Delphi. Ancient Greece was obsessed with divination. Oracles and seers became astrologers in medieval times. Astrologers became social scientists in the nineteenth century. There, they became part of public policy and introduced prediction as a way to make decisions. For a short time, they became economists. Now the role of prophet is filled by computer science and data analysts and tech. Oracles have always been in a position of power because they have secret knowledge to provide certainty about the future.
From Logical Conclusion to Make-Believe
There is a wide range in predictions, from reasonable assumption to wild conjecture. And there are two points where it tends to cross the line from forecasting to fantasy. The first is trying to predict the far future. The farther out you go, the less trustworthy the prediction is.
The further away in time the prediction, the more untrustworthy it is.
Carissa Véliz
The second line is whether you’re predicting natural phenomena or the social world. That’s because predictions can influence the social world. If you predict the weather, the weather isn’t influenced by what you say. If it’s going to rain anyway, it doesn’t matter what you say about it. But predictions about people influence the expectations of those people and those around them. They tend to become self-fulfilling prophecy. If Carissa wrote a letter of recommendation for a student saying she didn’t think they’d get a job in academia, they probably won’t – not necessarily because they couldn’t have, but because Carissa’s prediction influenced interviewers. Because she predicted it, it came true.
Every time you hear a prediction about social realities, you should be skeptical. If a tech executive says we’ll be using AI for everything tomorrow, that’s predicting a social future that lines their pockets. In philosophy terms, that’s a “speech act” – a sentence that creates reality. A priest pronouncing someone man and wife isn’t describing the world but making something happen. When someone makes a perdition about the social world, they’re not describing the future. They’re issuing a command telling you to create a version of the world they want. And when you believe that prediction, you’re essentially obeying.
When somebody makes a prediction about the social world, what they’re actually doing is saying, ‘Go out there, fulfill my vision of the world.’
Carissa Véliz
Predictions and Power
The difference between predictions about things like the weather and social realities is whether or not they’re about people. Clouds don’t have any say in their future. But humans have agency and can make changes. When you hear someone say something like this, whether it’s an algorithmic prediction or something less scientific-sounding, you first have to ask yourself whether it’s a prediction or a description.
Sometimes, it’s not straightforward to identify the difference. If you’re not sure, look at who’s making it and why. Are they looking to get knowledge, power, or money? If the prediction is true, who benefits? What would need to happen for it to come true? Answering these questions can help you figure out where it falls.
The general argument of Carissa’s book is that we’ve been overall naïve about predictions in the recent past, and also in history. We tend to interpret them as quests for knowledge or hypotheses about the world. But most often, predictions are power plays in disguise. When you listen to a prediction and examine it instead of believing and obeying, it’s an opportunity to push back.
We tend to interpret predictions as quests for knowledge or hypotheses about the world when more often than not, they’re power plays in disguise.
Carissa Véliz
The Supernatural and the Scientific
In the history of predictions, we have switched over from supernatural predictions done by priests and mystics to more scientific and algorithmic predictions done by academics and scientists. This makes us more likely to trust them. But we’re wrong.
That’s part of the argument of Carissa’s book Prophecy. We tend to think about things like AI as cutting-edge tech. But if you were to ask an ancient Greek what the cutting-edge decision-making method was, they’d say the Oracle of Delphi. It’s not actually that different. Astrology was extremely technical, and some advances there led to advances in astronomy. Astronomy was actually the first scientific discipline to get close to something like big data because of all the math and star measurements that went into astrology.
And on the other side of the coin, there are some trends with AI that are very superstitious. God works in mysterious ways, and for most people so do algorithms. Some people are even arguing that god works through algorithms now. Some groups in Silicon Valley that are obsessed with longevity or general AI resemble religious sects from a sociological perspective. Carissa questions whether the divide between science and mysticism is really justified.
Another important thing to note is that there is still doubt about whether AI is actually improving science or making it worse. More and more academics are using AI to come up with papers. But AI makes important mistakes that can be hard to recognize. And AI is designed to support our own desires and views because people like to be validated. But science wants to pursue truth, which means knowing when we’re wrong. AI doesn’t want to tell us we’re wrong. So the jury is still out about whether it’s an improvement for science.
The Risks of Overreliance on AI
Very few people understand what kind of algorithmic predictions are going on behind an AI. Some argue that that’s not a bad thing – after all, the majority of people don’t understand how a car engine works, either, but we can still drive. But car engines aren’t made to be misguiding or an impersonation like an AI chatbot is. It’s not just about understanding, but the product being designed to seem like something it’s not.
Most industries have regulations. The auto industry has regulations to make sure your car engine isn’t going to explode or poison you. But the AI industry doesn’t have that yet. The burden is still on the individuals to figure out what’s safe and what isn’t.
In many ways, people are also misdiagnosing the problem. If you don’t have a good diagnosis, it’s harder to get to solutions. We’ve heard a lot about AI bias, and that’s a huge problem. But we haven’t had any conversations about why we think it’s okay to let anyone, including a chatbot, make algorithmic predictions about humans and act in accordance with those without asking the person’s input or even informing them.
Even ancient Rome regulated their prophecies. It was illegal to predict the emperor’s death, for example, because those were often self-fulfilling. Thousands of years later, we haven’t caught up with those lessons. We’re using prediction and more and more in areas that are supposed to be about facts, like the justice system. And predictions aren’t facts – at best, they’re just educated guesses.
Predictions are never facts. At best, they’re educated guesses, but often not even that.
Carissa Véliz
When Predictions Make Things Worse
In her book, Carissa wrote about insurance, actuarial tables, and where those came from. Insurance was traditionally about pooling risk. We had statistics about roughly how many people in a million would get such-and-such disease and what that meant for life expectancy. We could tell how many people would be unlucky in that million, but not which people specifically that would be. Nobody knew whether they would be lucky or unlucky. Insurance let them benefit from the law of large numbers in statistics by distributing the risk. That’s a benefit, because if you’re unlucky and uninsured, it will crush you.
In practice, we’re seeing a concerning trend in insurance. They’re going from high-level large population data to algorithmic predictions about individual humans. The company has data on what you buy, how much you earn, where you live, how fast you drive, and even your genes, and use that to make predictions about your individual life expectancy. That prediction affects your premiums. It’s hugely different because it’s now pushing risk on individuals instead of pooling it, which is the whole point of insurance. Unlucky people have to pay their way, and if your insurance company thinks you have bad genes, there’s nothing you can do about it.
That’s not just bad for unlucky individuals, it’s bad for society, because society depends on the robustness of individuals. In the 2008 financial crisis, banks gave risky loans to people they knew wouldn’t be able to pay them. When too many people shoulder risk they can’t handle, they break, and society breaks with them. Not long ago, a UK insurance executive said he was worried AI was going to make some people uninsurable. Once they’re deemed uninsurable, they end up with more risk because they don’t have insurance, and it’s a vicious cycle.
Self-Fulfilling Prophecies are the Perfect Crime
The dangers of self-fulfilling prophecies are evident in medicine. If someone is deemed beyond all help in triage, they’ll most likely die. These kinds of decisions are the perfect crime because they erase all evidence. We’ll never have the data for what would have happened if they’d gotten care. That points towards the fallacy of thinking anything can be data driven.
If you’re making algorithmic predictions of whether or not people are going to be employed, you don’t get data on people who don’t get jobs. And more and more, everyone is using the same algorithm. If you don’t get one job, you’re probably not going to get any. And the algorithm might not give you a job not because you don’t deserve one, but because you’re unusual in some way. Algorithmic decisions are great on a normal curve. If you fall out of that curve – whether it’s because you’re worse or because you’re exceptional – you’re going to get overlooked. And the world will never know what it missed.
The algorithm tends to be good at identifying patterns in the normal curve. And if you fall out of the normal curve, either because you’re worse than the normal curve or because you’re extraordinary in some way … then we might be discriminating against you.
Carissa Véliz
When we’re thinking about how we want to build society to better thrive, we should let people defy the odds. The biggest heroes in history are people who have been extraordinary. But if you have a system that makes algorithmic predictions about them even before they have a chance, we’re just narrowing human agency. And we’ll end up not just shackling extraordinary people, but holding back society, because these could be the people who solve humanity’s greatest problems.
Don’t Get Caught in the Prediction Wheel
Carissa talks a lot in her book about making decisions that let you not get caught in the algorithmic prediction wheel. Part of it has to do with not resisting uncertainty. If you look at how uncertain the world is, it’s natural to be anxious. Carissa feels it too. That anxiety pushes us to ask someone to predict the future. And nobody knows.
Instead of the knee-jerk reaction to reduce uncertainty, we should reflect on the fact that it’s good we don’t know what the future holds. It’s only when we aren’t certain of the future that we can have democracy. The future isn’t a script to discover. Instead, we can ask what future we’re going to build, what we want to see, and how we can get there.
Instead of wanting to know what the future holds, as if it was a script to discover, we should ask ourselves, what future are we going to build?
Carissa Véliz
To do that, we have to not give too much credence to prophets. The next time you hear someone say this is inevitable or this is what the future looks like, remind yourself that it’s just one future. Is it the one you want? If not, what are you going to do to make sure we choose a different one? It’s about being critical of predictions, and also having conversations about what predictions we’re making, what we shouldn’t try to predict, and where those lines lie. Avoiding getting caught in the prediction wheel isn’t about one particular thing – it’s a view of life.
The Benefits of Uncertainty
We can also pay attention to the opportunities and benefits of uncertainty. Making the most of it increases your exposure to serendipity. Think about the best things that have happened in your life – they were probably pretty hard to predict. That has lots of applications. Carissa loves going to bookshops instead of shopping online because she comes across books she would have never read otherwise.
One of Carissa’s favorite cities is Madrid. It’s a very friendly city and it’s normal to stop someone on the street and ask for directions. She often takes the opportunity to do that instead of looking at maps on her phone. And people often tell her not just directions, but things she wouldn’t have thought to ask. They’ll tell her how to get there, but also that she should try a particular coffee shop because it’s the best in the neighborhood, or something similar. She meets people and has conversations she wouldn’t have otherwise.
We shouldn’t allow algorithmic prediction to determine who we meet, what we do, where we go, or what we see. And we shouldn’t substitute algorithms for relationships, either. The world is incredibly rich, and you have more power and agency than you possibly imagined. Most of us spend so much time dazzled by the digital, but we forget the wonders of the analog world around us. It’s time to recapture that and start thinking about what a good life really is.
We shouldn’t allow predictive algorithms to determine who we meet, what we do, where we end up, what we see.
Carissa Véliz
Connect with Carissa Véliz on Blueksy @carissaveliz.bsky.social, on LinkedIn, or on her website carissaveliz.com. Find her books Privacy is Power and Prophecy wherever books are sold.
Related Articles
- All
- Easy Prey Podcast
- General Tech Topics, News & Emerging Trends
- Home Computing to Boost Online Performance & Security
- IP Addresses
- Networking Basics: Learn How Networks Work
- Online Privacy Topics to Stay Safe in a Risky World
- Online Safety
- Uncategorized
Scams Targeting Seniors Aren’t Going Away Soon.
So, what makes scammers an appealing target group for cyber criminals? Here’s what the consensus on that...
[Read More]
What Art Fraud can Teach Us About Art, Fraud, and Buying Smart
In movies and on TV, the world of art fraud, forgeries, and theft is glamorous and exciting….
[Read More]
Call the Local News for Fraud Help. Scam Victims Turn to an Alternative Resource for Results.
It’s frustrating for victims of scams and fraud who have taken what they believe are the right...
[Read More]
Identity Theft Resource Center: Support, Education, and More
There are plenty of resources available to consumers to report crimes, scams, and identity theft. However, not...
[Read More]
The Hidden Downsides of Predicting the Future
Humans have always wanted to know what the future has in store. And with AI and machine…
[Read More]
Digital Privacy Protection for the Real World
A lot of advice for staying safe online sounds great on paper, but is hard to follow…
[Read More]