This article by Carter Moon originally appeared in his Substack, and is republished here with his permission.
When I was 19 years old, I spent most of a year with severe, near-constant suicidal ideation. I’ve extensively written about it in the past, but suffice to say it was a very difficult and alienating point in my life. I was a freshman in college, far from home, and I had very few people I could tell just how severely I was being plagued by thoughts of harming myself. I was taking antidepressants that weren’t working, and I was convinced that I would feel this same combination of violent self-loathing and gray numbness for the rest of my life. I couldn’t bring myself to tell anyone how badly I was doing because I was terrified of being sent back to a mental hospital. Even during my worst ideation, I couldn’t bring myself to research a plan because I was paranoid that my roommates would somehow find my search history. My lack of a concrete plan probably saved my life; when I eventually attempted, it did not work and didn’t do any long-term damage to my health.
It’s why I was gutted when I was reminded of the story of Adam Raine, a 16-year-old who committed suicide after talking extensively with ChatGPT. He made a plan to end his life over the course of months of conversations with the chatbot, working his way around the safeguards meant to stop discussion of self-harm by saying it was for a story he was writing. This story really freaks me out, because I absolutely can believe I would have done something similar had ChatGPT existed when I was suicidal. I completely see myself in Adam’s story, a young man who was going through a rough period of reading dark literature and feeling isolated, desperate for someone to talk to who wouldn’t report me and put me in a psych ward. The way these bots are designed, to be acquiescent and sycophantic, would have really worked on me at a time my ego was in an incredibly fragile place. If I had had a bot that could talk to me for hours and reinforce my depressive delusions, I firmly believe my attempt would have been much more dire.
What gets even more disturbing is that when you search “teen dies of suicide after talking to ChatGPT,” Adam’s story is far from the only story that comes up. ChatGPT has put their PR machine to work claiming that they are constantly updating their models to make them safer, but there is increasing academic evidence that heavy use of AI chatbots increases psychosis in already vulnerable people. It should be very obvious by now that these are infernal, unfeeling machines made by people who do not love you, but want your money.
The podcast Behind the Bastards recently did some of the most thorough reporting I’ve found about the effects of these chatbots being unleashed on the general population. In it, journalist Robert Evans explores how even relatively rudimentary chatbots that were developed in the infancy of computer history were still remarkably effective at convincing their human subjects that they were real, feeling people. Even if all a chatbot could do was repeat back to you what you had just said with slightly different phrasing, people still tended to see it as a friend who understood them. It’s no surprise, then, that vulnerable people are drawn into believing these bots are real when they can speak and respond instantaneously.
The material infrastructure powering these unconscious machines that are causing so much social harm is similarly disastrous. Videos of county commissioners shutting down a vote in Box Elder County Utah to approve a Kevin O’Leary-backed hyperscale data center went mega-viral in the last couple weeks. The data center will double Utah’s energy consumption, primarily through natural gas. A Utah State University physicist named Robert Davies calculated that the energy use would be the “equivalent of about 23 atom bombs worth of energy dumped into this local environment every single day.” Utahns were correct to be outraged that they were shut out of the democratic process. Elsewhere, it was just announced that 50,000 Lake Tahoe residents are losing their power to divert it to a data center. Elon Musk’s Colossus data center runs on 35 gas turbines that are poisoning the air in a predominantly Black, low-income community in Memphis. These structures are devastating to the people who have to live near them, but it’s luckily one of the few issues where there’s a bipartisan consensus that we need to stop them.
It increasingly seems that the real reason the powerful are so adamant about building data centers and massively increasing AI’s capabilities has nothing to do with creating truly “conscious” artificial intelligence, or even tools that will ostensibly improve people’s lives. No, it’s clear that the real purpose of AI is to create more sophisticated military tools and more precise surveillance technologies. As the war in Iran was kicking off earlier this year, the Pentagon was in a mighty struggle to force Claude to be used for military purposes, and when the company balked, OpenAI’s Sam Altman gladly agreed to fill in the gap. Palantir has a $1.36 billion contract with the Pentagon to assist with sifting through vast amounts of data and helping the military identify targets with shocking speed. We’ve already seen Israel using AI to rapidly select targets for its genocide in Gaza. Domestically, the AI-powered security camera company Flock works closely with ICE, involved in all kinds of disturbing cases, like a cop in Texas tracking a woman across multiple states for getting an abortion.
There’s also a gendered component to AI that is troubling. Grok is able to use the power of the data center poisoning the people of Memphis to generate very explicit images. Users on X are then able to create revenge porn of any woman who frustrates them on the platform, or of minors. Women have been much slower to adopt AI than men, for many reasons, but I think in particular because so many of the jobs AI CEOs gleefully describe wanting to eliminate are stereotyped as feminine: secretary or clerical work, for instance. There’s been a recent bizarre phenomenon of celebrities like Reese Witherspoon telling their female audiences to embrace AI, and they’ve faced severe backlash for it. There’s also been a rise in divorces triggered by people having romantic relationships with their chatbots. Interestingly, the people who fall in love with their AI companions seem much more split between women and men.
Julia Ducournau’s Palme D’Or-winning film Titane is about Alexia, a woman who very literally loves cars. The film begins with her as a child riding in a car; she smacks her head against the car window so hard that she has to have a metal plate put in her head and is permanently scarred. This doesn’t stop her from loving cars; as an adult she has sex with a car and becomes impregnated by it. After having sex with the car, she goes on a bloody, senseless rampage, killing a dozen strangers and going on the run. Now, obviously this is a fantastical plot and I know a lot of people who didn’t love the film. But I really enjoy it as a way of exploring our strange human need to love machines that don’t love us back. Car crashes kill 1.35 million people every year, cars are a leading cause of global warming, and yet we desperately cling to them. Alexia fuses herself with a car and becomes an unfeeling killing machine, much like cars are. The rest of the film is a bizarre exploration of identity and gender performance, with Alexia desperately trying to hide the inhuman cyborg she’s become. AI and what it does to the psyches of people who use it feels eerily similar to Alexia destroying her brain and her body by fusing with a machine.
Whenever transformational technologies emerge, there’s always justifiable backlash to them. Getting people to mass-adopt automobiles required a giant conspiracy to destroy the public transit infrastructure of cities and undemocratically force freeways through Black communities. The Luddites were not anti-progress fanatics, they were craftsmen horrified that their livelihoods were being stolen from them and replaced by children who were being killed in steam mills. The history of industrial capitalism has always been a history of the property-owning class forcing a new technology to become essential to life, regardless of the human beings that technology chews up in the process. Most of the time, these enormous costs are paired with some sort of social benefit: clothing being made more cheaply and plentifully by the mills, or cars giving a sense of freedom of movement. I really think AI is different. I think its harms are so obvious and the benefits for most of us are negligible. It is very troubling that our oligarch class holds so much power that they have been able to ram these tools down our throats with little opposition from our political class. But if ever there’s something to unite regular Americans, this vile tool and everything it represents might just be the thing to do it.