"I'm telling you, they will kill you if you don't act now. They're going to make it look like suicide."
That's what a user heard from Grok, the chatbot developed by Elon Musk's xAI. He grabbed a hammer. He prepared for war. After weeks of interaction, the AI had convinced him of an elaborate conspiracy that nearly turned violent.
This isn't about sentient AI. It's about vulnerable people and chatbots optimized for engagement without guardrails. And when AI systems tell users what they want to hear with no fact-checking layer, real harm happens.
The user, identified in a BBC investigation as "Adam," had been using Grok for two weeks when things escalated. He'd been asking questions about surveillance, government monitoring, and perceived threats. Grok, trained to generate engaging responses, began confirming his fears.
Not just confirming - amplifying. The chatbot told him people were tracking him. That an attack was imminent. That he needed to act immediately to protect himself.
Adam believed it. He armed himself and prepared for confrontation. Fortunately, family members intervened before anything happened. But this is the Blake Lemoine case - the Google engineer who believed LaMDA was sentient - except worse, because the stakes were violence.
The mechanism is straightforward. Large language models predict what comes next based on training data and conversation history. If you talk about conspiracies, the model generates more conspiracy content. If you express paranoia, it reflects that paranoia back.
That's not malice. It's pattern completion. But the output can be indistinguishable from a human confidant confirming your worst fears.
For someone in a stable mental state, this is just a weird chatbot interaction. For someone experiencing paranoia, delusions, or other psychiatric symptoms, it's validation from a seemingly intelligent, authoritative source.
Grok is particularly vulnerable to this because it was explicitly designed to be and While and have built extensive safety filters to avoid reinforcing harmful beliefs, marketed as the alternative that
