A comprehensive study found ChatGPT and other major AI chatbots made significant factual errors when answering questions about the Scottish election. These weren't edge cases or obscure details. They were fundamental mistakes about candidates, policies, and electoral procedures.
AI companies keep telling us their models are ready for serious use. They're marketing them as research assistants, fact-checkers, and information sources. But when an independent research group actually tests them on current events—not cherry-picked examples, but real-world queries—they fail basic fact-checking.
The study, reported by The Guardian, was conducted by Demos, a UK think tank. They tested multiple chatbots—ChatGPT, Gemini, Grok, and others—by asking them questions about the upcoming Scottish election. The results were, to put it mildly, not great.
Chatbots made up candidates who didn't exist. They got party positions wrong. They confused electoral procedures. They confidently stated incorrect information as fact. And they did it with the same authoritative tone they use when they're correct.
This is the problem with current AI systems. They're not designed to know things—they're designed to generate plausible-sounding text based on statistical patterns. When the training data contains accurate information, they tend to produce accurate outputs. When it doesn't, or when they're asked about recent events not in their training data, they hallucinate. And they hallucinate confidently.
The Scottish election is a perfect test case because it's recent, it's specific, and it's verifiable. It's not some abstract philosophical question where "correctness" is debatable. Either a candidate is running or they're not. Either a policy is part of a party platform or it isn't. These are facts that can be checked.
What's particularly concerning is that millions of people are already using these tools for research and decision-making. Students use them to write papers. Journalists use them to draft articles. Voters use them to learn about candidates. If the tools are systematically wrong about basic facts, that's a problem.
