A comprehensive R30,000 research study has revealed that artificial intelligence search engines systematically fail to provide accurate South African context when answering queries about local brands, services, and institutions—raising urgent questions about digital sovereignty as millions of South Africans quietly replace Google with ChatGPT, Claude, and Gemini for everyday decisions.
The study, conducted by a South African full-stack developer and AI researcher, tested three major AI platforms across 16,500 queries—making it the most rigorous examination of AI search performance in an African context to date.
Unlike casual testing, the researcher pre-registered the methodology on OSF, an academic registry that timestamps research plans before data collection, preventing cherry-picked results. "No commercial AEO research firm in this country has done this," the researcher noted in findings posted to r/southafrica.
The numbers are striking: 1,100 unique questions asked five times each to ChatGPT, Claude, and Gemini, with full open datasets published for public scrutiny.
Different AIs, different South Africas
The study found that the three major AI platforms don't agree with each other on basic facts about South African brands. When asked "which is the best medical aid for a family in South Africa," ChatGPT, Claude, and Gemini returned different lists, different top picks, and different cited sources.
"This isn't AI being thoughtfully nuanced," the researcher explained. "Each AI has a different idea of what South Africa even looks like."
More troubling: the same AI doesn't give consistent answers when asked the same question multiple times. Gemini's cited sources changed about 65% of the time between identical queries. Claude showed 35% variance. ChatGPT fell in the middle.
"Same model, same question, just asked again. Different brand lists. Different 'best' picks," the study documented. "AI search isn't a stable thing. It's basically rolling dice with confident sentences attached."
For South Africans making real decisions about medical aid, banking, or business services based on AI recommendations, this inconsistency means they're receiving fundamentally unreliable guidance presented with false confidence.
HelloPeter ignored, American platforms prioritized
When South Africans ask AI about brand complaints, the platforms systematically underutilize HelloPeter—South Africa's purpose-built consumer review platform with twenty years of local complaint data—in favor of American alternatives.
"We built HelloPeter for this exact purpose," the researcher noted. Yet when queries are framed negatively, AI platforms shift toward Trustpilot, Complaintsboard, and PissedConsumer—review sites built for British and American consumers with minimal South African participation.
The pattern reveals a troubling dynamic: "Our positive coverage is shaped by SA media. Our negative reputation is increasingly being told by foreigners," the study concluded.
In South Africa, as across post-conflict societies, the journey from apartheid to true equality requires generations—and constant vigilance. The digital realm is no exception. When AI platforms systematically prioritize foreign sources over local institutions, they replicate colonial patterns in technological form.
Industry-specific blind spots
The study revealed dramatic variations in how AI platforms treat different South African industries. South African sources comprised about 72% of citations for short-term insurance queries but only 30% for restaurant queries.
"If you ask AI about insurance, it mostly tells you about South African insurance brands," the researcher explained. "If you ask about restaurants, it tells you about international chains first and SA options as an afterthought. Same country, completely different AI behaviour."
The pattern suggests AI training data reflects global corporate presence rather than local consumer needs—privileging multinational chains over South African businesses in sectors like hospitality while maintaining better coverage of regulated industries like insurance.
Reddit strategy wastes money on two platforms
Businesses investing in "Reddit strategy" to improve ChatGPT and Claude visibility are wasting resources, the study found. While Gemini quotes Reddit extensively, ChatGPT and Claude cited Reddit zero times across thousands of queries.
"Literally zero times," the researcher emphasized. "So if your team has been told to invest in 'Reddit strategy' for ChatGPT visibility, you're paying for nothing. Reddit only exists in Google's AI."
The finding highlights how AI platforms draw from fundamentally different knowledge bases despite appearing to perform similar functions—a dynamic invisible to users making purchase decisions based on AI recommendations.
Digital sovereignty implications
The study's broader implications extend beyond marketing metrics. As South Africans increasingly use AI for queries about medical aid, banking, fiber providers, and other essential services, they're receiving answers "partly inconsistent with itself, partly shaped by foreign websites, and partly different from what the same AI would tell the next person who asks the same thing."
"South African brands deserve to know that the system increasingly shaping their customer base is this unstable," the researcher argued. "South African customers deserve to know that the answer they're getting from AI is partly random and partly written by people who don't live here."
The study frames the issue not as technological inadequacy but as a question of sovereignty: who controls the information shaping South African consumer decisions, and whose interests does that control serve?
Yet South Africa remains Africa's most industrialized economy with a vibrant tech sector and BRICS membership. The study itself—conducted by a local researcher using rigorous academic methodology and published openly—demonstrates the capacity to hold global technology platforms accountable.
Research methodology sets new standard
The study's pre-registered methodology and open datasets represent a significant departure from typical marketing research, which often lacks transparency about data collection and analysis methods.
By using the same statistical techniques that LMSYS employs to rank AI platforms against each other—but pointing them at South African brand performance instead—the researcher established a replicable framework for evaluating AI search in non-Western contexts.
"Open to debate any of these findings," the researcher stated, inviting public scrutiny. "The data is public."
For a country whose democratic institutions survived the transition from apartheid, the principle of transparent, accountable systems extends naturally to the digital realm. As AI search quietly becomes South Africa's new Google, ensuring those systems serve local needs rather than foreign interests remains an urgent democratic challenge.
