A Tennessee woman was arrested based on AI facial recognition matching her to crimes committed in a state she says she's never visited. This is exactly the nightmare scenario we've been warning about: technology deployed at scale before accuracy rates are good enough for the consequences.

Facial recognition is sold as scientific. Objective. More reliable than human witnesses. The reality is messier. These systems have error rates. They have bias problems. They make mistakes. And when they make mistakes with policing applications, people's lives get destroyed.

What happened here: police used an AI facial recognition system to match surveillance footage from a crime scene to a database of photos. The system returned a match with what appeared to be high confidence. Based on that match, a woman in Tennessee was arrested for crimes allegedly committed in another state - a state she claims she's never been to.

Now she's facing charges. She's hiring lawyers. She's trying to prove she wasn't even in the state when the crimes occurred. All because an algorithm said her face matched.

Here's what we don't know from the reporting: what confidence threshold triggered the arrest? Most facial recognition systems return a percentage match - 70%, 85%, 95%. At what point did police decide that was good enough to make an arrest? And what was the system's claimed accuracy rate versus its real-world performance?

Vendors love to advertise accuracy rates above 95%. But those numbers are typically measured on clean, well-lit, frontal photos. Surveillance footage is grainy, angled, partially obscured, compressed. Real-world accuracy on the kind of images police actually use is significantly lower than lab benchmarks.

There's also the demographic question. Facial recognition systems have documented higher error rates for women and people with darker skin tones. The technical reasons are well understood: training datasets skewed toward certain demographics, different feature extraction performance across skin tones, lighting bias. If the arrested woman falls into a demographic category where the system has higher error rates, that makes false positives more likely.

What makes me angry about cases like this: the technology is being deployed without meaningful accountability for errors. If a human witness misidentifies someone, we understand that witness testimony is fallible. We corroborate. We verify. But when a computer says someone matches, there's an implicit trust that it's scientific and therefore correct.

It's not. These systems are probabilistic. They make educated guesses. Sometimes those guesses are wrong. The question is what safeguards exist when the system confidently points at the wrong person.

From what I can tell, many jurisdictions using facial recognition don't have clear policies about: minimum confidence thresholds for arrests, requirements for additional evidence beyond the AI match, disclosure to defendants that facial recognition was used, or accountability when the system makes errors.

The correct use of facial recognition in policing is as an investigative lead, not as evidence. The system can suggest "this person might be worth looking into." Then you do actual detective work: check alibis, verify locations, gather corroborating evidence. The AI match is where investigation starts, not where it ends.

Instead, what appears to be happening in some cases is: AI match → arrest warrant → the defendant has to prove the AI was wrong. That's backwards. The burden of proof should be on the state to confirm the match, not on the defendant to disprove it.

There's a deeper problem here about deploying AI systems in high-stakes situations before we've figured out the error rates and consequences. Tech companies love to move fast and break things. When what you're breaking is people's freedom, that philosophy needs serious revision.

What needs to happen: mandatory disclosure when facial recognition is used in arrests, minimum accuracy standards before systems can be deployed, independent auditing of error rates, and civil liability when systems make demonstrable errors. Right now, when facial recognition destroys someone's life, nobody is accountable except the person wrongly arrested.

The technology has uses. It can help find missing persons. It can generate investigative leads. But treating it as infallible is dangerous. And when it fails, someone needs to be responsible for the consequences beyond just the person whose face matched wrong.