A Tennessee woman sits in jail for crimes committed in a state she says she's never set foot in. Her arrest? Based on AI facial recognition. This isn't a Black Mirror episode—it's Wednesday.

The technology is impressive. The question is whether law enforcement should be using it without better guardrails. Facial recognition systems can process thousands of faces in seconds, matching surveillance footage against massive databases with confidence scores that look authoritative on a warrant application. But those scores don't tell you about lighting conditions, camera angles, or the statistical reality that confidence is not the same as accuracy.

I've built ML systems. I know how seductive those metrics can be. A model that's 99% accurate sounds bulletproof until you remember that 1% error rate means one wrong arrest out of every hundred. And we're not even talking about lab conditions here—we're talking about grainy security footage, bad angles, and algorithms trained on datasets that have documented racial bias.

The deeper problem is procedural. Facial recognition should be investigative, not dispositive. It's a lead, not a smoking gun. But once an algorithm spits out a match, it becomes the story law enforcement tells itself. Confirmation bias takes over. Other evidence gets interpreted through the lens of "we already know it's her."

Here's what should happen: Facial recognition generates a list of possible matches. Human investigators verify with actual evidence—alibis, financial records, corroborating witnesses. The tech suggests; humans decide. Instead, we're seeing arrests based on little more than an algorithmic suggestion and a detective's hunch.

The tech industry loves to say tools are neutral. They're not. A tool that produces false positives will be misused if the incentive structure rewards quick arrests over accurate ones. And right now, that's exactly what we have.

This isn't about banning facial recognition. It's about admitting that an AI match isn't proof of anything except that two faces look similar to a neural network. Until law enforcement treats it that way—as one data point among many, not a digital fingerprint—we're going to keep seeing more people arrested for crimes they didn't commit in places they've never been.