A 73-year-old North Dakota grandmother spent months in jail after facial recognition software incorrectly identified her in a fraud case - a stark reminder that AI deployment without adequate safeguards has real human costs.This isn't a thought experiment about AI ethics debated in academic journals. This is an actual person who lost months of her life because an algorithm got it wrong and no one properly validated its output before locking her up.What happenedThe woman was arrested on fraud charges after facial recognition software matched her to surveillance footage of the crime. The match was convincing enough for prosecutors to file charges and a judge to deny bail. She sat in jail for months before the actual perpetrator was caught and the error discovered.The details matter here: which facial recognition system was used? What were its tested accuracy rates? What demographic groups show higher error rates? And most critically - did anyone manually verify the AI's identification before depriving someone of their freedom?Based on reporting, it appears the answer to that last question is no. The AI said it was a match, and the system trusted the AI.The accuracy problemFacial recognition systems are known to have significantly higher error rates for certain demographic groups. Studies have repeatedly shown worse performance for women, elderly individuals, and people of color. A 73-year-old woman falls squarely into categories where these systems struggle.But even setting aside demographic accuracy concerns, the fundamental issue remains: facial recognition should be investigative evidence, not conclusive evidence. It's a lead for human investigators to follow up on, not grounds for incarceration on its own.When I was working in tech, we had a rule: AI systems can suggest, but humans must decide - especially when the stakes are high. Jail time qualifies as high stakes.Who's responsible?The easy answer is "the AI got it wrong." The honest answer is that humans deployed an AI system without proper safeguards, then trusted its output without adequate verification.Law enforcement agencies are rapidly adopting facial recognition technology, often without clear protocols for validating results before taking action. The technology is impressive. The question is whether we've built the institutional frameworks to use it responsibly.This grandmother's months in jail suggest we haven't. The system failed at multiple points: the AI misidentified her, investigators didn't adequately verify the match, prosecutors proceeded with charges, and a judge found probable cause - all based on an algorithm's mistake.The technology exists. The safeguards apparently don't. And until that changes, we're going to see more innocent people losing months or years of their lives to algorithmic errors that should have been caught by basic human oversight.