Police used AI facial recognition to wrongly arrest TN woman for crimes in ND

We can barely convince powers thar be that eye-witness testimony is unreliable, after all.

It was worse than that, the reporting from an earlier story[0]

  ...Unable to pay her bills from jail, she lost her home, her car and even her dog.

(Also, what happened to journalism - no Oxford comma?)

[0] https://news.ycombinator.com/item?id=47356968

From reading more into the case, it seems the issue may be related to how her lawyer handled the case.

They probably did “identity challenge” arguing that she is not the right person. But from Tennessee’s perspective, she was considered the correct person to be arrested, so there was no “mistaken identity” in their system. In other words, North Dakota Wanted person x and here is person x.

Once a judge in North Dakota reviewed the full evidence (and found that person they issued warrant for arrest is not one they want), the case was dismissed.

Case in point, the US is supposed to have a strong mechanism to prevent this sort of thing. The right to a speedy trial.

In practice they generally make it de facto impossible to get one. You get charged with something and if you want to have the trial right now, before you have any idea what's going on, then you can insist, which basically nobody does because it's pretty crazy to go in blind. But if you don't then you've "waived your right to a speedy trial" and now you're stuck in limbo until the system feels like getting to you.

Her picture was used as part of a fake id card, in the commission of a crime. The fuzzy camera footage looked like her (from stills I've seen) and her picture was on the fake ID. Those 2 circumstantial items were, apparently, enough to have a warrant issued.

They picked her up in TN and held her for 4 months, even after:

The ND police knew the ID was fake and the person using it was not her. The ND police knew she had been in TN before, during, and after the crime.

She is still technically a suspect, even after all of this has come out.

She was not

Source: I live in Fargo and have been following this story closely. Everyone here is pissed

That is the first I have heard of that. A small unexplained blurb in this article. Already in jail on parole violation..

Maybe she objected to the extradition order without good counsel.

"I aint never been to N.Dakota". She found out the hard way how the law works..

What about the banks being hit. Surely they have good cameras. This was bad mojo. I would think a Wells Fargo/BoA has a unit for this stuff.

Finincial crimes handled like this. The banks will be sued too I suspect.. Deep pockets settle out.

I doubt this happened here, but FWIW, AI does have a habit of "cleaning up" (read: hallucinating) interview transcript quotes if you ask it to go through a transcript and pull quotes. You have to prompt AI very specifically to get it to not "clean up" the quotes when you ask it to do that task.

Courts are already refusing to accept this excuse.

https://pub.towardsai.net/the-air-gapped-chronicles-the-cour...

... which is why the institutions that assign responsibility and consequences need to make it really clear that excuse won't fly. With illustrative examples.

There is enormous variability in how hard a tool is to use correctly, how likely it is to go wrong, and how severe the consequences are. AI has a wide range on all those variables because its use cases vary so widely compared to a hammer.

The use case here is police facial recognition. Not hitting nails. The parent wasn't saying "AI is a liability" with no context.

This tool, however, is specifically built for mass surveillance. It serves no other purpose. The tool is broken, and everybody knows it. The tool makers are at least as guilty as those who use it.

Used incorrectly will lead to errors.

Only one small little problem --- there is no way to tell if you are using it "correctly".

The only way to be sure is to not use it.

Using it basically boils down to, "Do you feel lucky?".

The Fargo police didn't get lucky in this case. And now the liability kicks in.

What kind of outcome results from misuse? Clearly a hammer's misuse has very little in common with a global, hivemind network used in high-stake campaigns.

Now, if I misused a hammer and it hurt everyone's thumb in my country, then maybe what you said would have some merit.

Otherwise, I'd say it's an extremely lazy argument

Unlike hammers people preface things with "claude says", etc. I never see that kind of distancing with tools that aren't AI.

Dynamite is a tool. But we don't hand it out to anyone who wants to play with it.

AI feels closer to a firearm than a hammer when accessing law enforcement's ability to quickly do massive, unrecoverable harm.

This, she likely had a shitty public defender that did the bare minimum requirements because they were catering to paying clients. The state was playing hardball because they wanted to make a profit off the poor person with a shitty defense and the public defender was sitting on the bench at a teeball tournament because they werent getting paid enough and didn't want to try.

Probably just reading the room, with States like texas making abortions illegal and allowing random citizens from enforcing that.

Famously, abortions are a woman thing.

Anyway, looking through the facts, it's just some random woman. There's better evidence that these facial recognition systems are much worse at minorities rather than genders.

Interesting biases are own-gendeR: https://pmc.ncbi.nlm.nih.gov/articles/PMC11841357/

Racial bias:

https://mitsloan.mit.edu/ideas-made-to-matter/unmasking-bias...

Miss rates:

https://par.nsf.gov/servlets/purl/10358566

Although you can probably interpret the facts differently, we've seen how any search function gets enshittified: Once people get used to searching for things, they tend to select something that returns results vs something that fails to return results.

Rather than the user blaming themselves, they blame the searcher. As such, any search system overtime will bias towards returning search (eg, Outlook), rather than accuracy.

So if these systems easily miss certain classes of people, women, minorities, they'll more likely be surfaced as inaccurate matches rather than men who'll have a higher confidence of being screened out.

That's how I interpret this 2 second commment.

Police departments are known to avoid hiring people that get high marks in school, under the principle that such individuals will become bored with the job and quit. They literally look for average people with average intelligence: C students.

Now factor in the slow decline of our educational institutions, where grade inflation has systematically diminished the credibility of a degree. I would wager that many C students today would have failed out completely 30 years ago.

In that light, it is not surprising that people are seeing ICE agents behave like brown shirts. No one in power wants those people asking any kind of hard questions about what they are being ordered to do.

Having a degree is a very low bar for intelligence.

> I don’t think they’re as stupid as you suggest.

I'll just leave this here:

https://abcnews.com/US/court-oks-barring-high-iqs-cops/story...