I think that the injection of AI into American life is way premature and fueled by someone's greed.
#2 | Posted by Zed

Agreed.
Did any practicing physicians ask for this?
I doubt it.

Honestly that should not be too hard. The domains are highly segregated and specific. We ran speech to text on Op Reports nearly thirty years ago because the vocabulary is so tight compared to normal spoken English.

"Training a model for cancer detection using annotated MRI data is a data science 1 topic, college Jr. level."

Radiologist told me his front office girls could spot cancer in an X-ray, after seeing so many of them.

This was back when R2 and SecondLook was becoming a thing, thirty-ish years ago. So maybe the front office girls are dumber now.

The biggest unresolved legal issue around AI in medicine has always been:
If AI gets it wrong, who do you put on the stand in the malpractice trial?

A doctor, you can ask them what they were thinking, and how they came up with whatever procedure they tried, was it supported by the literature or their training.

AI can't explain how it got its answer. The hidden layers are just that -- hidden.

This is not to deny "hidden layers" in the human brain -- implicit bias, all kinds of fun things.

But that's a separate problem.

"They're not hidden, that's a euphemism that means it's not part of training data or prediction target."

Dude. They can't be interrogated with any sense of what their respective values mean.

it's just an encoded representation.

I'm aware.

Without the ability to decide it, we can't establish "what it was thinking." Or how. Or why.

It's a liability nightmare.

Not that the law seems to matter any more, since the first step in building a LLM is "Download The Internet" which clearly ignores centuries of copyright law dictating that royalties are owed when copyrighted material is used in a commercial application.

"If you were correct then backpropagation, debugging, and pruning would not work."

If I was incorrect we wouldn't need those tools because we could know explicitly the contribution of every node in every hidden layer.

I'm willing to be wrong about this but it's going to take more than high paying jobs to do it. Maybe a paper written by one of those guys in those fancy jobs could convince me.

US East 1 went down last fall because of a clanker. If Amazon knew what was broken it would have been fixed in minutes, not hours.

"Rumelhart etc, Nature, 1986. A foundational paper for ML."

Imagine you're a hot shot defense lawyer for a doctor who took a course of action on the advice of AI, and it harmed the patient, and experts in the field are testifying the AI made a bad decision and the doctor should have known better.

Now imagine you're a hot shot defense lawyer for an AI company and doctor who took a course of action on the advice of AI, and it harmed the patient, and experts in the field are testifying the AI made a bad decision and the doctor should have known better.

Which lawyer wins?

" You only need the human now because an AI is not a person yet and you need someone to stand up in court."

That's what I'm saying.

The medical AI gives bad advice, there's a malpractice lawsuit, does the AI take the stand? Does the AI vendor take the stand? Who holds the liability?

"Do FDA standards apply to the device?"

If AI was a drug, it would have a hard time getting approved because the precise mechanism of action cannot be stated.