Clickbait on an MIT article? What the hell.
I just read this after somebody posted it into a large forum I run. I almost didn't approve it because of it's click baity characteristics... alarmist title, quotes out of context, implications there is a major problem where there isn't one.. etc etc. But I did because I thought it might be a good conversation starter about how to work with and troubleshoot AI systems, and mostly I wanted to hear "This guy doesn't know what he is talking about" from the community.
Now, I'm used to people not understanding things, being confused by things and uncomfortable. That is ok and that is why addoption of new technologies is often slow. But as a general rule, people who don't understand something ask questions so they can expand their understanding. Thinking that something is unknowable or "requires a leap of faith" is not something I expect out of a math or science community.
This biline has me kind of pissed off though, I'll be honest: "No one really knows how the most advanced algorithms do what they do. That could be a problem."....WHAT!? This is so far from the truth I don't even know where to start... The algorithms are all understood at a statistical inference and weighted model buliding perspective from the "How" perspective....
The author then takes a quote from Tommmie Jaakola out of context to lend credence to his assertion that "Nobody understands" All you have to do is look at Tommi's publishing record to know that Dr Jaakkola understands these algorithms. The quote is "It is a problem that is already relevant, and it’s going to be much more relevant in the future,” says Tommi Jaakkola, a professor at MIT who works on applications of machine learning. “Whether it’s an investment decision, a medical decision, or maybe a military decision, you don’t want to just rely on a ‘black box’ method.” This is an advocacy for building systems that can be analyzed, not a condemnation of the technology or the intellectual equivalent of "It's magic and unknowable.
"it might be unknowable", it's "mind boggling"... statements like that are common in the article and lend to an atmosphere of confusion by the author...
Sure, I'll ive you that this stuff is complicated. I'll concede that probability statistical models are difficult to explain and understand. I'll even concede that people who don't understand how these things should be designed, trained and implemented are using them in ways that probably they shouldn't, I'll even concede that if you design a system poorly you won't have any idea how it works... poorly written software has a lot of defects in general and why would AI be any different?
But....if you follow the scientific method, follow a statistical probability methodology, validate your results, etc etc. This stuff isn't a black box. Stop telling people it is please. You are just making the adoption of the technology that much more difficult.
What is the old adage, it's better to remain silent and have everyone think you are an idiot than to open your mouth and prove them right? Now I haven't reached out to Dr J, but I'm guessing he doesn't like his one line quote being used to sell "No one really knows how the most advanced algorithms do what they do." Look at this guys body of work... I am sure he understands how these systems generally work and how to troubleshoot them.
"Without any expert instruction, Deep Patient had discovered patterns ...." - who structured the data? Who defined which algorithms to utilize in the analysis? Who gathered the releveant data? Who normalzied and cleaned it? No expert instruction? You don't build models and validate them without experts.
“We can build these models,” Dudley says ruefully, “but we don’t know how they work.” - Deep patient.... OK....Not exactly a marketing genius is Dudley. You should hire a Statistician or data scientist or system designer to help you implement the fundamentals into the system that allow you to determine the weights built into the models so you can interpret them better... is what I would say if I hadn't bothered to look Dudley up.... but look at this guy. And google him some more... This quote also reeks as out of context the same way Dr. J's quote does. Are you telling me that these to guys who have essentially spent their entire lives working on probability systems don't know how to reverse engineer and troubleshoot a probability matrix?
There are some good, informative, bits of information in the article, but as a whole it's sensationalistic click bait.
If you dont' know how to troubleshoot your AI system... Give me a call and I'll show you how re-design it so you can.
For the record, nobody quoted in that article needs my help... the author is using those researchers authority in the subject to lend credence to his silly article.