Explainable Intelligence Part 1 – XAI, the third wave of AI

Ajay Malik

Explainable Intelligence

Artificial Intelligence (AI) is democratized in our everyday life. Tractica forecasts the global artificial intelligence software market revenues will grow from around 9.5 billion US dollars in 2018 to an expected 118.6 billion by 2025.

Gartner predicts that by 2020, AI technologies will be virtually pervasive in almost every new software product and service. It also predicts that the business value created by AI will reach $3.9T in 2022.

We are becoming accustomed to AI making decisions for us in our daily life, from product and movie recommendations on Netflix and Amazon to friend suggestions on Facebook and tailored advertisements on Google search result pages.

NetFlix uses AI for personalization of movie recommendations, auto-generation and personalization of thumbnails / artwork, location scouting for movie production (pre-production), movie editing (post-production), streaming quality and a lot more (Business Insider).

AI plays a huge role in Amazon’s recommendation engine, which generates 35% of the company’s revenue as well as in Amazon’s GO hardware, which includes color and depth cameras, as well as weight sensors and algorithms.

23% of North American enterprises have Machine Learning embedded in at least one company function as of last year. 19% of enterprises in developing markets including China and 21% in Europe also have successfully integrated machine learning into functions. Most businesses are considering adding AI capability in their systems for certain Business Value Proposition.

By 2021, the term AI will no longer be considered a differentiator in marketing tech provider solutions.

Although AI is powerful in terms of results and predictions, AI algorithms suffer from opacity, that it is difficult to get insight into their internal mechanism of work. And, there are times when we need to know why it made a specific decision.

In March 2018, an autonomous vehicle operated by Uber hit and killed a woman in Tempe, Ariz., as she was walking her bicycle across the street.

So what happened? Uber car’s computer system had spotted Ms. Herzberg six seconds before impact, but classified Ms. Herzberg, who was not in a crosswalk, first as an unrecognized object, then as another vehicle and finally as a bicycle. The self-driving software decided not to take any actions after the car’s sensors detected the pedestrian. Uber’s autonomous mode disables Volvo’s factory-installed automatic emergency braking system, according to the US National Transportation Safety Board preliminary report on the accident. Uber suspended testing of self-driving vehicles after the crash. In December, the vehicles returned to public roads, though at reduced speeds and in less-challenging environments.

“This product is a piece of sh**” wrote a doctor at Florida’s Jupiter Hospital regarding IBM’s flagship AI program Watson, according to internal documents obtained by Stat. It recommended ‘unsafe and incorrect’ cancer treatments.

In 2013 IBM developed Watson’s first commercial application for cancer treatment recommendation, and the company secured a number of key partnerships with hospitals and research centers over the past five years. But Watson AI health has not impressed doctors. Some complained it gave wrong recommendations on cancer treatments that could cause severe and even fatal consequences. In February 2017, Forbes reported that MD Anderson had “benched” the Watson for Oncology project.

Amazon HR reportedly used an AI-enabled recruiting software between 2014 and 2017 to help review resumes and make recommendations. The software was, however, found to be more favorable to male applicants.

The software reportedly downgraded resumes that contain the word “women” or implied the applicant was female, for example, because they had attended a women’s college. Amazon has since abandoned the software.

AI is a journey, and in no way, any of these failures are suggesting that we should do to Artificial Intelligence what Luddites tried.

Although Artificial Intelligence has captured the imagination of the world since its inception in 1956, at a historic conference at Dartmouth, it has been stuck in the discovery phase with hand-crafted and custom engineered AI applications. Researchers worked on figuring out how to solve a particular problem, and, then do traditional coding. A key characteristic these applications shared was no learning ability and poor handling of uncertainty. It is the Deep Learning systems, starting in the early 2010’s, aided by huge amounts of training data and massive computational power that have shown us the substantive glimpses of the power and application of AI. The period in AI until 2010 essentially represents the “First Wave” and then the deep learning starting the “Second Wave”.

Deep learning is an architecture modeled loosely on the human brain. It makes use of neural networks that consist of thousands or even millions of nodes (neurons) that are densely interconnected and organized into multiple layers. An individual node might be connected to several nodes in the layer beneath it, from which it receives data, and several nodes in the layer above it, to which it sends data. The essential concept is that there is a “weight” assigned to each of the incoming connection of a node, the node computes the sum of the weighted inputs based on data and weight from all connected nodes beneath it, and then uses a threshold to pass or not pass the data to all its outgoing connections to the nodes in the layer above it (akin to “firing of a neuron”).

For training, huge amounts of labeled data is fed into the neural network. An object recognition system, for Instance, might be fed thousands of labeled images of cats, dogs, digits, and so on, and it would find visual patterns in the images that consistently correlate with particular labels. During training, the weights and thresholds are continually adjusted until training data with the same labels consistently yield similar outputs.

You have “learned” to mathematically recognize incredibly subtle patterns within the mountains of data

Using this “pattern recognition,” you can now “provide” an answer, a decision or a prediction to an input data that you have never seen before.

However, this AI using deep learning still has a long way to go to approach human-level learning, thinking, and problem-solving ability. It is a great tool but is unlike the human brain in basic learning. These algorithms suffer from opacity, and do not provide reasoning. They require mountains of data to train. It has no common sense, conceptual learning, creativity, planning, cross-domain thinking, self-awareness, emotions, etc. It is basically an optimizer based on data for a very specific vertical single task.

Deep Learning is still very weak in intelligence.

Deep Learning

Until we figure out the engineering paths to the general intelligence capabilities, the “fourth wave”, the focus of AI is shifting to opening the blackbox.

We are now entering “third wave” of AI where AI systems will become capable of explaining the reasoning behind every decision made by them. The AI systems themselves will construct models that will explain how it works.

XAI is all about improving trust of AI-based systems. At one end it brings fairness, accountability and transparency to the front and center of AI; and on the other end it enables us to control and continuously improve our AI systems.



To me, this third wave, XAI, is the sine qua non for AI to continue making steady progress without disruption. In last year and a half, I have been spending a lot of time on learning XAI. In this series of articles, I will share my learnings of the strategy, approach, scope, and techniques for XAI.

By Ajay Malik

Martin Mendelsohn

Who Should Protect Our Data?

Who Should Protect Our Data in The Cloud? You would think that cloud service providers are safe havens for your personal data – they all ...
Yuri Sagalov

IT Culture Clash Where Employees Use Multiple Devices To Collaborate

Employees use multiple devices to collaborate It used to be that company IT decision makers could simply dictate the software that business units would use ...
Sangeeta Chhabra

What Accountants Should Know About The Cloud

Cloud Accounting Cloud technology has been at the top of the charts of new-age technologies for a long time now. Almost every industry in the ...
Martin Mendelsohn

Supporting CISOS, CIOS and CTOS That Are Overwhelmed During the COVID Battle

The Covid Era and CISO Stress Even before COVID-19, senior technology executives, including CISOs, CIOs and CTOs were overwhelmed, and felt an increasing lack of ...

Machine Learning: The Importance of Actionable Data

The Importance of Actionable Data How awesome would it be to know for sure exactly what your customers want to see from your business? Imagine ...
Sebastian Grady

Leveraging Hybrid IT Now to Power Digital Transformation 

Leveraging Hybrid IT Summary: Cloud is a dominant force in enterprise software today. Global market turbulence is forcing some companies to accelerate moving parts of ...