John Pientka

Will XAI Fix WMD? Or, No More Black Boxes!

Will XAI Fix WMD?

Why did I get turned down for a loan? Why is my insurance so expensive? Algorithms are the answer: black box formulas that determine so many aspects of today’s life. Critics call them WMD’s – Weapons of Math Destruction. But maybe there is relief in sight from XAI.

Algorithm –  An Arabic word from 825CE that has come down to us through the centuries to mean a series of steps to process input in such a way as to produce a desired outcome. Coupled with Big Data (another gift from our increasingly digital world) they have come to surround and saturate our lives. They cover everything from recommending things to buy or watch, to will you get a job, keep the one you have or even go to jail.

Designed by humans, they are opaque and often unintentionally embody human biases and prejudices. Author Cathy O’Neil exposes their harmful effects in her book – “Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy”. (Here is also her TED Talk.) Besides numerous examples she points out that algorithms are also driving class distinctions in our society. The masses get measured and processed by computer; the rich elite gets personal review and attention. It’s pretty sobering reading but things are getting worse.

O’Neil describes our current world where the most of these algorithms are designed by humans (hence their flawed nature). If we could hold their feet to the fire and expose hidden assumptions we stand a chance of fixing some of the issues. But as organizations increasingly turn to Artificial Intelligence (AI) to analyze big data and predict outcomes, they cannot even explain how the AI reaches its conclusions.

These are truly black boxes. Advocates promote them as tireless assistants that will make our lives and work easier while boosting our performance and accuracy. But they raise troubling questions. Should a doctor accept a diagnosis made by an AI when the machine cannot explain how it arrived at that diagnosis?

AI’s are not programmed. They are given a whole bunch of big data as input, given a goal to achieve and then turned loose. No one describes the steps – a program – that they should go through to arrive at the goal instead they learn from the inputs. Feels a little spooky doesn’t it? But, that’s how Google’s Deep Mind AlphaZero learned how to play world-class chess in four hours.

AI’s appear to see patterns that we humans do not pick up. In one notorious case a researcher’s AI appeared to predict whether an individual was straight or gay just from their picture! It can’t explain how it does it but it appears to be uncannily accurate.

That’s where a new trend is emerging. It’s called XAI or Transparent Artificial Intelligence. Deep machine learning performs an amazing number of recursive steps as it seeks patterns in the data provided as input. The XAI concept is for the machine to explain how it arrived at the particular chain that led to its conclusion.

Image Courtesy DARPA

An organization called AI Now, comprised of researchers from Google Open Research, Microsoft Research, and New York University, recently issued a call for 10 changes the A.I. community needs to make in 2017. At the top of that list: “Core public agencies, such as those responsible for criminal justice, healthcare, welfare, and education should no longer use ‘black box’ A.I. and algorithmic systems.”

I would add quite a few more actors beyond public agencies like those who assess credit worthiness, job qualifications, etc. Which ones would you recommend be added?

It’s funny but kind of tragic. We have trusted the “secret sauce” of algorithms designed by humans to operate on big data sets but when it comes to trusting machines – look out. Sigh! – Maybe it’s a round about way to get there but hopefully the rise of the machines will help us spread sunshine on this whole algorithm mess.

By John Pientka

John Pientka

John is currently the principal of Pientka and Associates which specializes in IT and Cloud Computing.

Over the years John has been vice president at CGI Federal, where he lead their cloud computing division. He founded and served as CEO of GigEpath, which provided communication solutions to major corporations. He has also served as president of British Telecom’s outsourcing arm Syncordia, vice president and general manager of a division at Motorola.

John has earned his M.B.A. from Harvard University as well as a bachelor’s degree from the State University in Buffalo, New York.

View Website
Bryan Doerr

Cyber-Threats and the Need for Secure Industrial Control Systems

Secure Industrial Control Systems (ICS) Industrial Control Systems (ICS) tend to be “out of sight, out of mind.” These systems ...
10 Ways The Enterprise Can Prevent Data Leaks In The Cloud

10 Ways The Enterprise Can Prevent Data Leaks In The Cloud

Prevent Data Leaks In The Cloud More companies are turning to the cloud for storage. In fact, over 60 percent ...
Leading Multicloud Strategies

Solving the Complexities of Leading Multicloud Strategies

Leading Multicloud Strategies To avoid the dreaded cloud lock-in, many organizations are now managing multiple clouds to service their business ...
5% Of Companies Have Embraced The Digital Innovation Fostered By Cloud Computing

5% Of Companies Have Embraced The Digital Innovation Fostered By Cloud Computing

Embracing The Cloud We love the stories of big complacent industry leaders having their positions sledge hammered by nimble cloud-based ...
IoT Security Intel

Cyber IoT Security: McAfee on Threats and Autonomous Cars

IoT Security Autonomous cars are just around the corner, there have been IoT security controversies surrounding their safety, and a ...
The Shift from Monolithic to Microservices: What It Means for CTOs.

The Shift from Monolithic to Microservices: What It Means for CTOs.

The Shift to Microservices The shift in application development strategies is moving from monolithic design to isolated and resilient components ...
Four Providers of Smart City Kiosks Named IDC Innovators

Four Providers of Smart City Kiosks Named IDC Innovators

FRAMINGHAM, Mass., July 11, 2018 – International Data Corporation (IDC) today published an IDC Innovators report profiling four providers that are considered key emerging vendors in the Smart City kiosks market. The four companies named as ...
'What the Hock?'; Broadcom shares sink on shock software deal

‘What the Hock?’; Broadcom shares sink on shock software deal

(Reuters) - Broadcom Inc’s (AVGO.O) surprise bid to buy software company CA Inc (CA.O) knocked $11 billion off the value of the chipmaker in trading before the bell on Wall Street on Thursday, with analysts ...
Closer Collaboration Between C-Suite and CISOs Needed to Bridge Gap in Cyber Readiness, Finds Accenture Report

Closer Collaboration Between C-Suite and CISOs Needed to Bridge Gap in Cyber Readiness, Finds Accenture Report

New Accenture survey finds fewer than one-third of CISOs and business leaders collaborate on a cybersecurity plan and budget NEW YORK; July 10, 2018 – With the proliferation of more and more sensitive data, expanding ...