IDC Expects Asia/Pacific* Artificial Intelligence Systems to Reach Nearly USD 5.5 Billion in 2019

IDC Expects Asia/Pacific* Artificial Intelligence Systems to Reach Nearly USD 5.5 Billion in 2019

SINGAPORE, May 21st, 2019 – According to the latest Worldwide Semiannual Artificial Intelligence Systems Spending Guide , Asia/Pacific* spending on artificial intelligence (AI) systems is forecast to reach nearly USD 5.5 billion in 2019, an increase of almost 80% in spent versus 2018. As industries invest
The ‘Unprecedented Challenge’ of Cybersecurity in an Age of Burgeoning Threats

The ‘Unprecedented Challenge’ of Cybersecurity in an Age of Burgeoning Threats

The ‘Unprecedented Challenge’ of Cybersecurity Technological and legal complexities abound in this age of heightened cybersecurity threats—including a rise in state-sponsored hacking. This “unprecedented challenge” was the topic of conversation between Dorian Daley, Oracle executive vice president and general counsel, and Edward Screven, Oracle’s chief

Creating Smart Products from “The Incredibles”

Nothing strikes terror into the hearts of humans more than the idea of an intelligent robot gone bad.  The fear is that a robot can acquire the ability to learn and adapt to the point of superseding their human creators…and with evil intentions.

From Gort (“The Day the Earth Stood Still”) to Sonny (“I, Robot”), films provide a wide variety of potential robot scenarios.  Only a few of these film robots have demonstrated artificial intelligence to the point where they have threatened humankind (like the Cyberdyne Systems series T-800 Model 101 in “Terminator” and the unnamed NS-5 robots in “I, Robot” ).

However the one evil robot that demonstrated its ability to continuously learn through experimentation and failure would be the Omnidroid from the “The Incredibles”.  As representative of the data science process in Figure 1, the development of the Omnidroid occurred over many iterations, with each failure a chance to learn and perfect its killing skills.

Figure 1: Data Science Engagement Process

As we were warned early in the movie by the character Mirage: “It’s a learning robot. Every minute you spend fighting it only increases its knowledge of how to beat you.”  The end result was a seemingly undefeatable robot, the Omnidroid V10 (see Figure 2).

Figure 2: The Omnidroid v10

As we kickoff 2019, a year which promises to be full of simultaneous Artificial Intelligence-induced exuberance and terror, let’s see what we can learn about the future of “smart” products from the Omnidroid’s evolution; that is, how do we create products that increase their knowledge, insights and predictive capabilities through every interactions in order to better defeat you…er, um…serve you.

3 Levels of Advanced Analytics

I wrote a blog a while back titled “Artificial Intelligence is not ‘Fake’ Intelligence” where I introduced the 3 levels of Advanced Analytics because I needed a simple positioning of the advanced technologies like machine learning, deep learning and reinforcement learning for my class at the University of San Francisco.  Let’s apply our learnings from that blog to see how one would apply the 3 levels of advanced analytics to create “smart”, autonomous products, using the development of the Omnidroid as our measuring stick.

Level 1: Insights and Foresight

Level 1 is about codifying cause-and-effect and measuring goodness of fit

Level 1 forms the foundational level that seeks to quantify cause-and-effect, establish confidence levels, measure goodness of fit, and predict what is likely to happen next at some measurable level of confidence.  Level 1 includes the following analytic capabilities:

  • Statistics, which is a branch of mathematics dealing with the collection, analysis, interpretation, presentation and organization of data. Statistical methods are used to support hypotheses (decisions) and provide credibility to modeling results and outcomes (via confidence levels and “goodness of fit” measures).
  • Data Mining, which is the analysis of large data sets to uncover previously unknown patterns such as groupings of similar activities (cluster analysis), unusual events (anomaly detection), and dependencies between entities (association rule mining, sequential pattern mining).

The Omnidroid results from Level 1 Advanced Analytics? “You clinking clanking clattering collection of caliginous junk!” to quote the Wizard from the “The Wizard of Oz” (see Figure 3).

Figure 3:  Level 1: Insights and Foresight for the Omnidroid

The early Omnidroid versions weren’t successful, though they did have victories over some of the lesser “Supers” (Universal Man, Psycwave, Everseer, Macroburst, Phylange, Blazestone) that inspired further investment. Time to move to the next level of advanced analytics.

Level 2: Optimized Machine Decision-Making

Level 2 is about codifying trends, patterns and relationships buried in the data in order to predict what’s likely to happen next

Level 2 seeks to enable computers to learn on their own; to identify patterns, trends, associations and relationships in data, build models that explain the data, and predict outcomes without having pre-programmed rules and analytic models. Level 2 includes the following advanced analytic capabilities:

  • Supervised Machine Learning “learns” of relationships (known unknowns) buried in the data given a labelled result or outcome (fraud, attrition, failure, web click); it uses different Supervised Machine Learning algorithms to “learn” the relations between multiple input variables and an output variable (labelled result) using 1) Regression-based methods to predict outputs based on input variables and uses regression algorithms (e.g., Linear, Logistic, Polynomial, Ridge, and Lasso Regression) and 2) Classification methods to identify to which category a set of classifications belong using classification algorithms (e.g., Support Vector Machines, Decision Trees, Random Forest, and Nearest Neighbor).
  • Unsupervised Machine Learning does not have labelled outcomes, consequently it uses multiple input variables to discover the previously unknown relationships between those variables (unknown unknowns).  A common Unsupervised Machine Learning algorithm example is clustering analysis where the machine finds those clusters after the analysis.  Other Unsupervised Machine Learning algorithms include: association models/market basket analysis, K-means, Principal Component Analysis (PCA), Birch, Ward and Independent Component Analysis (ICA).
  • Deep Learning leverages a system of highly interconnected analytic layers to decompose complex data formats (e.g., images, audio, video) in order to learn about the data and create inferences about the data.  Deep Learning or Neural Network algorithms include Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Generative Adversarial Neural Network (GANN) and Markov Chain.  Check out the blog “The Mostly Complete Chart of Neural Networks, Explained” for a most excellent compilation of the different Neural Network algorithms.

By the way, the lines that separate these analytic categories tends to blur. For example, is a Markov Chain or a Support Vector Machine classified as Unsupervised Machine Learning or Deep Learning? To be honest, algorithms like these probably fall into both classifications.

And what are the Omnidroid results from the Level 2 Advanced Analytics (see Figure 4)?

Figure 4: Level 2: Optimized Machine Decision-Making

The second generation Omnidroids were successful at amassing insights about the individual Supers in order to defeat them (Gazerbeam, Stormicide, Gamma Jack).  Unfortunately, they had reached the limits of what they could accomplish with Level 2 analytics (which was confirmed by its ass-kicking at the hands of Mr. Incredible).  Time to move onto the Level 3 of the advanced analytics.

Level 3: The Learning and Intelligent Enterprise

Level 3 creates interactive environment that uses trial-and-error in order to maximize rewards while minimizing costs

Level 3 focuses on advanced analytic algorithms that can self-monitor, self-diagnose, self-adjust/fix/heal and self-learn. These analytics perceive the world around them (to assess and quantify rewards and costs), create goals, make decisions towards those goals, measure decision effectiveness, and learn in order to refine the decisions that advance towards the goals.

  • Reinforcement Learningfocuses on how software agents take actions in an environment so as to maximize cumulative rewards while minimizing costs. Reinforcement learning uses trial-and-error to map situations to actions so as to maximize rewards. Actions may affect immediate rewards but also affect subsequent or longer-term rewards, so the full extent of rewards must be considered when evaluating the reinforcement learning effectiveness.  Examples of Reinforcement Learning algorithms include Q-Learning and SARSA.

And what are the Omnidroid development results from this third generation of Omnidroids (see Figure 5)?

Figure 5:  Level 3: The Learning and Intelligent Product

The Level 3 Omnidroid v10 ultimately suffered the same fact as the earlier Omnidroid versions – defeat.  Some of the failings that lead to the failure of the Omnidroid v10 include:

  • No experience fighting multiple Supers simultaneously.  It didn’t know how to respond to multiple Supers, each with a different super power, with a coordinated attack pattern (helps the Elastic Girl, Mr. Incredible and Frozone had worked together in the past).
  • No experience fighting anywhere but on its native island.  It was unfamiliar fighting in a large metropolitan area which offered different fighting resources and hiding places than its native island.

But maybe the biggest downfall for the Omnidroid v10 was the improper assessment of its analytic model “success”.  The Omnidroid v10 thought it had defeated Mr. Incredible (which it hadn’t), which lead to the pre-mature launch of the product which ultimately lead to its failure.  For the analytic models that powered the Omnidroid v10, “good enough” wasn’t actually “good enough”.

Creating Smart Products Summary

Hopefully this was a fun way to look at what is going to be required from an analytics perspective to create “smart” products.  There are many valuable lessons that data scientists can learn from the movie “Mr. Incredible” about the challenges of creating smart products like autonomous vehicles, trains and factory robots.

And maybe the biggest challenge for the development of smart, autonomous products is knowing when “good enough” is actually “good enough”.  When trying to optimize the operations of these smart, autonomous products, one must be prepared to realize that the current path to performance optimization may not actually be the optimal path, and the data science team must be prepared to jettison their existing work and be willing to try a different path that might lead to a better performing analytic model (see Figure 6).

Figure 6: Saddle Points and Challenge Optimizing across Multiple Dimensions

This is an important lesson for the creation of our AI-induced “smart” products – that there must be constant testing, learning, and maybe even some unlearning and re-starting afresh in order to find the optimal models.  But Syndrome, the arch villain and the creator of the Omnidroids, will never know if he was on the right optimization path, because he died at the end of the movie. Oh well, end of lesson for him.

By Bill Schmarzo

Bill Schmarzo

CTO, IoT and Analytics at Hitachi Vantara (aka “Dean of Big Data”)

Bill Schmarzo, author of “Big Data: Understanding How Data Powers Big Business” and “Big Data MBA: Driving Business Strategies with Data Science”. He’s written white papers, is an avid blogger and is a frequent speaker on the use of Big Data and data science to power an organization’s key business initiatives. He is a University of San Francisco School of Management (SOM) Executive Fellow where he teaches the “Big Data MBA” course. Bill also just completed a research paper on “Determining The Economic Value of Data”. Onalytica recently ranked Bill as #4 Big Data Influencer worldwide.

Bill has over three decades of experience in data warehousing, BI and analytics. Bill authored the Vision Workshop methodology that links an organization’s strategic business initiatives with their supporting data and analytic requirements. Bill serves on the City of San Jose’s Technology Innovation Board, and on the faculties of The Data Warehouse Institute and Strata.

Previously, Bill was vice president of Analytics at Yahoo where he was responsible for the development of Yahoo’s Advertiser and Website analytics products, including the delivery of “actionable insights” through a holistic user experience. Before that, Bill oversaw the Analytic Applications business unit at Business Objects, including the development, marketing and sales of their industry-defining analytic applications.

Bill holds a Masters Business Administration from University of Iowa and a Bachelor of Science degree in Mathematics, Computer Science and Business Administration from Coe College.

Delivering Serverless Applications Using AWS Well-Architected Frameworks

Delivering Serverless Applications Using AWS Well-Architected Frameworks

This is part 1 in a 2-part series on serverless cloud computing. Rapidly expanding connectivity options and increased development in ...
How to Transform Your Operating Model for the Cloud

How to Transform Your Operating Model for the Cloud

Transform Your Operating Model It can be tough for established organizations to embrace change, so when they start working with ...
Infosec thought leaders

How a Connection Broker Manages Complexity and Remote Access

Hyperconverged and Hybrid Environments Consolidating desktop workloads in the datacenter using hyperconverged infrastructure and virtualization optimizes resources, reduces power consumption, ...
5 Ways the Cloud and IoT Have Transformed the Transportation Industry

5 Ways the Cloud and IoT Have Transformed the Transportation Industry

IoT Transportation Industry The Internet of Things has caused many industries to evolve - but few more than transportation. Here ...