Being relevant, leading and remaining differentiated in the era of AI

Being relevant, leading and remaining differentiated in the era of AI

In the previous post "Yes, AI could be smart enough to take your job," I mentioned that AI would affect the way you make a living. It is different from the “industrial revolution” or the “dot-com era” as: AI aims to replace us, not simply make a specific
Driverless Cars

The Overall Implementation of Driverless Cars

The Implementation of Driverless Cars When is the last time that another driver cut you off? Is that the only use case for driverless cars on our roads? No, it isn’t, but as you sit there at your computer reading this, is it a good

Leveraging machine learning models

As per lightreading’s service assurance and analytics research study conducted with 100+ network operators and service providers, nearly 40% reported that issues around service assurance as a massive challenge.

Service assurance is a big focus area for all the digital service providers (DSPs) and they are looking at ways to address the common problems. Service outages and network performance issues can have a serious impact on brand reputation and any such incident could easily cost millions of dollars to the DSPs. One solution to avoid such unexpected outages is to be able to analyze data at detailed levels to track the underlying quality of network performance.

Key challenges faced by DSPs in the service assurance domain

  • Lower QoS & performance issues
  • Accumulated network faults
  • Network traffic congestion
  • Multivendor-based network device upgrades
  • High downtime of network devices
  • Exhausted capacity
  • Reactive network event handling
  • More open incidents

With the overwhelming volumes and complexity of data coming from the service assurance domain, AI/ML techniques are proving to bring much value. As shown above, reactive network event handling is one among the top three issues. Many DSPs are looking for innovative predictive analytics solution based on machine learning techniques for network event prediction use case.

Machine learning (ML) in network event prediction

This article explores various AI/ML classification techniques that can be used to build an effective network event prediction model. It further discusses various factors to be considered, while improving the prediction accuracy levels.

Leveraging insights from operational data

DSPs are moving towards improving service assurance by leveraging insights from operational data using AI/ML techniques. Data streams from multiple sources such as:

  • Physical and logical inventory systems
  • Fault-related data from NMS/EMS applications
  • Scheduled jobs & case ID related data from workforce management systems
  • Service transactional data from performance systems

These are fed to the AI/ML engine to process the data, visualize and predict actionable results.

Applying event prediction model on operational data

Event prediction model predicts a network event along with its severity beforehand from the performance data of the network node. Whenever a new performance metrics data arrives, the ML algorithm predicts it as either an event or non-event based on built intelligence.

 Leveraging machine learning models

Various machine learning models used in network event prediction use case

DSPs are exploring different ways in finding the optimal solution for demanding network event prediction. Below are the recommended algorithms, which can be used for prediction.

  1. Random Forest Classifier

Random forest is the most commonly used algorithm for classification problem. It accurately does the predictions with less quantity of good data and is easy to train with required inputs.

  1. Gradient Boosting Classifier

It is more advanced than the random forest classifier model. The prediction accuracy is better in most cases and is also easy to train to achieve the best results. It also has a better prediction accuracy with less data.

  1. Neural Networks

This requires a lot of tweaks by SMEs and greater memory, and high-end hardware (larger VM or GPU) for training. It is scalable for larger datasets.

  1. Advanced Machine Learning Algorithms

The following algorithms work well with high quality & scalable data

  • Perceptron (P)
  • Boltzmann Machines (BM)
  • Restricted Boltzmann Machines (RBM)
  • Learning Vector Quantization (LVQ)
  • Recurrent Neural Networks (RNN)
  • Temporal Convolutional Nets (TCNs)
  • Support Vector Machine (SVM)

Deeper tests, validations, and implementations prove Gradient Boosting Model to be more effective & provide higher levels of prediction accuracy for the network event prediction use case

Recommendations on improving prediction accuracy

DSPs are already leveraging AI/ML techniques to improve service assurance issues. However, they face major challenges in achieving higher precision levels. Following are the list of recommended techniques, which help DSPs to improve network event prediction accuracy.

  • Obtain the SME’s inputs to tag the contributing parameters by marking respective fields in the performance data during training or modeling phase
  • Consider low severity network events data before they become promoted alarm
  • Ignore the demoted events data during the calculation of misclassification error percentage
  • Proper consideration of seasonal network event data such as the data collected during a soccer match or a cricket match
  • Performance data collection interval must be in real time or near real time

A trade-off between prediction accuracy and forecast window

DSPs need to make a trade-off between higher accuracy – shorter forecast window vs. lesser accuracy – higher forecast window to take any preventive or corrective measures based on the business use case. The following graph represents the variation of prediction accuracy across different forecast time window using a negative exponential curve. Prediction accuracy is higher with a shorter prediction time window and it tends to decrease as the forecast time increases gradually.

Using these recommended AI/ML techniques, one of the leading DSPs in the US was able to predict 75% of network faults, 30 mins ahead of time

By Avaiarasi S

Avaiarasi S

Avaiarasi heads the NextGen Labs (AI/ML, Big Data, IoT, Microservices) in Prodapt, a global leader in providing software, engineering, and operational services to the communications industry. She has 19+ years of experience spread across various domains/technologies like IoT/M2M, Telecom, Healthcare, AI/ML and Big Data. She has significant expertise in setting up CoE/Incubator Labs and concept-to-commercialization of end-to-end solutions in the NextGen areas. Avaiarasi plays a key role in DX (Digital transformation) initiatives as well.

View Website

TOP ARCHIVES

Infosec thought leaders

Cryptocurrencies and Ransomware: How VDI Can Help Defend Against the Next Ransomware Attack

Cryptocurrencies and Ransomware The WannaCry ransomware made headlines back in May when it crippled hospitals across the UK and put ...
Why should SMEs embrace Cloud ERP solutions?

Why should SMEs embrace Cloud ERP solutions?

SMEs & ERP Solutions Remaining competitive in the market is the primary goal of every business. For SMEs, moving to ...
How artificial intelligence and analytics helps in crime prevention

How artificial intelligence and analytics helps in crime prevention

How Artificial Intelligence Helps Crime Prevention According to a study released by FBI, there is an annual increase of 4.1% ...
Chatbots vs Virtual Assistants

Chatbots vs Virtual Assistants: What Is The Difference?

Chatbots vs Virtual Assistants If you are running an online service or e-commerce store, chances are you offer automated online ...
Secure Business Agility

The First Steps on a CISOs DevOps Journey

CISOs DevOps The marriage between DevOps and Security is rapidly gaining traction. Security is shifting from its former mindset of ...
How Security Certification Helps Cloud Service Providers Stay Transparent and Credible

How Security Certification Helps Cloud Service Providers Stay Transparent and Credible

Security Certification Helps Cloud Service Providers If you are a cloud service provider (CSP), you know your customers have a choice as to who to work with, but do you know what will help tip the scales in your favor? ...
Worldwide Spending on Augmented and Virtual Reality Expected to Surpass $20 Billion in 2019, According to IDC

Worldwide Spending on Augmented and Virtual Reality Expected to Surpass $20 Billion in 2019, According to IDC

FRAMINGHAM, Mass., December 6, 2018 – Worldwide spending on augmented reality and virtual reality (AR/VR) is forecast to be nearly $20.4 billion in 2019, an increase of 68.8% over the $12.1 billion International Data Corporation (IDC) expects will be spent this ...