Preventing Disastrous Events Through The Power of Anomaly Detection In Machine Data

Preventing Disastrous Events Through The Power of Anomaly Detection In Machine Data

Preventing Disastrous Events Through the Power of Anomaly Detection in Machine Data

In August, a single server failed and the NASDAQ went down for three hours. In January, GlobalPayments reported that a hack compromising millions of credit card accounts cost them $93 million to recover from. In both cases, nobody took action until the damage was done because quite simply, nobody could. IT planners could not proactively head off the failure or breach because they had no suspicious behavior or early warning system to alert them that something was going amiss; all they had were mountains of event logs that needed to be pored through after the damage was already said and done.

nasdaq2

(Image Source: Shutterstock)

To prevent such catastrophes in the future, three things are required:

  • A machine learning engine that can analyze and learn from data – as well as human interaction and feedback – in order to get smarter over time;
  • Big Data technologies;
  • And of course, a cloud platform for ease of management.

Machine logs are the output of every application, website, server and supporting IT infrastructure component in the enterprise. This means that IT teams are inundated by massive amounts of machine log data. Digging through all of this data for something meaningful is not only unwieldy and unappealing, but also ultimately drives down productivity and increases costs. IT teams need to be able to not only visualize, but also analyze machine data in a way that can provide clear insight into what events in that stream of data are benign and what events are malicious that would require immediate attention. By being able to make sense of machine data from an “event” perspective, IT teams can create optimal functionality around any environment and also bring a true, proactive approach to IT management.

The goal of combining Big Data and a machine learning engine all within a cloud platform is to make events easily known and to procure insight on such events prior to their occurrences, which would result in significantly fewer headaches for IT managers and CIOs. Big Data technologies enable a holistic approach to analysis of data without binding to schemas, volumes or batch analytics. A machine learning engine provides advanced algorithms that learn and analyze from data as well as humans to increase intelligence over time. And lastly, the combination of these components in a cloud-based management platform enables an elastic compute at the massive scale that’s needed to analyze this amount of data in real-time across all vectors. By having this capability, IT managers can then create playbooks and remediation steps to prevent certain events and anticipate the impact to their organization.

Event detection can play a big role in optimizing system availability and performance; when a process, application or infrastructure component fails or slows down, it’s typically presaged by multiple events occurring simultaneously or in rapid succession. Out-of-the-ordinary or “anomaly” event detection can decipher how this series of events and their patterns vary from the norm, and what the variation means to the business. The power behind anomaly detection is neither a single technology nor a single technique. It’s typically a set of algorithms that work synergistically, leveraging machine learning techniques as well as mathematical and statistical analysis.

The benefits are clear, but there are many solutions out there that claim to do this, but do not leverage the three critical components for successful anomaly detection: machine learning engine, big data analytics and a cloud management platform. Without these three things working harmoniously together, it is that much more difficult to proactively manage the IT environment. Coming full circle, disastrous events such as the NASDAQ going down or a company losing millions of dollars could have been prevented through the power of machine data and anomaly detection.

sanjayBy Sanjay Sarathy, CMO of Sumo Logic

Sanjay joins Sumo Logic with over seventeen years of marketing, business development and community building experience in both SaaS and enterprise software environments. Prior to Sumo Logic, he was at Vindicia, a SaaS company that provides online billing and marketing solutions for for companies selling digital content and services. He also spent time at Above All Software, Qualys, Sun Microsystems and NetDynamics. Sanjay has a BA in Quantitative Economics from Stanford University and a MBA from the Haas School of Business at UC Berkeley.

About CloudTweaks

Established in 2009, CloudTweaks is recognized as one of the leading authorities in connected technology information and services.

We embrace and instill thought leadership insights, relevant and timely news related stories, unbiased benchmark reporting as well as offer green/cleantech learning and consultive services around the world.

Our vision is to create awareness and to help find innovative ways to connect our planet in a positive eco-friendly manner.

In the meantime, you may connect with CloudTweaks by following and sharing our resources.

View All Articles

Sorry, comments are closed for this post.

Comic
Cloud Computing – The Good and the Bad

Cloud Computing – The Good and the Bad

The Cloud Movement Like it or not, cloud computing permeates many aspects of our lives, and it’s going to be a big part of our future in both business and personal spheres. The current and future possibilities of global access to files and data, remote working opportunities, improved storage structures, and greater solution distribution have…

The FTC, Data Privacy and Facebook

The FTC, Data Privacy and Facebook

Data Protection Facebook is in deep water over their recent decision to start harvesting phone numbers from one of the apps they own, called WhatsApp. WhatsApp is a mobile phone app that allows people to place long distance phone calls and send SMS messages for free. A complaint was filed with the Federal Trade Commission…

Three Tips To Simplify Governance, Risk And Compliance

Three Tips To Simplify Governance, Risk And Compliance

Governance, Risk and Compliance Businesses are under pressure to deliver against a backdrop of evolving regulations and security threats. In the face of such challenges they strive to perform better, be leaner, cut costs and be more efficient. Effective governance, risk and compliance (GRC) can help preserve the business’ corporate integrity and protect the brand,…

The Annual Compliance & Ethics Institute:  Hot Topics – Cyber Security, Big Data, Privacy Breach Response

The Annual Compliance & Ethics Institute: Hot Topics – Cyber Security, Big Data, Privacy Breach Response

Cyber Security, Big Data, Privacy Breach Response MINNEAPOLIS, Aug. 30, 2016 /PRNewswire-USNewswire/ — Cyber security, social media, modern slavery, anti-corruption, export controls and sanctions, and privacy top the list of “Hot Issues” compliance and ethics professionals face each day. These and many other compliance and ethics concerns will be addressed at the 2016 Compliance & Ethics…

Top 5 Digital Health Trends

Top 5 Digital Health Trends

Digital Health Trends It is very important to keep up with the changing technology. However, it is also just as important to advance the consumer experience, care delivery methods and create opportunities for career development for the healthcare workforce. Five trends that are proven to be effective in winning in the digital age have been…

Having Your Cybersecurity And Eating It Too

Having Your Cybersecurity And Eating It Too

The Catch 22 The very same year Marc Andreessen famously said that software was eating the world, the Chief Information Officer of the United States was announcing a major Cloud First goal. That was 2011. Five years later, as both the private and public sectors continue to adopt cloud-based software services, we’re interested in this…

Get Ready For Virtual Reality and the Cloud

Get Ready For Virtual Reality and the Cloud

Virtual Reality Cloud We’re lucky to live in an era where virtual reality is no longer relegated to the confines of a sci-fi movie universe. Thanks to technology introduced by products like Oculus Rift, consumers now have access to virtual environments with fully immersive graphic capabilities. As a result, companies have only just begun to…

How Formal Verification Can Thwart Change-Induced Network Outages and Breaches

How Formal Verification Can Thwart Change-Induced Network Outages and Breaches

How Formal Verification Can Thwart  Breaches Formal verification is not a new concept. In a nutshell, the process uses sophisticated math to prove or disprove whether a system achieves its desired functional specifications. It is employed by organizations that build products that absolutely cannot fail. One of the reasons NASA rovers are still roaming Mars…

Are Cloud Solutions Secure Enough Out-of-the-box?

Are Cloud Solutions Secure Enough Out-of-the-box?

Out-of-the-box Cloud Solutions Although people may argue that data is not safe in the Cloud because using cloud infrastructure requires trusting another party to look after mission critical data, cloud services actually are more secure than legacy systems. In fact, a recent study on the state of cloud security in the enterprise market revealed that…

Protecting Devices From Data Breach: Identity of Things (IDoT)

Protecting Devices From Data Breach: Identity of Things (IDoT)

How to Identify and Authenticate in the Expanding IoT Ecosystem It is a necessity to protect IoT devices and their associated data. As the IoT ecosystem continues to expand, the need to create an identity to newly-connected things is becoming increasingly crucial. These ‘things’ can include anything from basic sensors and gateways to industrial controls…

Cloud Computing Myths That SMBs Should Know

Cloud Computing Myths That SMBs Should Know

Cloud Computing and SMBs Cloud Computing is the hottest issue among IT intellects of Small and Medium Businesses (SMBs). Like any other computer-orientated technology, Cloud Computing has some misconceptions and myths that often kick-start arguments among the two opposing groups: Cloud Supporters and Cloud Opponents. Both of these groups have their own ideology and reasons…

Infographic Introduction – Benefits of Cloud Computing

Infographic Introduction – Benefits of Cloud Computing

Benefits of Cloud Computing Based on Aberdeen Group’s Computer Intelligence Dataset, there are more than 1.6 billion permutations to choose from when it comes to cloud computing solutions. So what, on the face of it, appears to be pretty simple is actually both complex and dynamic regardless of whether you’re in the market for networking,…

Cloud Computing Is Greener Than You Think

Cloud Computing Is Greener Than You Think

Cloud Computing Is Greener Than You Think Last week we touched upon how a project in Finland had blended two of the world’s most important industries, cloud computing and green technology, to produce a data centre that used nearby sea water to both cool their servers and heat local homes.  Despite such positive environmental projects, there…

Low Cost Cloud Computing Gives Rise To Startups

Low Cost Cloud Computing Gives Rise To Startups

Balancing The Playing Field For Startups According to a Goldman Sachs report, cloud infrastructure and platform spending could reach $43 billion by 2018, which is up $16 billion from last year, representing a growth of around 30% from 2013 said the analyst. This phenomenal growth is laying the foundation for a new breed of startup…

Utilizing Digital Marketing Techniques Via The Cloud

Utilizing Digital Marketing Techniques Via The Cloud

Digital Marketing Trends In the past, trends in the exceptionally fast-paced digital marketing arena have been quickly adopted or abandoned, keeping marketers and consumers on their toes. 2016 promises a similarly expeditious temperament, with a few new digital marketing offerings taking center stage. According to Gartner’s recent research into Digital Marketing Hubs, brands plan to…

The Future of M2M Technology & Opportunities

The Future of M2M Technology & Opportunities

The Future Of The Emerging M2M Here at CloudTweaks, most of our coverage is centered around the growing number of exciting and interconnected emerging markets. Wearable, IoT, M2M, Mobile and Cloud computing to name a few. Over the past couple of weeks we’ve talked about Machine to Machine (M2M) such as the differences between IoT and…