Understanding Big Data

Understanding Big Data

What do Wal-Mart, Facebook and the Hadron Collider have in common? They are just three of many large organizations who are major consumers and processors of Big Data, a term that is becoming a greater priority for companies around the world as they struggle with a ceaseless and ever growing ocean of information.

The short definition of Big Data is that it represents all of the data in your organization – not just one type. Data resides in all business functions: marketing, finance, operations, research & development, customer experience – everywhere – and it essentially comes in three flavors: structured, unstructured and binary.

Structured data is the type of information that is organized and indexable, and consequently is most often stored in databases or annotated documents. This could include records and files. Unstructured data refers to loose material such as emails and tweets, and according to some estimates, may comprise 80% or even 90% of a company’s potentially useable information. Binary data refers to photographs and other media that is generally stored using binary formats.

In a recent podcast delivered by Mike Gualtieri, Principal Analyst, Forrester Research, Inc., and Milan Vaclavik, Senior Director & Solution Lead for CenturyLink Technology Solutions, it was noted that 70% of IT decision-makers see big data analytics as a priority within one year. This makes sense. For a company to fully understand where it is going, what its customers need, and how it compares to the marketplace, it must be able to access and use all of its data quickly and comprehensively. Currently, for most companies, this data is segmented into silos, with different storage mechanisms running on different platforms overseen by different people. So in a sense, Big Data at this moment in time, does not so much represent bits of information. The term better represents a concept, a problem, and a solution.

 

The concept highlights an awareness of just how much company-related information is out there to process, such as inventory, transactions, emails, images, software applications. The problem is in accepting the need to categorize, store and access this data at any time, without delay. The concept lies in the management of all of this data to a more sophisticated approach to its storage, access and use. What kind of data is Big Data used for?

Common use cases include:

  • Marketing campaign analysis
  • Data refining
  • Sentiment and social graph analysis
  • Customer churn analysis
  • Risk and fraud compliance
  • Real-time recommendations and offers
  • Customer experience analysis
  • Predictive analytics
  • Machine-generated data analysis.

When a company does not employ a sufficiently robust approach to managing its data, Vaclavik says, it gives way to a biased or inaccurate view of the business. Analytics, for example, the analysis of key data, often relies on a very small percentage of the entire data pool; only 12% on average, which is a wholly inaccurate method for understanding what is going on. The idea of using cloud infrastructure for data analytics is gaining traction with IT managers tasked with the challenges of analyzing large amounts of data from diverse sources

The key driver of big data, then, is a breakdown of the silos to allow for better cross-functional analysis. Big data specialists such as those that CenturyLink Technology Solutions, seek to set up a system that has four goals:

  • First, to capture and store all the data required for business functions.
  • Second, have a platform or solution to continuously integrate more data.
  • Third, to allow for continuous access.
  • Fourth, to allow insight, or understanding of the data itself.

If any of these layers are missing then the system does not work. In a sense big data represents an old problem, but one that is much larger today, thanks to the increased number of devices connected to the Internet, and the resultant explosion of information. It can be perceived more as an ecosystem that a new technology.

Infrastructure availability, scalability and reliability is critical and cloud increasingly is filling this need: IT managers need to focus on infrastructure that can scale elastically but not be overly complex to manage and secure; and it must offer high-performance computing with low latency; The cloud presents a compelling solution to this bundle of big data challenges.

Many organizations are turning to Hadoop, an open-source application for large-scale data processing, to form the centerpiece of the big data solution. Milan points out that although Hadoop is well-suited for managing big data at the data layer, it still must solve three major challenges that emerge:

  • The first is integration: moving into a modality in which traditional data silos are broken down.
  • The second is staff skills, since applications such as Hadoop require specific skill sets in order to be maintained and run properly.
  • Third is the inevitable rush on the market that occurs when a new data management platform works so well that it simultaneously increases demand for more data.

Ultimately, what Big Data comes down to is the consolidation, processing, and access to the information that drives a company. As world attention moves from gigabytes to petabytes and exabytes, the scope of operations expands logarithmically, forcing an enterprise-wide big data model to keep pace. And increasingly, for managing big data and big workloads, IT is turning to cloud vendors who offer a reliable, highly available infrastructure that can scale elastically without being overly complex to manage.

Big Data services available through CenturyLink can be found at: http://www.centurylinktechnology.com/big-data

By Steve Prentice

Post Sponsored By Century Link

Jeremy Cioara

Demand for Cloud and AI Skills Continues To Increase

Demand for Cloud Skills Increases Thinking about adding more cloud skills to your repertoire? Stop thinking. The time to do it is now. For IT professionals, cloud computing skills are becoming an essential resume item.  ...
David Friend

Tech Evolution – Why Multi-Cloud Will Win

Why Multi-Cloud Will Win When I was growing up in the 1970’s, IBM ruled the roost in corporate data centers. If you walked into a typical data center, nearly every piece of gear had an ...
Christian Buckley

The Evolution of SharePoint Customization

When I started working with SharePoint back in 2005, deploying WSS 2.0 followed and then SharePoint Portal Server 2003 for a large client, the concept of "customization" as we define it today was not really ...
Matt Holleran

Cloud Marketplaces Give Startups A Leg Up – Part 2

Cloud Marketplaces In my last post, Cloud Platforms, Marketplaces, and Startups Part One, I examined the proliferation of partner ecosystems within the cloud software business, beginning with Salesforce AppExchange. Here, we’ll look at how startups ...
Mor Cohen Tal1

The Top 2 Challenges of Next-Gen Applications

Challenges of Next-Gen Applications When you think of why customers move to the cloud, there are a few key things that they're trying to achieve. Agility How do I do more with less. How do ...
Garry Connolly

What’s Behind Smart Devices? A Data Centre, Of Course

Smart TV's, Smart Phones, What’s Behind Smart Devices? It’s not difficult to be “smart” these days. We wake up in the morning and check our social media feeds on our smart phone while turning on ...