Understanding Big Data
What do Wal-Mart, Facebook and the Hadron Collider have in common? They are just three of many large organizations who are major consumers and processors of Big Data, a term that is becoming a greater priority for companies around the world as they struggle with a ceaseless and ever growing ocean of information.
The short definition of Big Data is that it represents all of the data in your organization – not just one type. Data resides in all business functions: marketing, finance, operations, research & development, customer experience – everywhere – and it essentially comes in three flavors: structured, unstructured and binary.
Structured data is the type of information that is organized and indexable, and consequently is most often stored in databases or annotated documents. This could include records and files. Unstructured data refers to loose material such as emails and tweets, and according to some estimates, may comprise 80% or even 90% of a company’s potentially useable information. Binary data refers to photographs and other media that is generally stored using binary formats.
In a recent podcast delivered by Mike Gualtieri, Principal Analyst, Forrester Research, Inc., and Milan Vaclavik, Senior Director & Solution Lead for CenturyLink Technology Solutions, it was noted that 70% of IT decision-makers see big data analytics as a priority within one year. This makes sense. For a company to fully understand where it is going, what its customers need, and how it compares to the marketplace, it must be able to access and use all of its data quickly and comprehensively. Currently, for most companies, this data is segmented into silos, with different storage mechanisms running on different platforms overseen by different people. So in a sense, Big Data at this moment in time, does not so much represent bits of information. The term better represents a concept, a problem, and a solution.
The concept highlights an awareness of just how much company-related information is out there to process, such as inventory, transactions, emails, images, software applications. The problem is in accepting the need to categorize, store and access this data at any time, without delay. The concept lies in the management of all of this data to a more sophisticated approach to its storage, access and use. What kind of data is Big Data used for?
Common use cases include:
- Marketing campaign analysis
- Data refining
- Sentiment and social graph analysis
- Customer churn analysis
- Risk and fraud compliance
- Real-time recommendations and offers
- Customer experience analysis
- Predictive analytics
- Machine-generated data analysis.
When a company does not employ a sufficiently robust approach to managing its data, Vaclavik says, it gives way to a biased or inaccurate view of the business. Analytics, for example, the analysis of key data, often relies on a very small percentage of the entire data pool; only 12% on average, which is a wholly inaccurate method for understanding what is going on. The idea of using cloud infrastructure for data analytics is gaining traction with IT managers tasked with the challenges of analyzing large amounts of data from diverse sources
The key driver of big data, then, is a breakdown of the silos to allow for better cross-functional analysis. Big data specialists such as those that CenturyLink Technology Solutions, seek to set up a system that has four goals:
- First, to capture and store all the data required for business functions.
- Second, have a platform or solution to continuously integrate more data.
- Third, to allow for continuous access.
- Fourth, to allow insight, or understanding of the data itself.
If any of these layers are missing then the system does not work. In a sense big data represents an old problem, but one that is much larger today, thanks to the increased number of devices connected to the Internet, and the resultant explosion of information. It can be perceived more as an ecosystem that a new technology.
Infrastructure availability, scalability and reliability is critical and cloud increasingly is filling this need: IT managers need to focus on infrastructure that can scale elastically but not be overly complex to manage and secure; and it must offer high-performance computing with low latency; The cloud presents a compelling solution to this bundle of big data challenges.
Many organizations are turning to Hadoop, an open-source application for large-scale data processing, to form the centerpiece of the big data solution. Milan points out that although Hadoop is well-suited for managing big data at the data layer, it still must solve three major challenges that emerge:
- The first is integration: moving into a modality in which traditional data silos are broken down.
- The second is staff skills, since applications such as Hadoop require specific skill sets in order to be maintained and run properly.
- Third is the inevitable rush on the market that occurs when a new data management platform works so well that it simultaneously increases demand for more data.
Ultimately, what Big Data comes down to is the consolidation, processing, and access to the information that drives a company. As world attention moves from gigabytes to petabytes and exabytes, the scope of operations expands logarithmically, forcing an enterprise-wide big data model to keep pace. And increasingly, for managing big data and big workloads, IT is turning to cloud vendors who offer a reliable, highly available infrastructure that can scale elastically without being overly complex to manage.
Big Data services available through CenturyLink can be found at: http://www.centurylinktechnology.com/big-data
By Steve Prentice
Post Sponsored By Century Link
