Understanding Big Data

Understanding Big Data

Understanding Big Data

What do Wal-Mart, Facebook and the Hadron Collider have in common? They are just three of many large organizations who are major consumers and processors of Big Data, a term that is becoming a greater priority for companies around the world as they struggle with a ceaseless and ever growing ocean of information.

The short definition of Big Data is that it represents all of the data in your organization – not just one type. Data resides in all business functions: marketing, finance, operations, research & development, customer experience – everywhere – and it essentially comes in three flavors: structured, unstructured and binary.

Structured data is the type of information that is organized and indexable, and consequently is most often stored in databases or annotated documents. This could include records and files. Unstructured data refers to loose material such as emails and tweets, and according to some estimates, may comprise 80% or even 90% of a company’s potentially useable information. Binary data refers to photographs and other media that is generally stored using binary formats.

In a recent podcast delivered by Mike Gualtieri, Principal Analyst, Forrester Research, Inc., and Milan Vaclavik, Senior Director & Solution Lead for CenturyLink Technology Solutions, it was noted that 70% of IT decision-makers see big data analytics as a priority within one year. This makes sense. For a company to fully understand where it is going, what its customers need, and how it compares to the marketplace, it must be able to access and use all of its data quickly and comprehensively. Currently, for most companies, this data is segmented into silos, with different storage mechanisms running on different platforms overseen by different people. So in a sense, Big Data at this moment in time, does not so much represent bits of information. The term better represents a concept, a problem, and a solution.

Mike Gualtieri - Forrester Research

The concept highlights an awareness of just how much company-related information is out there to process, such as inventory, transactions, emails, images, software applications. The problem is in accepting the need to categorize, store and access this data at any time, without delay. The concept lies in the management of all of this data to a more sophisticated approach to its storage, access and use. What kind of data is Big Data used for?

Common use cases include:

  • Marketing campaign analysis
  • Data refining
  • Sentiment and social graph analysis
  • Customer churn analysis
  • Risk and fraud compliance
  • Real-time recommendations and offers
  • Customer experience analysis
  • Predictive analytics
  • Machine-generated data analysis.

When a company does not employ a sufficiently robust approach to managing its data, Vaclavik says, it gives way to a biased or inaccurate view of the business. Analytics, for example, the analysis of key data, often relies on a very small percentage of the entire data pool; only 12% on average, which is a wholly inaccurate method for understanding what is going on. The idea of using cloud infrastructure for data analytics is gaining traction with IT managers tasked with the challenges of analyzing large amounts of data from diverse sources

The key driver of big data, then, is a breakdown of the silos to allow for better cross-functional analysis. Big data specialists such as those that CenturyLink Technology Solutions, seek to set up a system that has four goals:

  • First, to capture and store all the data required for business functions.
  • Second, have a platform or solution to continuously integrate more data.
  • Third, to allow for continuous access.
  • Fourth, to allow insight, or understanding of the data itself.

If any of these layers are missing then the system does not work. In a sense big data represents an old problem, but one that is much larger today, thanks to the increased number of devices connected to the Internet, and the resultant explosion of information. It can be perceived more as an ecosystem that a new technology.

Infrastructure availability, scalability and reliability is critical and cloud increasingly is filling this need: IT managers need to focus on infrastructure that can scale elastically but not be overly complex to manage and secure; and it must offer high-performance computing with low latency; The cloud presents a compelling solution to this bundle of big data challenges.

Many organizations are turning to Hadoop, an open-source application for large-scale data processing, to form the centerpiece of the big data solution. Milan points out that although Hadoop is well-suited for managing big data at the data layer, it still must solve three major challenges that emerge:

  • The first is integration: moving into a modality in which traditional data silos are broken down.
  • The second is staff skills, since applications such as Hadoop require specific skill sets in order to be maintained and run properly.
  • Third is the inevitable rush on the market that occurs when a new data management platform works so well that it simultaneously increases demand for more data.

Ultimately, what Big Data comes down to is the consolidation, processing, and access to the information that drives a company. As world attention moves from gigabytes to petabytes and exabytes, the scope of operations expands logarithmically, forcing an enterprise-wide big data model to keep pace. And increasingly, for managing big data and big workloads, IT is turning to cloud vendors who offer a reliable, highly available infrastructure that can scale elastically without being overly complex to manage.

Big Data services available through CenturyLink can be found at: http://www.centurylinktechnology.com/big-data

By Steve Prentice

Post Sponsored By Century Link

About Steve Prentice

Steve Prentice is a project manager, writer, speaker and expert on productivity in the workplace, specifically the juncture where people and technology intersect. He is a senior writer for CloudTweaks.

Find out more
View All Articles

2 Responses to Understanding Big Data

  1. I think it is beyond IT professionals now. Everyone is looking for Big Data Analytics solutions, specially analysts and they demand solutions that they can use with minimal IT involvement. That is why we see smaller and yet disruptive players emerge in this market. What you think?

  2. Hi Steve Prentice
    You shared some critical points about big data that were difficult to discuss. It helped me to increase knowledge about how to store big data. Your post are really nice and helpful to everyone. Thanks to share such useful information.

Comic
Fully Autonomous Cars: How’s It REALLY Going To Work?

Fully Autonomous Cars: How’s It REALLY Going To Work?

Pros and Cons and What the Experts Think Science fiction meets reality, and modern civilization is excitedly looking forward to the ubiquity of self-driving cars. However, an omnipresence of fully autonomous cars won’t happen as quickly as even some hopeful experts anticipate. While the autonomous car pros versus the cons race (See infographic discovered via…

The Lighter Side Of The Cloud – Bottlenecking

The Lighter Side Of The Cloud – Bottlenecking

By David Fletcher Please feel free to share our comics via social media networks such as Twitter, Facebook, LinkedIn, Instagram, Pinterest. Clear attribution (Twitter example: via @cloudtweaks) to our original comic sources is greatly appreciated.

Recent Articles - Posted by
Fintech Systems, Advancements and Investments

Fintech Systems, Advancements and Investments

Fintech Growth According to a recent report, global investment in fintech companies including both venture-backed and non-venture-backed businesses reached $9.4 billion in the second quarter of 2016; investment in venture capital-backed fintech startups, however, fell by 49%. Nevertheless, the Pulse of Fintech, published jointly by KPMG International and CB Insights, suggests venture capital investment in…

How Identity Governance Can Secure The Cloud Enterprise

How Identity Governance Can Secure The Cloud Enterprise

Securing The Cloud Enterprise Cloud adoption is accelerating for most enterprises, and cloud computing is becoming an integral part of enterprise IT and security infrastructure. Based on current adoption trends, it’s clear that the vast majority of new applications purchased by organizations will be SaaS applications. The allure is evident, from cost savings to speed…

Data Breaches: Incident Response Planning – Part 1

Data Breaches: Incident Response Planning – Part 1

Incident Response Planning – Part 1 The topic of cybersecurity has become part of the boardroom agendas in the last couple of years, and not surprisingly — these days, it’s almost impossible to read news headlines without noticing yet another story about a data breach. As cybersecurity shifts from being a strictly IT issue to…

Your Biggest Data Security Threat Could Be….

Your Biggest Data Security Threat Could Be….

Paying Attention To Data Security Your biggest data security threat could be sitting next to you… Data security is a big concern for businesses. The repercussions of a data security breach ranges from embarrassment, to costly lawsuits and clean-up jobs – particularly when confidential client information is involved. But although more and more businesses are…

Cloud-based GRC Intelligence Supports Better Business Performance

Cloud-based GRC Intelligence Supports Better Business Performance

Cloud-based GRC Intelligence All businesses need a strategy and processes for governance, risk and compliance (GRC). Many still view GRC activity as a burdensome ‘must-do,’ approaching it reactively and managing it with non-specialized tools. GRC is a necessary business endeavor but it can be elevated from a cost drain to a value-add activity. By integrating…

How You Can Improve Customer Experience With Fast Data Analytics

How You Can Improve Customer Experience With Fast Data Analytics

Fast Data Analytics In today’s constantly connected world, customers expect more than ever before from the companies they do business with. With the emergence of big data, businesses have been able to better meet and exceed customer expectations thanks to analytics and data science. However, the role of data in your business’ success doesn’t end…

Four Recurring Revenue Imperatives

Four Recurring Revenue Imperatives

Revenue Imperatives “Follow the money” is always a good piece of advice, but in today’s recurring revenue-driven market, “follow the customer” may be more powerful. Two recurring revenue imperatives highlight the importance of responding to, and cherishing customer interactions. Technology and competitive advantage influence the final two. If you’re part of the movement towards recurring…

Cloud Infographic – Disaster Recovery

Cloud Infographic – Disaster Recovery

Disaster Recovery Business downtime can be detrimental without a proper disaster recovery plan in place. Only 6% of businesses that experience downtime without a plan will survive long term. Less than half of all businesses that experience a disaster are likely to reopen their doors. There are many causes of data loss and downtime —…

Cloud Computing Is Greener Than You Think

Cloud Computing Is Greener Than You Think

Cloud Computing Is Greener Than You Think Last week we touched upon how a project in Finland had blended two of the world’s most important industries, cloud computing and green technology, to produce a data centre that used nearby sea water to both cool their servers and heat local homes.  Despite such positive environmental projects, there…

How Your Startup Can Benefit From Cloud Computing And Growth Hacking

How Your Startup Can Benefit From Cloud Computing And Growth Hacking

Ambitious Startups An oft-quoted statistic, 50% of new businesses fail within five years. And the culling of startups is even more dramatic, with an estimated nine out of ten folding. But to quote Steve Jobs, “I’m convinced that about half of what separates the successful entrepreneurs from the non-successful ones is pure perseverance.” So while…

5 Reasons Why Your Startup Will Grow Faster In The Cloud

5 Reasons Why Your Startup Will Grow Faster In The Cloud

Cloud Startup Fast-tracking Start-ups face many challenges, the biggest of which is usually managing growth. A start-up that does not grow is at constant risk of failure, whereas a new business that grows faster than expected may be hindered by operational constraints, such as a lack of staff, workspace and networks. It is an unfortunate…

Cloud Computing – The Real Story Is About Business Strategy, Not Technology

Cloud Computing – The Real Story Is About Business Strategy, Not Technology

Enabling Business Strategies The cloud is not really the final destination: It’s mid-2015, and it’s clear that the cloud paradigm is here to stay. Its services are growing exponentially and, at this time, it’s a fluid model with no steady state on the horizon. As such, adopting cloud computing has been surprisingly slow and seen more…

Cloud Infographic – Big Data Analytics Trends

Cloud Infographic – Big Data Analytics Trends

Big Data Analytics Trends As data information and cloud computing continues to work together, the need for data analytics continues to grow. Many tech firms predict that big data volume will grow steadily 40% per year and in 2020, will grow up to 50 times that. This growth will also bring a number of cost…