Passing Big Data Through A Drinking Straw

Passing Big Data Through A Drinking Straw

Passing Big Data Through A Drinking Straw

Big Data has all the corporate heads up and about in excitement since it promises to uncover golden nuggets of information out from an ocean of mundane and redundant data. But here’s the problem sticking everybody in the side, Big Data is big, as in it can reach the levels of “we-can’t-come-up-with-enough-names” bytes big. And with current upload speeds nowhere near as fast as download speeds, all the fancy analytics software and techniques aren’t going to do us any good if we can’t get our data where we need them.

bandwidth-straw

It is called the Skinny Straw or Drinking Straw problem and it is the biggest and most obvious problem being faced by Big Data. The analogy is simple; imagine passing an elephant through a drinking straw. Sure you can grind the elephant into very tiny bits so it can fit through the straw, but how long is that going to take? I admit that was a little gory, the real analogy was filling a swimming pool using a drinking straw, but you get the picture. The straw represents bandwidth and how small it is compared to the amount of data that needs to get to the other side of that straw.

The only real solution we can think of right off the bat is to get a bigger straw, but usually that would require major infrastructure upgrades on the part of the ISP or backbone provider, and we are talking about extreme amounts of cash (or credit if that’s how you roll). There are also the obvious technology limitations, we can upgrade to the best there is and it might not still be 100% enough. Some Big Data providers have tried their own proprietary ideas to try and get around this issue, or at least lessen it to some degree.

Here are some ways and techniques that are being used in the industry right now:

  1. We have the data compression and de-duplication techniques to make data transfers faster. That’s the “grinding the elephant and pushing it through the straw as fast as possible” solution.
  2. There is the “tinker with current protocols” direction by combining the reliability of TCP connections and the speed and bandwidth of UDP transfers into something that they call FASP. This ensures that communication is fast and secure while doing away with various handshaking processes that TCP requires.
  3. We can also work with various protocol optimizations in order to get around the problem. But one way that is really worth mentioning is the tried and tested transfer method –the old SneakerNet approach. Providers that use this method allow their customers to mail their hard drives to the company address so that they can transfer the data and then mail the hard drives back. This method is often faster at moving extremely large amounts of data quickly even taking into consideration the delivery time.

By Abdul Salam

(Image Source: ShutterStock)

About Abdul

Abdul is a senior consultant with Energy Services, and author of numerous blogs, books, white papers, and tutorials on cloud computing and accomplished technical writer with CloudTweaks. He earned his bachelor’s degree in Information Technology, followed by an MBA-IT degree and certifications by Cisco and Juniper Networks.

He has recently co-authored: Deploying and Managing a Cloud Infrastructure: Real-World Skills for the CompTIA Cloud+ Certification (Wiley).

View All Articles

Sorry, comments are closed for this post.

The Cloud Showdown: How To Determine The Best Cloud Solution For Your Business

The Cloud Showdown: How To Determine The Best Cloud Solution For Your Business

The Cloud Showdown For small and medium businesses (SMB) seeking to grow in 2016, options for increasing efficiency abound. The Cloud offers many benefits for businesses, including slashing IT expenses, providing a more efficient and reliable way to store and back up data, and facilitating collaboration among employees. In fact, 39 percent of SMBs claim…

The Open Performance Grid: Utopia Comes To Tech

The Open Performance Grid: Utopia Comes To Tech

The Open Performance Grid “For what greater wealth can there be than cheerfulness, peace of mind, and freedom from anxiety?” So wrote the English author and philosopher Sir Thomas More in his novel, Utopia, almost 500 years ago. More’s utopian dreams continue to thrive in one guise or another to the present day; in the…

Risks Of Virtualization In Public And Private Clouds

Risks Of Virtualization In Public And Private Clouds

Risks Of Virtualization Server virtualization is one of the cornerstone technologies of the cloud. The ability to create multiple virtual servers that can run any operating system on a single physical server has a lot of advantages. These advantages can appear in public, Infrastructure as a Service (IaaS), as well as in private clouds. However,…

The True Power And Potential Of Wearables

The True Power And Potential Of Wearables

The Power of Wearables The potential of wearable computing for efficiency and innovation is enormous. For business leaders and CIOs, wearable technology represents an opportunity to find more intelligent solutions to real problems and to leverage these devices for the benefit of the organization. But before wearables can successfully realize these expectations, there remain basic…

Bimodal IT Is Not The Death of Traditional IT

Bimodal IT Is Not The Death of Traditional IT

Not The Death of Traditional IT If you are a CIO, no doubt you are getting pressure from your line of business managers to provide a more agile IT environment so they can take advantage of faster, easier to use cloud applications. If you’re not providing those services, they are probably using them anyway and…

New Smartphones From Apple, Samsung and HTC Promise To Light Up 2016

New Smartphones From Apple, Samsung and HTC Promise To Light Up 2016

New Smartphones from Apple, Samsung and HTC (Sponsored post courtesy of Verizon Wireless) The launch of the Galaxy S7 Edge at the Mobile World Congress in Barcelona during February was the first shot in a vintage year for mobile phones. The S7 is an incredible piece of hardware, but launches from HTC and Apple later in the…

Featured Sponsored Articles
How Successful Businesses Ensure Quality Team Communication

How Successful Businesses Ensure Quality Team Communication

Quality Team Communication  (Sponsored post courtesy of Hubgets) Successful team communication and collaboration are as vital to project and overall business success as the quality of products and services an organization develops. We rely on a host of business tools to ensure appropriate customer interactions, sound product manufacturing, and smooth back-end operations. However, the interpersonal relationships…

Featured Sponsored Articles
How To Develop A Business Continuity Plan Using Internet Performance Management

How To Develop A Business Continuity Plan Using Internet Performance Management

Internet Performance Management Planning CDN Performance Series Provided By Dyn In our previous post, we laid out the problems of business continuity and Internet Performance Management in today’s online environment.  In this article, we will take a look at some of the ways you can use traffic steering capabilities to execute business continuity planning and…

Featured Sponsored Articles

CloudTweaks is recognized as one of the leading influencers in cloud computing, infosec, big data and the internet of things (IoT) information. Our goal is to continue to build our growing information portal by providing the best in-depth articles, interviews, event listings, whitepapers, infographics and much more.

Sponsor