The Lighter Side Of The Cloud – Keys To The Car
The Lighter Side Of The Cloud – Expendable
The Lighter Side Of The Cloud – Data Breach
Passing Big Data Through A Drinking Straw

Passing Big Data Through A Drinking Straw

Passing Big Data Through A Drinking Straw

Big Data has all the corporate heads up and about in excitement since it promises to uncover golden nuggets of information out from an ocean of mundane and redundant data. But here’s the problem sticking everybody in the side, Big Data is big, as in it can reach the levels of “we-can’t-come-up-with-enough-names” bytes big. And with current upload speeds nowhere near as fast as download speeds, all the fancy analytics software and techniques aren’t going to do us any good if we can’t get our data where we need them.

bandwidth-straw

It is called the Skinny Straw or Drinking Straw problem and it is the biggest and most obvious problem being faced by Big Data. The analogy is simple; imagine passing an elephant through a drinking straw. Sure you can grind the elephant into very tiny bits so it can fit through the straw, but how long is that going to take? I admit that was a little gory, the real analogy was filling a swimming pool using a drinking straw, but you get the picture. The straw represents bandwidth and how small it is compared to the amount of data that needs to get to the other side of that straw.

The only real solution we can think of right off the bat is to get a bigger straw, but usually that would require major infrastructure upgrades on the part of the ISP or backbone provider, and we are talking about extreme amounts of cash (or credit if that’s how you roll). There are also the obvious technology limitations, we can upgrade to the best there is and it might not still be 100% enough. Some Big Data providers have tried their own proprietary ideas to try and get around this issue, or at least lessen it to some degree.

Here are some ways and techniques that are being used in the industry right now:

  1. We have the data compression and de-duplication techniques to make data transfers faster. That’s the “grinding the elephant and pushing it through the straw as fast as possible” solution.
  2. There is the “tinker with current protocols” direction by combining the reliability of TCP connections and the speed and bandwidth of UDP transfers into something that they call FASP. This ensures that communication is fast and secure while doing away with various handshaking processes that TCP requires.
  3. We can also work with various protocol optimizations in order to get around the problem. But one way that is really worth mentioning is the tried and tested transfer method –the old SneakerNet approach. Providers that use this method allow their customers to mail their hard drives to the company address so that they can transfer the data and then mail the hard drives back. This method is often faster at moving extremely large amounts of data quickly even taking into consideration the delivery time.

By Abdul Salam

(Image Source: ShutterStock)

Abdul

Abdul Salam is IT professional and an accomplished technical writer with CloudTweaks. He earned his undergraduate degree in Information Technology followed by a postgraduate degree in Business Informatics. Abdul possess over 3 years’ experience in technical & business writing with deep knowledge in Cloud Computing, VMware,Oracle, Oracle ERP, Cloud ERP, Microsoft Technologies and Network Communications (Cisco, Juniper). Visit his LinkedIn profile at: http://linkd.in/TtFu7X

Sorry, comments are closed for this post.

Popular Archives

PaaS vs Docker – why is it such a heated debate?

PaaS vs Docker – why is it such a heated debate?

PaaS vs Docker Docker started as just a software container on top of a Linux operating system which seemed like a simple optimization for a fat hypervisor. Its disruptive force however comes from the fact that it does force us to rethink many of the layers of the cloud stack. Starting from the way we…

Cloud Infographic – The Internet Of Things In 2020

Cloud Infographic – The Internet Of Things In 2020

The Internet Of Things In 2020 The growing interest in the Internet of Things is amongst us and there is much discussion. Attached is an archived but still relevant infographic by Intel which has produced a memorizing snapshot at how the number of connected devices have exploded since the birth of the Internet and PC.…

Five Cloud Questions Every CIO Needs To Know How To Answer

Five Cloud Questions Every CIO Needs To Know How To Answer

The Hot Seat Five cloud questions every CIO needs to know how to answer The cloud is a powerful thing, but here in the CloudTweaks community, we already know that. The challenge we have is validating the value it brings to today’s enterprise. Below, let’s review five questions we need to be ready to address…

Cloud Infographic – The Future (IoT)

Cloud Infographic – The Future (IoT)

The Future (IoT) By the year 2020, it is being predicted that 40 to 80 billion connected devices will be in use. The Internet of Things or IoT will transform your business and home in many truly unbelievable ways. The types of products and services that we can expect to see in the next decade…

Five Reasons SMBs Fear The Cloud

Five Reasons SMBs Fear The Cloud

Five Reasons SMBs Fear the Cloud Fear of the cloud has been around since the Cloud began. SMBs were traditionally afraid of security issues, while large companies fretted about increasing the complexity of their IT infrastructure. What many budding start-up companies don’t realise is Cloud Computing helps place them on a level playing field with…

Recent

Cloud Security Hottest Issue At RSA

Cloud Security Hottest Issue At RSA

Cloud Security Hottest Issue The integral integration of cyber security and cloud technology seemed to be the hottest issue at the busy RSA 2015 Conference in San Francisco. Interested parties packed security and cloud service booths for the duration of the conference. Several prominent publications covered the increased importance of securing their private information that’s…

Imperfect Security: The RSA Conference And The Illusion Of Safety

Imperfect Security: The RSA Conference And The Illusion Of Safety

The RSA Conference And The Illusion Of Safety This year’s 2015 RSA Conference is taking place from April 20th to 24th, in San Francisco, California. Here, security leaders from across the vast expanse of tech, politics, and more will gather to discuss the past, present, and future of security. From application security to technology infrastructure,…

The Lighter Side Of The Cloud – Day 5

The Lighter Side Of The Cloud – Day 5

By David Fletcher Are you looking to supercharge your Newsletter, Powerpoint presentation, Social media campaign or Website? Our universally recognized tech related comics can help you. Contact us for information on our commercial licensing rates. About Latest Posts AbdulAbdul Salam is IT professional and an accomplished technical writer with CloudTweaks. He earned his undergraduate degree…

Contact Us

Sending

Technology Sponsors

hp Logo CityCloud-PoweredByOpenstack-Bluesquare_logo_100x100-01
cisco_logo_100x100 vmware citrix100
Site 24x7 200px-KPMG

Established in 2009, CloudTweaks is recognized as one of the leading influencers in cloud computing, big data and internet of things (IoT) information. Our goal is to continue to build our growing information portal, by providing the best in-depth articles, interviews, event listings, whitepapers, infographics and much more.

CloudTweaks Comic Library

Advertising