Newsletter Subscribe

Bringing you thought leadership, news, infographics, resources and our own brand of comics each week to your inbox...

Passing Big Data Through A Drinking Straw

Passing Big Data Through A Drinking Straw

Big Data has all the corporate heads up and about in excitement since it promises to uncover golden nuggets of information out from an ocean of mundane and redundant data. But here’s the problem sticking everybody in the side, Big Data is big, as in it can reach the levels of “we-can’t-come-up-with-enough-names” bytes big. And with current upload speeds nowhere near as fast as download speeds, all the fancy analytics software and techniques aren’t going to do us any good if we can’t get our data where we need them.

bandwidth-straw

It is called the Skinny Straw or Drinking Straw problem and it is the biggest and most obvious problem being faced by Big Data. The analogy is simple; imagine passing an elephant through a drinking straw. Sure you can grind the elephant into very tiny bits so it can fit through the straw, but how long is that going to take? I admit that was a little gory, the real analogy was filling a swimming pool using a drinking straw, but you get the picture. The straw represents bandwidth and how small it is compared to the amount of data that needs to get to the other side of that straw.

The only real solution we can think of right off the bat is to get a bigger straw, but usually that would require major infrastructure upgrades on the part of the ISP or backbone provider, and we are talking about extreme amounts of cash (or credit if that’s how you roll). There are also the obvious technology limitations, we can upgrade to the best there is and it might not still be 100% enough. Some Big Data providers have tried their own proprietary ideas to try and get around this issue, or at least lessen it to some degree.

Here are some ways and techniques that are being used in the industry right now:

  1. We have the data compression and de-duplication techniques to make data transfers faster. That’s the “grinding the elephant and pushing it through the straw as fast as possible” solution.
  2. There is the “tinker with current protocols” direction by combining the reliability of TCP connections and the speed and bandwidth of UDP transfers into something that they call FASP. This ensures that communication is fast and secure while doing away with various handshaking processes that TCP requires.
  3. We can also work with various protocol optimizations in order to get around the problem. But one way that is really worth mentioning is the tried and tested transfer method –the old SneakerNet approach. Providers that use this method allow their customers to mail their hard drives to the company address so that they can transfer the data and then mail the hard drives back. This method is often faster at moving extremely large amounts of data quickly even taking into consideration the delivery time.

By Abdul Salam

(Image Source: ShutterStock)

About CloudBuzz

Daily tech news snapshots and insights from around the world...

View Website
Philips spotlights connected technology, predictive analytics software, and artificial intelligence advancing population health and precision medicine at HIMSS 2017 AMSTERDAM, Feb. 17, 2017 /PRNewswire/ -- Featuring new and enhanced connected health offerings at the 2017 HIMSS Conference & Exhibition (HIMSS17), Royal Philips (NYSE: PHG,AEX: PHIA), a global leader in health technology, will showcase a broad range of population health management, ...
Read More
Cupertino, California — Apple today announced its 28th annual Worldwide Developers Conference (WWDC) — hosting the world’s most talented developer community — will be held at the McEnery Convention Center in San Jose. The conference, kicking off June 5, will inspire developers from all walks of life to turn their passions into the next great innovations and apps that customers ...
Read More
When Cisco Systems Inc. reports earnings Wednesday, the big question will be if the networking giant’s repeated gambles on software can reverse a yearlong sales slide, or at least point to a reversal of that trend in the future. Cisco CSCO, +1.06%  is scheduled to report fiscal second-quarter earnings less than a month after announcing its latest multibillion-dollar software acquisition, ...
Read More
Offering Integrated and Automated Solutions, Expansive Partner Ecosystem, Advanced Architecture with Cross-Industry Collaboration SAN FRANCISCO, Feb. 14, 2017 – Today Intel Security outlined a new, unifying approach for the cybersecurity industry that strives to eliminate fragmentation through updated integrated solutions, new cross-industry partnerships and product integrations within the Intel Security Innovation Alliance and Cyber Threat Alliance (CTA). “Transforming isolated technologies ...
Read More
IoT Enablement, Analytics Offer Strong Monetisation Opportunities HAMPSHIRE, UNITED KINGDOM--(Marketwired - February 13, 2017) - A new study from Juniper Research has calculated that mobile network operators can realise an additional $85 billion in revenues over the next five years through the deployment and enhancement of non-core services including Big Data analytics and IoT (Internet of Things) enablement. Operators "Can ...
Read More