Potential Powerful Pitfalls – Big Data, Big Trouble

Potential Powerful Pitfalls – Big Data, Big Trouble

Potential Powerful Pitfalls – Big Data, Big Trouble

Information has always been a great source of power. Kingdoms were often won and lost through scout reports, misinformation and treachery. Even now we fear those that hold too much of it and often cry foul in the name of privacy. But if you asked me, as a collective social generation, we have forsaken our privacy willingly so we should all just calm down about it.

Data Breach Comic

Traditionally, we have collected only relevant data and stored those according to some form of sorting process so that we can make sense of all of them. For example if we wanted to know how many babies are born in Manhattan in a day we would only need to collect data from hospitals and clinics within Manhattan. This is the traditional way of collecting data because of the limited capacities of our data systems. But Big Data aims to change all of that, it is not about just collecting specific data like the number of births in Manhattan, but rather it wants to collect from each hospital from each state and the whole world as well, and not just regarding births. This can provide us with any information we need and allow us to predict future trends more accurately, or so we thought.

But Big Data is prone to some very powerful pitfalls. One is called the ‘lottery paradox,’ here we tend to give emphasis on something that is very improbable to happen to us simply because of the payoff just like in a lottery. There will always be a winner, and though chances are 175,000,000 to 1, we still hold on to the chance that we get to be that “1”. How this applies to big data is simple: the larger the data sets are the smaller the chance it is to find that piece of important information that gives out the biggest payoff. These small gems within the vast sea of data increases in number but does not increase in frequency, we simply find more of them the more data we have. So we then invest more in finding these gems, these “outrageous events”, and when we fail to find what we expect, we simply wonder if it was all worth the effort and money.

Another pitfall according to ZapThink is the ‘more is better’ paradox which is the assumption that if a certain quantity of data is good, then more of it is better. But this is not necessarily true as we might simply be encouraging the collection of more irrelevant and redundant data. Then we rely on services like Hadoop to make sense of all that chaff. But the truth is no process or software is ever capable of making sense of everything. Big Data is not about being selective in the collection process, a very big downside if you ask me.

By Abdul Salam

Sorry, comments are closed for this post.

Comic
Using Cloud Technology In The Education Industry

Using Cloud Technology In The Education Industry

Education Tech and the Cloud Arguably one of society’s most important functions, teaching can still seem antiquated at times. Many schools still function similarly to how they did five or 10 years ago, which is surprising considering the amount of technical innovation we’ve seen in the past decade. Education is an industry ripe for innovation…

5% Of Companies Have Embraced The Digital Innovation Fostered By Cloud Computing

5% Of Companies Have Embraced The Digital Innovation Fostered By Cloud Computing

Embracing The Cloud We love the stories of big complacent industry leaders having their positions sledge hammered by nimble cloud-based competitors. Saleforce.com chews up Oracle’s CRM business. Airbnb has a bigger market cap than Marriott. Amazon crushes Walmart (and pretty much every other retailer). We say: “How could they have not seen this coming?” But, more…

What Futuristic Tech Will You See In Your Lifetime?

What Futuristic Tech Will You See In Your Lifetime?

Futuristic Tech The world and what people can do is increasingly being driven by technology. It has already shaped the world we live in, but over the next few decades it is set to shape the world in ways that we can barely imagine. There have already been some great leaps in IoT technology recently,…

The Lighter Side Of The Cloud – Hiding Spots

The Lighter Side Of The Cloud – Hiding Spots

By David Fletcher Please feel free to share our comics via social media networks such as Twitter, Facebook, LinkedIn, Instagram, Pinterest. Clear attribution (Twitter example: via@cloudtweaks) to our original comic sources is greatly appreciated.

Recent Articles - Posted by
Fintech Exploiting AI and Blockchain Technology

Fintech Exploiting AI and Blockchain Technology

AI and Blockchain Technology The field of artificial intelligence (AI) had progressed rapidly in the last ten years, though first recognized in the 1950s. From autonomous motor vehicles to digital personal assistants, the technology is making its way into a variety of industries, enabling better task automation, language processing, and data analytics. But more recently,…

Are Cloud Solutions Secure Enough Out-of-the-box?

Are Cloud Solutions Secure Enough Out-of-the-box?

Out-of-the-box Cloud Solutions Although people may argue that data is not safe in the Cloud because using cloud infrastructure requires trusting another party to look after mission critical data, cloud services actually are more secure than legacy systems. In fact, a recent study on the state of cloud security in the enterprise market revealed that…

Are CEO’s Missing Out On Big Data’s Big Picture?

Are CEO’s Missing Out On Big Data’s Big Picture?

Big Data’s Big Picture Big data allows marketing and production strategists to see where their efforts are succeeding and where they need some work. With big data analytics, every move you make for your company can be backed by data and analytics. While every business venture involves some level of risk, with big data, that risk…

Cloud-based GRC Intelligence Supports Better Business Performance

Cloud-based GRC Intelligence Supports Better Business Performance

Cloud-based GRC Intelligence All businesses need a strategy and processes for governance, risk and compliance (GRC). Many still view GRC activity as a burdensome ‘must-do,’ approaching it reactively and managing it with non-specialized tools. GRC is a necessary business endeavor but it can be elevated from a cost drain to a value-add activity. By integrating…

Protecting Devices From Data Breach: Identity of Things (IDoT)

Protecting Devices From Data Breach: Identity of Things (IDoT)

How to Identify and Authenticate in the Expanding IoT Ecosystem It is a necessity to protect IoT devices and their associated data. As the IoT ecosystem continues to expand, the need to create an identity to newly-connected things is becoming increasingly crucial. These ‘things’ can include anything from basic sensors and gateways to industrial controls…

The Cancer Moonshot: Collaboration Is Key

The Cancer Moonshot: Collaboration Is Key

Cancer Moonshot In his final State of the Union address in January 2016, President Obama announced a new American “moonshot” effort: finding a cure for cancer. The term “moonshot” comes from one of America’s greatest achievements, the moon landing. If the scientific community can achieve that kind of feat, then surely it can rally around…

Using Big Data To Make Cities Smarter

Using Big Data To Make Cities Smarter

Using Big Data To Make Cities Smarter The city of the future is impeccably documented. Sensors are used to measure air quality, traffic patterns, and crowd movement. Emerging neighborhoods are quickly recognized, public safety threats are found via social networks, and emergencies are dealt with quicklier. Crowdsourcing reduces commuting times, provides people with better transportation…

Are Women Discriminated Against In The Tech Sector?

Are Women Discriminated Against In The Tech Sector?

Women Discriminated Against In Tech Sector It is no secret that the tech industry is considered sexist since most women are paid less than men; there are considerably fewer women in tech jobs; and generally men get promoted above women. Yet the irony is twofold. Firstly, there is an enormous demand for employees with skills…

Cloud Infographic – Cloud Public, Private & Hybrid Differences

Cloud Infographic – Cloud Public, Private & Hybrid Differences

Cloud Public, Private & Hybrid Differences Many people have heard of cloud computing. There is however a tremendous number of people who still cannot differentiate between Public, Private & Hybrid cloud offerings.  Here is an excellent infographic provided by the group at iWeb which goes into greater detail on this subject. Infographic source: iWeb

Disaster Recovery – A Thing Of The Past!

Disaster Recovery – A Thing Of The Past!

Disaster Recovery  Ok, ok – I understand most of you are saying disaster recovery (DR) is still a critical aspect of running any type of operations. After all – we need to secure our future operations in case of disaster. Sure – that is still the case but things are changing – fast. There are…

12 Promising Business Intelligence (BI) Services For Your Company

12 Promising Business Intelligence (BI) Services For Your Company

12 Promising Business Intelligence (BI) Services Business Intelligence (BI) services have recently seen an explosion of innovation and choices for business owners and entrepreneurs. So many choices, in fact, that many companies aren’t sure which business intelligence company to use. To help offer you a solution, we’ve compiled a list of 12 Business Intelligence companies…

Cloud Computing – The Game Changer

Cloud Computing – The Game Changer

Global Cloud Index In October, Cisco released its Global Cloud Index (GCI) report for 2014-2019, projecting a near 3-fold growth of global data center traffic, with predictions that this traffic will reach 8.6 zettabytes (cloud data center traffic) and 10.4 zettabytes (total data center traffic) per year in 2019 and 80% of it will come…