Taming Big Data
The cookie monster in the long running television show ‘Sesame Street” is famous for his big appetite and famous for the words “Me want cookie!”, “Me eat cookie!” The Big Data Monster, who always seems hungry for data says: “Me want data!”, “Me want data!” Today, we have all the statistics in the world regarding the amount of Big Data being generated every second, minute, day, and year. While for IT professionals, it may be easier to manage, handle, and contain this Monster, from a customer’s perspective, there may be several challenges ahead.
As a customer, how do I handle this Monster? In order to get the most out of this uncontrollable beast, it is important to identify and establish clear goals and objectives, that is, what is it that your organization is trying to solve or achieve. For instance, the Balanced ScoreCard tool by Kaplan and Norton may be used to first identify and establish objectives in lieu of the strategic mission and vision of the organization. Once these objectives are identified and established, the next step would be to ascertain a better understanding of customer buying patterns and behaviors, keeping track of competition, purchasing requirements and patterns, and impact on financials. Moreover, the defined objectives serve as a roadmap for the identification of relevant data sourcing to generate new insights for evidence-based decision making by senior management team members.
The idea is simple: For the most part, you probably cannot control the monster from having fed with data, however, what you can do is ensure effective digestion of data in order to avoid having the Monster get an upset stomach! The consequences of which may result in poor performance and analytics as well as increased costs. The above notion seems easier said than done, especially from a customer’s point of view. The reason being that there are numerous challenges faced by the customer while handling Big Data Monster.
The Top 2 Challenges Faced Are:
The first and foremost is the availability of experts in this field. IT professionals can establish the entire infrastructure utilizing state-of-the-art hardware, software, business intelligence tools, or make use of cloud computing technologies. More importantly, though, is having business and statistical analysis experts as part of your team. The combination of the two skillsets can effectively allow you to put a rope around the monster and ensure optimal efficiency and maintain cost control.
The second challenge is handling unstructured data sets. Simply defined, unstructured information refers to information that either does not have a pre-defined data model or is not organized in a predefined manner. Unstructured information is typically text-heavy, but may contain data such as dates, numbers, and facts as well. Essentially the big challenge is to ensure aligning and organizing data in line of your strategic objectives as discussed above. Effective decision making requires making sense of the real-time data being generated every second of every day, whether from manufacturing and supply chains or data belonging to social data streams. For example, in Fast-moving Consumer Goods (FMCG) industries, the ability to make the right decisions in real-time yields real competitive advantage.
Thus, the Big Data Monster can be effectively controlled as well as tamed by establishing and deploying a successful and functional team of IT and business experts, equipping them with clearly defined business objectives.
(Image Credit: Harvard Business Review)
By Syed Raza