July 14, 2014

NoSQL databases, MapReduce & Hadoop

By Mojgan Afshari

Big Data – Productivity, Innovation And Competitiveness

NoSQL databases, MapReduce & Hadoop

Big data refers to datasets that are so large, diverse, and fast-changing which need advanced and unique storage, management, analysis, and visualization technologies.  According to McKinsey, Big Data is “the next frontier for innovation, competition and productivity”.  The right use of Big Data can increase productivity, innovation, and competitiveness for organizations. Inhi Suh, IBM vice president of big data, stated that businesses should place a greater emphasis on analytics projects. In fact, big data analytic is an important step to extract knowledge from a huge amount of data. It is a competitive advantage for most companies.

NoSQL databases, MapReduce & Hadoop

According to Gupta and Jyoti (2014), “Big data analytics is the process of analysing big data to find hidden patterns, unknown correlations and other useful information that can be extracted to make better decisions”.Agrawal et al. (2011) described the multiple phases in the big data analysis which are Data Acquisition and Recording; Information Extraction and Cleaning; Data Integration, Aggregation, and Representation; Data Modeling and Analysis; and Interpretation. All these phases are crucial and high accuracy in each of these steps will lead to effective big data analytic. In this way, the promised benefits of big data will be achieved.

A wide variety of analytical techniques and technologies can be used to extract useful information from large collections of data. Such information helps companies to gain valuable insights to predict customer behaviour, effective marketing, increased revenue and so on. Maltby (2011) reviewed several literatures on big data analytics and introduced several techniques, such as Machine learning, Data mining, Text analytics, Crowdsourcing, Cluster analysis, Time series analysis, Network analysis, Predictive modelling, Association rule, and Regression, that can be used to extract information from a data set and transform it into an understandable structure for further use . In fact, using data analytic techniques depends on the research objectives/ questions, nature of the data, and the available technologies.

Visualization products

In addition, there are a wide variety of software products and technologies to facilitate big data analytics. EDWs, Visualization products, NoSQL databases, MapReduce & Hadoop, and cloud computing are examples of the more common technologies used in big data analytics. All these techniques and technologies cannot be used for every project or organization. Needs and potential of each organization should be evaluated in order to choosing the appropriate tools for big data analytic.

Studies indicates that data analysis is considerably more challenging than simply locating, identifying, understanding, and citing data. Many researchers believe that the most of the challenges and concerns with data is related to volume and velocity. However, a recent survey conducted by the creator of open source computational database management system on more than 100 data scientist indicates that variety of data sources (not just data volume & velocity) is the main challenge in analysing data. Furthermore, results of this study indicated that Hadoop cannot be a viable solution for some cases that require complex analytics.  It would seem that data analysis is a clear bottleneck in many applications. In line with this idea, Agrawal and his colleagues (2011) reported common challenges in big data analysis: Heterogeneity and Incompleteness of data, Scale, Timeliness, Privacy, error-handling, lack of structure, and visualization. It is recommended that the highlighted challenges should be addressed for effective data analysis.

By Mojgan Afshari

Mojgan Afshari

Mojgan Afshari is a senior lecturer in the Department of Educational Management, Planning and Policy at the University of Malaya. She earned a Bachelor of Science in Industrial Applied Chemistry from Tehran, Iran. Then, she completed her Master’s degree in Educational Administration. After living in Malaysia for a few years, she pursued her PhD in Educational Administration with a focus on ICT use in education from the University Putra Malaysia. She currently teaches courses in managing change and creativity and statistics in education at the graduate level.
Randy

2024 Cloud Security Trends: Navigating the Evolving Landscape of Protection and Backup

2024 Cloud Security Trends Cloud protection and backup trends in 2024 are evolving rapidly, influenced [...]
Read more
Rakesh Soni

Cultivating a Culture of Cloud Innovation: Elevating Your Business Potential

Cloud computing has emerged as a game-changer, revolutionizing how organizations operate and transforming their growth [...]
Read more
finOps-tech

Cloud FinOps in the Age of AI: Key Trends

AI Era: Key Trends in Cloud FinOps Cloud FinOps goes beyond simply saving money on [...]
Read more
Lex Hegt

How Can Organizations Effectively Monitor and Analyze Their Azure Billing Data?

Monitor and Analyze Azure Billing Data With the ever-increasing investments in Azure, many organizations struggle [...]
Read more
Veljko

5 Recruiting Software Tools For Technology And Digital Companies

5 Recruiting Software Tools Finding the best candidate in the sea of thousands of developers [...]
Read more
Randy

Adapting to the Changing IT Landscape

The Rising Importance of Cloud Engineers The landscape of information technology and cloud computing is [...]
Read more

SPONSOR PARTNER

Explore top-tier education with exclusive savings on online courses from MIT, Oxford, and Harvard through our e-learning sponsor. Elevate your career with world-class knowledge. Start now!
© 2024 CloudTweaks. All rights reserved.