Traditional industries like mining have been slow to adapt to changing IP technology. Of course, coal and other mining types have adopted new technologies starting with mechanical drills powered by pistons, then compressed air in the Industrial Revolution to today’s newly-developed machines used for grinding and crushing can extract minerals from the earth with less energy than ever before. However, the adoption of IP in mining is in its formative years. And, in the last few years they have realized significant ROI through investing in new IP technologies.
Mining is a complex, global business with multiple stages of operation. New machinery enables massive amounts of data collection across the entire value chain starting from ore extraction, hauling, processing to beneficiation. The processes are often in a state of continual operation and require fast decision making. The capital-intensive nature of the industry means that even small inefficiencies and unanticipated interruptions can have significant financial impact and opportunity costs. Availability of actionable insights based on real time data analysis is a big factor in ensuring optimal equipment uptime, utilization and process productivity.
Legacy siloed systems are relatively slow and expensive to maintain and make it challenging to derive actionable insights from stored data, as they are fragmented and not easily accessible. On the other hand, companies with a pragmatic cloud strategy can benefit from the ability to bring together data from various sources and generate key business insights to drive strong competitive advantage. This can also help the company future-proof their business processes and decision making.
This has led executive teams at mining corporations to increasingly rely on real time data, from the equipment, to make quick decisions related to Health & Safety, Worker (or Driver) Productivity, and Equipment Utilization. Data analytics finds application across almost every step of the value chain, whether in predicting failure points, capturing inefficiencies, or managing costs and profitability.
For instance, a U.S. based Fortune 500 international mining company was facing challenges in generating actionable operational insights. With a network of 300+ trucks, with 200 sensors per truck, streaming petabytes of data from 30+ global sites, the company’s reliance on siloed on-premise systems resulted in analytical insights lagging behind its needs to drive efficiency and business growth. With much of the company’s decision making impacted, it was time to implement a strategy to unlock data for real time decision making.
Most of the company’s petabytes of data sat in a legacy on-premise data warehouse (DW) hosted on Teradata (an established leader in data warehousing) alongside Hadoop. The existing data warehouse was expensive and slow in processing the vast amount of IoT data generated from haul trucks at its sites. This resulted in multiple issues, as processing speed was essential to managing important decisions in real time, optimizing operations, and adapting to business fluctuations.
While the need to move to the cloud to manage, store and analyze its data was clear, they also had concerns about the cloud migration process. Moving workloads to the hybrid cloud (rehosting) could expose the company to potential data loss which could force it to repatriate data back to on-premise legacy systems.
The first step for the company in its cloud migration journey was to get a better understanding of its current environment and landscape. It approached this by taking a questionnaire-based cloud readiness assessment (based on TDWI principles) to help identify the best fit-for-purpose cloud stack that could form the basis of the larger solution.
Key assessment areas included:
With the insights gained from the assessment, the company identified the following alternative solutions for migrating its data platform: 1. Azure SQL Datawarehouse, 2. Databricks Delta Lakes and 3. Snowflake. The company selected the Snowflake platform to migrate all their data for providing a modern data warehouse and analytics ecosystem.
Snowflake is a zero maintenance, fully managed, cloud data platform and is one of the most popular choices of enterprises for their data warehousing and analytics needs. It provides a unique architecture for data analysts, data engineers, data scientists and data application developers to collaborate and work on any data without any limitations in performance, concurrency, or scale. Snowflake enables data professionals to support many data warehouses, data lakes, data engineering and data science workloads with virtually unlimited concurrency and compatibility with popular ETL, BI, and data science tools.
In moving from the existing Teradata and Hadoop data architecture to the Snowflake platform, they handled streaming data analytics by leveraging Azure Blob Storage. This was used for storing data before moving it to Snowflake data warehouse for historic reporting. Once the platform decision was made and the cloud solution selected, the company successfully implemented the right tools to fast-track the migration of data warehouse applications and workloads to the cloud with remarkable savings in both cost and time.
Hexaware proposed and leveraged Amaze™ for Data & AI, its proprietary state-of-the-art cloud modernization platform that enables rapid transformation and deployment of data and analytics ecosystems on leading cloud platforms. Amaze™ for Data & AI modernizes the data and analytics landscape and offers more than 60% reduction in TCO towards Snowflake adoption.
The migration of Data Warehouse on Teradata, and Data Lake on Hadoop to Snowflake followed a structured 5-step methodology. The entire process was supported by Amaze’s extensive automation at various stages to ensure reduced time and effort for the data warehouse transformation, while ensuring quality of the migrated data.
The key steps in the transformation process were:
This migration was executed in just 5 months leveraging heavy automation in the entire migration process. It resulted in the following benefits for the company:
In summary, migration to the cloud provides significant performance improvement, agility, and scalability. This company is an example of an organization that went beyond focusing on cloud investments for their current tactical needs. Instead, they adopted a holistic view with a commitment to the cloud, and a strategy of funding critical infrastructure investments—people, processes, and technology. This resulted in significant reduction in idling thus reduced fuel costs, improved route planning thru elimination of choke points, improved equipment uptime thru planned preventive maintenance.
This helped the company put in place a foundation that would ensure continual efficiency improvement, operation expansion and value addition, far into the future.
By Kamal Maggon
