Data-Heavy Heavens: Big Data, But Not As We Know It

Data-Heavy Heavens: Big Data, But Not As We Know It

Big Data, But Not as we Know It

As astronomy equipment becomes ever more complex and harvests ever increasing amounts of information from our skies, companies are applying the theories of big data to try and answer one of humanity’s most profound questions – are we alone?

Data-Heavy Heavens

The amount of data collected is mind-boggling. Take the example of the Square Kilometre Array Project (SKA) – a $2 billion joint radio telescope development between Australia, South Africa and the UK. Upon the launch of Phase One in 2020 the project will collect a staggering 915 Petabytes of data per day (approximately 960 million Gigabytes) – that’s nearly 2 million years’ worth of playback on a standard MP3 Player. By the time the operation is fully functional in 2025 the SKA’s will produce ten times the global internet traffic of the entire of 2013.

SKA-DATA

(Image source: http://www.skatelescope.org )

While the search for alien life is not the project’s primary purpose, if life does exist the dishes have a good chance of finding it. The SKA’s telescopes will be so powerful and sensitive that the scientists involved predict that it will be capable of detecting an airport radar signal on an airport 100 light-years away.

Processing and Managing

Naturally, making sense of all this data requires some serious computing power, and the SKA central supercomputer doesn’t disappoint – possessing the processing power of 100 million high-spec PCs and able to perform an ExaFLOP (1018 million trillion) of operations per second.

Storing and archiving this data also requires some creative solutions. NASA’s Jet Propulsion Laboratory (JPL) is already working on developing open-source software that uses innovative cloud computing techniques to organise, manage and catalogue the huge amounts of daily data.

All this alien chasing also has real-world benefits. Today’s computers are still limited to PetaFLOPs which means the SKA’s computational technology will be applicable to large-scale computing everywhere, and the JPL team hope the complex data management tools they develop will help businesses archive and access their data with more ease.

Back on Earth

The vast amount of data collected has offered exciting opportunities to companies back on earth. A good example is Skytree, a 20-employee start-up in San Jose specialising in developing machine learning through the use of big data. The company has recently partnered with the SETI Institute and hopes to use complex algorithms to sift through the massive volumes of data to find patterns, relationships, anomalies and outliers. The techniques allow the Institute to look for signs of extra-terrestrial life in data that was previously discarded due to its size – indeed it’s possible that evidence of an extra-terrestrial life is already hiding in their archives.

It’s not just tech companies and scientific researchers who have access to the big data that’s been collected. SETI Live allows enthusiasts to use their own PCs to access live radio frequency signals SETI Institute’s Allen Telescope Array (ATA) – giving them the tools to identify and report any suspicious patterns that could be indicative of alien life.

Successful?

Whether or not we will ever find extra-terrestrial is naturally debatable, but one thing is certain – if we do detect that life exists then big data will have played a massive part in the discovery.

What do you think? Is this a useful deployment of big data, or could the money and manpower be better spent elsewhere? Let us know in the comments below.

By Daniel Price

About Daniel Price

Daniel is a Manchester-born UK native who has abandoned cold and wet Northern Europe and currently lives on the Caribbean coast of Mexico. A former Financial Consultant, he now balances his time between writing articles for several industry-leading tech (CloudTweaks.com & MakeUseOf.com), sports, and travel sites and looking after his three dogs.

Find out more
View All Articles

One Response to Data-Heavy Heavens: Big Data, But Not As We Know It

  1. We are in the process of implementing Exalead CloudView in several companies to help them with Big Data mining and analytics. We are not nearly finished yet and its amazing how much hidden data we have uncovered, and the catastrophic inefficiencies we were able to find in their business operations. Literally 10s of millions of dollars wasted every year because one hand does not know what the other hand is doing. So keep at it and you will find the gold!

Cloud Infographic – Big Data Predictions By 2023

Cloud Infographic – Big Data Predictions By 2023

Big Data Predictions By 2023 Everything we do online from social networking to e-commerce purchases, chatting, and even simple browsing yields tons of data that certain organizations collect and poll together with other partner organizations. The results are massive volumes of data, hence the name “Big Data”. This includes personal and behavioral profiles that are stored, managed, and…

The Internet of Things – Redefining The Digital World As We Know It

The Internet of Things – Redefining The Digital World As We Know It

Redefining The Digital World According to Internet World Stats (June 30th, 2015), no fewer than 3.2 billion people across the world now use the internet in one way or another. This means an incredible amount of data sharing through the utilization of API’s, Cloud platforms and inevitably the world of connected Things. The Internet of Things is a…

Low Cost Cloud Computing Gives Rise To Startups

Low Cost Cloud Computing Gives Rise To Startups

Balancing The Playing Field For Startups According to a Goldman Sachs report, cloud infrastructure and platform spending could reach $43 billion by 2018, which is up $16 billion from last year, representing a growth of around 30% from 2013 said the analyst. This phenomenal growth is laying the foundation for a new breed of startup…

Cloud Infographic – The Internet Of Things In 2020

Cloud Infographic – The Internet Of Things In 2020

The Internet Of Things In 2020 The growing interest in the Internet of Things is amongst us and there is much discussion. Attached is an archived but still relevant infographic by Intel which has produced a memorizing snapshot at how the number of connected devices have exploded since the birth of the Internet and PC.…

Consequences Of Combining Off Premise Cloud Storage and Corporate Data

Consequences Of Combining Off Premise Cloud Storage and Corporate Data

Off Premise Corporate Data Storage Cloud storage is a broad term. It can encompass anything from on premise solutions, to file storage, disaster recovery and off premise options. To narrow the scope, I’ve dedicated the focus of today’s discussion to the more popular cloud storage services—such as Dropbox, Box, OneDrive—which are also known as hosted,…

Who’s Who In The Booming World Of Data Science

Who’s Who In The Booming World Of Data Science

The World of Data Science The nature of work and business in today’s super-connected world means that every second of every day, the world produces an astonishing amount of data. Consider some of these statistics; every minute, Facebook users share nearly 2.5 million pieces of content, YouTube users upload over 72 hours of content, Apple…

The Future Of Work: What Cloud Technology Has Allowed Us To Do Better

The Future Of Work: What Cloud Technology Has Allowed Us To Do Better

What Cloud Technology Has Allowed Us to Do Better The cloud has made our working lives easier, with everything from virtually unlimited email storage to access-from-anywhere enterprise resource planning (ERP) systems. It’s no wonder the 2013 cloud computing research IDG survey revealed at least 84 percent of the companies surveyed run at least one cloud-based application.…

Cloud Computing Is Greener Than You Think

Cloud Computing Is Greener Than You Think

Cloud Computing Is Greener Than You Think Last week we touched upon how a project in Finland had blended two of the world’s most important industries, cloud computing and green technology, to produce a data centre that used nearby sea water to both cool their servers and heat local homes.  Despite such positive environmental projects, there…

6 Tech Predictions To Have A Major Impact In 2016

6 Tech Predictions To Have A Major Impact In 2016

6 Tech Predictions To Have A Major Impact The technology industry moves at a relentless pace, making it both exhilarating and unforgiving. For those at the forefront of innovation it is an incredibly exciting place to be, but what trends are we likely to see coming to the fore in 2016? Below are six predictions…

4 Industries Being Transformed By The Internet of Things

4 Industries Being Transformed By The Internet of Things

Compelling IoT Industries Every year, more and more media organizations race to predict the trends that will come to shape the online landscape over the next twelve months. Many of these are wild and outlandish and should be consumed with a pinch of salt, yet others stand out for their sober and well-researched judgements. Online…