Data-Heavy Heavens: Big Data, But Not As We Know It

Data-Heavy Heavens: Big Data, But Not As We Know It

Big Data, But Not as we Know It

As astronomy equipment becomes ever more complex and harvests ever increasing amounts of information from our skies, companies are applying the theories of big data to try and answer one of humanity’s most profound questions – are we alone?

Data-Heavy Heavens

The amount of data collected is mind-boggling. Take the example of the Square Kilometre Array Project (SKA) – a $2 billion joint radio telescope development between Australia, South Africa and the UK. Upon the launch of Phase One in 2020 the project will collect a staggering 915 Petabytes of data per day (approximately 960 million Gigabytes) – that’s nearly 2 million years’ worth of playback on a standard MP3 Player. By the time the operation is fully functional in 2025 the SKA’s will produce ten times the global internet traffic of the entire of 2013.

SKA-DATA

(Image source: http://www.skatelescope.org )

While the search for alien life is not the project’s primary purpose, if life does exist the dishes have a good chance of finding it. The SKA’s telescopes will be so powerful and sensitive that the scientists involved predict that it will be capable of detecting an airport radar signal on an airport 100 light-years away.

Processing and Managing

Naturally, making sense of all this data requires some serious computing power, and the SKA central supercomputer doesn’t disappoint – possessing the processing power of 100 million high-spec PCs and able to perform an ExaFLOP (1018 million trillion) of operations per second.

Storing and archiving this data also requires some creative solutions. NASA’s Jet Propulsion Laboratory (JPL) is already working on developing open-source software that uses innovative cloud computing techniques to organise, manage and catalogue the huge amounts of daily data.

All this alien chasing also has real-world benefits. Today’s computers are still limited to PetaFLOPs which means the SKA’s computational technology will be applicable to large-scale computing everywhere, and the JPL team hope the complex data management tools they develop will help businesses archive and access their data with more ease.

Back on Earth

The vast amount of data collected has offered exciting opportunities to companies back on earth. A good example is Skytree, a 20-employee start-up in San Jose specialising in developing machine learning through the use of big data. The company has recently partnered with the SETI Institute and hopes to use complex algorithms to sift through the massive volumes of data to find patterns, relationships, anomalies and outliers. The techniques allow the Institute to look for signs of extra-terrestrial life in data that was previously discarded due to its size – indeed it’s possible that evidence of an extra-terrestrial life is already hiding in their archives.

It’s not just tech companies and scientific researchers who have access to the big data that’s been collected. SETI Live allows enthusiasts to use their own PCs to access live radio frequency signals SETI Institute’s Allen Telescope Array (ATA) – giving them the tools to identify and report any suspicious patterns that could be indicative of alien life.

Successful?

Whether or not we will ever find extra-terrestrial is naturally debatable, but one thing is certain – if we do detect that life exists then big data will have played a massive part in the discovery.

What do you think? Is this a useful deployment of big data, or could the money and manpower be better spent elsewhere? Let us know in the comments below.

By Daniel Price

About Daniel Price

Daniel is a Manchester-born UK native who has abandoned cold and wet Northern Europe and currently lives on the Caribbean coast of Mexico. A former Financial Consultant, he now balances his time between writing articles for several industry-leading tech (CloudTweaks.com & MakeUseOf.com), sports, and travel sites and looking after his three dogs.

Find out more
View All Articles

One Response to Data-Heavy Heavens: Big Data, But Not As We Know It

  1. We are in the process of implementing Exalead CloudView in several companies to help them with Big Data mining and analytics. We are not nearly finished yet and its amazing how much hidden data we have uncovered, and the catastrophic inefficiencies we were able to find in their business operations. Literally 10s of millions of dollars wasted every year because one hand does not know what the other hand is doing. So keep at it and you will find the gold!

Comic
Martech In A Content Crazed World

Martech In A Content Crazed World

Content Crazed World Everywhere you look there are pop-up ads and offers, at times it can feel like overload. What used to be a few online ads on websites has now grown into a wild world of offers that consume your every device. These advancements in marketing technology can not only be overwhelming to the…

Hubgets – Advanced Collaboration, Enriched Communication

Hubgets – Advanced Collaboration, Enriched Communication

Advanced Collaboration Tool Sponsored series provided in collaboration with Hubgets Collaboration tools have advanced leaps and bounds with the advent of cloud technology, and the services available are only getting better. Promising features such as sophisticated group communication, productive management of tasks and meetings, and the ultimate dream, working remotely from some gorgeous island destination, innovative collaboration…

Verizon To Acquire Yahoo For $4.83 Billion

Verizon To Acquire Yahoo For $4.83 Billion

Verizon Communications Inc (VZ.N) said it had agreed to buy Yahoo Inc’s (YHOO.O) core internet business for $4.83 billion in cash, ending a lengthy sale process for the fading Web pioneer. Buying Yahoo’s operations will boost Verizon’s AOL internet business, which it bought last year for $4.4 billion, by giving it access to Yahoo’s advertising…

When Sci-Fi Predictions Come To Fruition

When Sci-Fi Predictions Come To Fruition

Evolution of Technologies To paraphrase science fiction author Arthur C. Clark, those who make predictions about the future are either “considered conservative now and mocked later, or mocked now and proved right when they are no longer around to enjoy the acclaim.” The one thing we can be sure about, Clark ventured, is that “[the…

Facebook Hopes To Extend Internet Connectivity With Solar-Powered Drones

Facebook Hopes To Extend Internet Connectivity With Solar-Powered Drones

Facebook Inc (FB.O) said on Thursday it had completed a successful test flight of a solar-powered drone that it hopes will help it extend internet connectivity to every corner of the planet. Aquila, Facebook’s lightweight, high-altitude aircraft, flew at a few thousand feet for 96 minutes in Yuma, Arizona, Chief Executive Mark Zuckerberg wrote in…

Don’t Be Intimidated By Data Governance

Don’t Be Intimidated By Data Governance

Data Governance Data governance, the understanding of the raw data of an organization is an area IT departments have historically viewed as a lose-lose proposition. Not doing anything means organizations run the risk of data loss, data breaches and data anarchy – no control, no oversight – the Wild West with IT is just hoping…

The Security Gap: What Is Your Core Strength?

The Security Gap: What Is Your Core Strength?

The Security Gap You’re out of your mind if you think blocking access to file sharing services is filling a security gap. You’re out of your mind if you think making people jump through hoops like Citrix and VPNs to get at content is secure. You’re out of your mind if you think putting your…

Get Ready For Virtual Reality and the Cloud

Get Ready For Virtual Reality and the Cloud

Virtual Reality Cloud We’re lucky to live in an era where virtual reality is no longer relegated to the confines of a sci-fi movie universe. Thanks to technology introduced by products like Oculus Rift, consumers now have access to virtual environments with fully immersive graphic capabilities. As a result, companies have only just begun to…

Multi-Cloud Integration Has Arrived

Multi-Cloud Integration Has Arrived

Multi-Cloud Integration Speed, flexibility, and innovation require multiple cloud services As businesses seek new paths to innovation, racing to market with new features and products, cloud services continue to grow in popularity. According to Gartner, 88% of total compute will be cloud-based by 2020, leaving just 12% on premise. Flexibility remains a key consideration, and…

Why Small Businesses Need A Business Intelligence Dashboard

Why Small Businesses Need A Business Intelligence Dashboard

The Business Intelligence Dashboard As a small business owner you would certainly know the importance of collecting and analyzing data pertaining to your business and transactions. Business Intelligence dashboards allow not only experts but you also to access information generated by analysis of data through a convenient display. Anyone in the company can have access…

Three Factors for Choosing Your Long-term Cloud Strategy

Three Factors for Choosing Your Long-term Cloud Strategy

Choosing Your Long-term Cloud Strategy A few weeks ago I visited the global headquarters of a large multi-national company to discuss cloud strategy with the CIO. I arrived 30 minutes early and took a tour of the area where the marketing team showcased their award winning brands. I was impressed by the digital marketing strategy…

Cloud Computing – A Requirement For Greater Innovation

Cloud Computing – A Requirement For Greater Innovation

Cloud Computing Innovation Sao Paulo, Brazil has had trouble with both energy and water supplies as of late. Despite it is the rainy period. Unfortunately Sao Paulo is very dependent on its rain as a majority of its power is generated from large dams. No water, no energy. Difficult situation for a city of some…

Cloud Computing Myths That SMBs Should Know

Cloud Computing Myths That SMBs Should Know

Cloud Computing and SMBs Cloud Computing is the hottest issue among IT intellects of Small and Medium Businesses (SMBs). Like any other computer-orientated technology, Cloud Computing has some misconceptions and myths that often kick-start arguments among the two opposing groups: Cloud Supporters and Cloud Opponents. Both of these groups have their own ideology and reasons…

Surprising Facts and Stats About The Big Data Industry

Surprising Facts and Stats About The Big Data Industry

Facts and Stats About The Big Data Industry If you start talking about big data to someone who is not in the industry, they immediately conjure up images of giant warehouses full of servers, staff poring over page after page of numbers and statistics, and some big brother-esque official sat in a huge government building…

Are Women Discriminated Against In The Tech Sector?

Are Women Discriminated Against In The Tech Sector?

Women Discriminated Against In Tech Sector It is no secret that the tech industry is considered sexist since most women are paid less than men; there are considerably fewer women in tech jobs; and generally men get promoted above women. Yet the irony is twofold. Firstly, there is an enormous demand for employees with skills…