Data-Heavy Heavens: Big Data, But Not As We Know It

Big Data, But Not as we Know It

As astronomy equipment becomes ever more complex and harvests ever increasing amounts of information from our skies, companies are applying the theories of big data to try and answer one of humanity’s most profound questions – are we alone?

Data-Heavy Heavens

The amount of data collected is mind-boggling. Take the example of the Square Kilometre Array Project (SKA) – a $2 billion joint radio telescope development between Australia, South Africa and the UK. Upon the launch of Phase One in 2020 the project will collect a staggering 915 Petabytes of data per day (approximately 960 million Gigabytes) – that’s nearly 2 million years’ worth of playback on a standard MP3 Player. By the time the operation is fully functional in 2025 the SKA’s will produce ten times the global internet traffic of the entire of 2013.

While the search for alien life is not the project’s primary purpose, if life does exist the dishes have a good chance of finding it. The SKA’s telescopes will be so powerful and sensitive that the scientists involved predict that it will be capable of detecting an airport radar signal on an airport 100 light-years away.

Processing and Managing

Naturally, making sense of all this data requires some serious computing power, and the SKA central supercomputer doesn’t disappoint – possessing the processing power of 100 million high-spec PCs and able to perform an ExaFLOP (1018 million trillion) of operations per second.

Storing and archiving this data also requires some creative solutions. NASA’s Jet Propulsion Laboratory (JPL) is already working on developing open-source software that uses innovative cloud computing techniques to organise, manage and catalogue the huge amounts of daily data.

All this alien chasing also has real-world benefits. Today’s computers are still limited to PetaFLOPs which means the SKA’s computational technology will be applicable to large-scale computing everywhere, and the JPL team hope the complex data management tools they develop will help businesses archive and access their data with more ease.

Back on Earth

The vast amount of data collected has offered exciting opportunities to companies back on earth. A good example is Skytree, a 20-employee start-up in San Jose specialising in developing machine learning through the use of big data. The company has recently partnered with the SETI Institute and hopes to use complex algorithms to sift through the massive volumes of data to find patterns, relationships, anomalies and outliers. The techniques allow the Institute to look for signs of extra-terrestrial life in data that was previously discarded due to its size – indeed it’s possible that evidence of an extra-terrestrial life is already hiding in their archives.

It’s not just tech companies and scientific researchers who have access to the big data that’s been collected. SETI Live allows enthusiasts to use their own PCs to access live radio frequency signals SETI Institute’s Allen Telescope Array (ATA) – giving them the tools to identify and report any suspicious patterns that could be indicative of alien life.

Successful?

Whether or not we will ever find extra-terrestrial is naturally debatable, but one thing is certain – if we do detect that life exists then big data will have played a massive part in the discovery.

What do you think? Is this a useful deployment of big data, or could the money and manpower be better spent elsewhere? Let us know in the comments below.

By Daniel Price

Frank Suglia
Managing Data Sprawl Over the last two years, our world experienced a dramatic acceleration of digital transformation. The COVID-19 pandemic upended normal operations for many businesses and shifted the pace of technology adoption into warp ...
Dana Gardner
Low-code Development Has Entered a Maturity Spurt Closing the gap between the applications and services a company needs -- and the ones they can actually produce -- has long been a missing keystone for attaining ...
Rahul
How to Start Your Cloud Career Cloud computing is the present. And it is the future as well!! In fact, a quote by Chris Howard says, ‘Cloud Computing is a spectrum of things complementing one ...
Crozdesk Business Software
B2B SaaS Comparison Platforms B2B SaaS Comparison Platforms are designed for buyers looking for additional information on a particular vendor and service. These sites help ease the complexities for buyers by providing a detailed breakdown ...
Louis
Why Services CPQ Is Too Slow Today When PS organizations compete in sales cycles, the first competitor to have a complete quote with accurate pricing, schedules, and an engagement plan will often win. However, getting ...

SECURITY TRAINING

  • Isc2

    ISC2

    (ISC)² provides IT training, certifications, and exams that run online, on your premises, or in classrooms. Self-study resources are available. You can also train groups of 10 or more of your employees. If you want a job in cybersecurity, this is the route to take.

  • App Academy

    App Academy

    Immersive software engineering programs. No experience required. Pay $0 until you're hired. Join an online info session to learn more

  • Cybrary

    Cybrary

    CYBRARY Open source Cyber Security learning. Free for everyone, forever. The world's largest cyber security community. Cybrary provides free IT training and paid IT certificates. Courses for beginners, intermediates, and advanced users are available.

  • Plural Site

    Pluralsite

    Pluralsight provides online courses on popular programming languages and developer tools. Other courses cover fields such as IT security best practices, server infrastructure, and virtualization.