Preserving The Cloud: The Wayback Machine

Preserving The Cloud: The Wayback Machine

Preserving The Cloud: The Wayback Machine

News broke this week that The Wayback Machine has now archived 4 hundred billion webpages. The Wayback Machine enables users to see how another website looked in the past, and after launching in 1996 the project now requires an incredible 5 petabytes of storage to maintain all its data.

The figures behind the project are impressive; The Wayback Machine’s database is queried over 1,000 times every second by over 500,000 people a day – making Archive.org the 250th most popular site on the entire internet.

wayback_machine_logo

The company’s mission is equally impressive, explicitly expressing their ideology on their website “Most societies place importance on preserving artifacts of their culture and heritage. Without such artifacts, civilization has no memory and no mechanism to learn from its successes and failures. Our culture now produces more and more artifacts in digital form. The Archive’s mission is to help preserve those artifacts and create an Internet library for researchers, historians, and scholars”.

Brewster Kahle launched the project in 1996 at the same time he started the now-famous web crawling company Alexa Internet. The project has its roots in the development of software that could crawl and download all publicly accessible World Wide Web pages, the Gopher hierarchy, the Netnews bulletin board system, and downloadable software. The archived content itself wasn’t available until 2001 but by 1999 the archive had already expanded its collections to include texts, audio, moving images and software.

Uses of such an archive are widespread. There is a day-today practical use, as evidenced when The Wayback Machine provided access to important Federal Government sites that went dark during the Federal Government shutdown in the United States. There is also an educational aspect, with importance lessons to be learned from the vast amount of big data stored within its archives. Finally, there is a historical aspect, as the development of our internet has been preserved for future generations to enjoy.

The cloud-based project has not been without its controversies. In 2012 China restored access to the database after blocking it for several years, while in the USA an activist sued the organisation for $100,000 after claiming that archiving her site breached her terms of service. The dispute was ultimately settled out of court.

The success of the project has started to attract imitators. Online companies such as Archive.It, Freezepage, and iTools all offer similar services, but not of them can offer the same quality and depth of content as The Wayback Machine.

Is this a vital project or a waste of valuable storage space? Are there ethical questions surrounding the unhibited archiving of so many sites, or are there taking a virtual photograph of events? Let us know in the comments below.

By Daniel Price

About Daniel Price

Daniel is a Manchester-born UK native who has abandoned cold and wet Northern Europe and currently lives on the Caribbean coast of Mexico. A former Financial Consultant, he now balances his time between writing articles for several industry-leading tech (CloudTweaks.com & MakeUseOf.com), sports, and travel sites and looking after his three dogs.

Find out more
View All Articles

Sorry, comments are closed for this post.

Comic
When Sci-Fi Predictions Come To Fruition

When Sci-Fi Predictions Come To Fruition

Evolution of Technologies To paraphrase science fiction author Arthur C. Clark, those who make predictions about the future are either “considered conservative now and mocked later, or mocked now and proved right when they are no longer around to enjoy the acclaim.” The one thing we can be sure about, Clark ventured, is that “[the…

Facebook Hopes To Extend Internet Connectivity With Solar-Powered Drones

Facebook Hopes To Extend Internet Connectivity With Solar-Powered Drones

Facebook Inc (FB.O) said on Thursday it had completed a successful test flight of a solar-powered drone that it hopes will help it extend internet connectivity to every corner of the planet. Aquila, Facebook’s lightweight, high-altitude aircraft, flew at a few thousand feet for 96 minutes in Yuma, Arizona, Chief Executive Mark Zuckerberg wrote in…

When Will Women In Tech Become The Norm?

When Will Women In Tech Become The Norm?

Tech Diversity It is well known that the technology industry has been dominated by men, but it is also clear that the industry is working to change that. Diversity in the tech industry, especially where it applies to women in tech, has been a topic of discussion for years. Recently the Washington Technology Industry Association…

Four Keys For Telecoms Competing In A Digital World

Four Keys For Telecoms Competing In A Digital World

Competing in a Digital World Telecoms, otherwise largely known as Communications Service Providers (CSPs), have traditionally made the lion’s share of their revenue from providing pipes and infrastructure. Now CSPs face increased competition, not so much from each other, but with digital service providers (DSPs) like Netflix, Google, Amazon, Facebook, and Apple, all of whom…

Edtech and Virtual Reality – Exciting Learning Environment

Edtech and Virtual Reality – Exciting Learning Environment

Customizing Edutech Customized edtech learning solutions are becoming more commonplace as the education industry recognises their potential and begins transforming the traditional structures so as to incorporate innovative developments. From textbooks to tablets, chalkboards to virtual reality, edtech promises not only dynamic and exciting learning environments but better learning strategies and solutions. Virtual Reality and…

Despite Record Breaches, Secure Third Party Access Still Not An IT Priority

Despite Record Breaches, Secure Third Party Access Still Not An IT Priority

Secure Third Party Access Still Not An IT Priority Research has revealed that third parties cause 63 percent of all data breaches. From HVAC contractors, to IT consultants, to supply chain analysts and beyond, the threats posed by third parties are real and growing. Deloitte, in its Global Survey 2016 of third party risk, reported…

Digital Transformation: Not Just For Large Enterprises Anymore

Digital Transformation: Not Just For Large Enterprises Anymore

Digital Transformation Digital transformation is the acceleration of business activities, processes, and operational models to fully embrace the changes and opportunities of digital technologies. The concept is not new; we’ve been talking about it in one way or another for decades: paperless office, BYOD, user experience, consumerization of IT – all of these were stepping…

Cloud Services Providers – Learning To Keep The Lights On

Cloud Services Providers – Learning To Keep The Lights On

The True Meaning of Availability What is real availability? In our line of work, cloud service providers approach availability from the inside out. And in many cases, some never make it past their own front door given how challenging it is to keep the lights on at home let alone factors that are out of…

Maintaining Network Performance And Security In Hybrid Cloud Environments

Maintaining Network Performance And Security In Hybrid Cloud Environments

Hybrid Cloud Environments After several years of steady cloud adoption in the enterprise, an interesting trend has emerged: More companies are retaining their existing, on-premise IT infrastructures while also embracing the latest cloud technologies. In fact, IDC predicts markets for such hybrid cloud environments will grow from the over $25 billion global market we saw…

The Fully Aware, Hybrid-Cloud Approach

The Fully Aware, Hybrid-Cloud Approach

Hybrid-Cloud Approach For over 20 years, organizations have been attempting to secure their networks and protect their data. However, have any of their efforts really improved security? Today we hear journalists and industry experts talk about the erosion of the perimeter. Some say it’s squishy, others say it’s spongy, and yet another claims it crunchy.…

Using Big Data To Make Cities Smarter

Using Big Data To Make Cities Smarter

Using Big Data To Make Cities Smarter The city of the future is impeccably documented. Sensors are used to measure air quality, traffic patterns, and crowd movement. Emerging neighborhoods are quickly recognized, public safety threats are found via social networks, and emergencies are dealt with quicklier. Crowdsourcing reduces commuting times, provides people with better transportation…

Explosive Growth Of Data-Driven Marketing

Explosive Growth Of Data-Driven Marketing

Data-Driven Marketing There is an absolute endless amount of data that is being accumulated, dissected, analyzed with the important bits extracted and used for a number of purposes. With the amount of data in the world has already reached into multiple zettabytes annually. A Zettabyte is one million petabytes or one thousand exabytes. With data…

5 Surprising Ways Cloud Computing Is Changing Education

5 Surprising Ways Cloud Computing Is Changing Education

Cloud Computing Education The benefits of cloud computing are being recognized in businesses and institutions across the board, with almost 90 percent of organizations currently using some kind of cloud-based application. The immediate benefits of cloud computing are obvious: cloud-based applications reduce infrastructure and IT costs, increase accessibility, enable collaboration, and allow organizations more flexibility…

Report: Enterprise Cloud Computing Moves Into Mature Growth Phase

Report: Enterprise Cloud Computing Moves Into Mature Growth Phase

Verizon Cloud Report Enterprises using the cloud, even for mission-critical projects, is no longer new or unusual. It’s now firmly established as a reliable workhorse for an organization and one that can deliver great value and drive transformation. That’s according to a new report from Verizon entitled “State of the Market: Enterprise Cloud 2016.” which…

Utilizing Digital Marketing Techniques Via The Cloud

Utilizing Digital Marketing Techniques Via The Cloud

Digital Marketing Trends In the past, trends in the exceptionally fast-paced digital marketing arena have been quickly adopted or abandoned, keeping marketers and consumers on their toes. 2016 promises a similarly expeditious temperament, with a few new digital marketing offerings taking center stage. According to Gartner’s recent research into Digital Marketing Hubs, brands plan to…

Five Signs The Internet of Things Is About To Explode

Five Signs The Internet of Things Is About To Explode

The Internet of Things Is About To Explode By 2020, Gartner estimates that the Internet of Things (IoT) will generate incremental revenue exceeding $300 billion worldwide. It’s an astoundingly large figure given that the sector barely existed three years ago. We are now rapidly evolving toward a world in which just about everything will become…