Storing Massive Files On The Cloud

Storing Massive Files On The Cloud

On December 9th, Amazon announced, in a blog post, an increase in the maximum size of
file that could be stored on the Amazon Simple Storage Service (S3). Before the increase, the largest
file that could be stored was 5GB – anything larger had to be split into chunks and put back together
within an application, using an intermediate server or on the client device. Now, the limit has been
increased to 5TB, an increase of approximately 1000 times.

This new limit allows the storage of large databases, video files or scientific data, but to make use of
it properly, the object needs to be uploaded using Amazon’s new Multipart Upload API. The original
objects are split into smaller parts and then uploaded separately before being stitched together on
the Amazon S3 cloud. Objects can then be worked on either as a whole, or in part, on the Amazon
Elastic Compute Cloud (EC2) before storing the finished product back on S3.

Data stored on Amazon S3 is designed to have 99.99% annual availability and to withstand the
concurrent failure of data in two facilities, so you can be certain your data would be available at all
times. For less critical data, Amazon’s Reduced Redundancy Storage (RRS) offers a cheaper way of
storing your non-critical data, by utilizing less storage facilities. There is an associated loss of data
durability, but the quoted availability is still 99.99%.

OpenStack have also announced that they are working on increasing the size of files that can be
stored on its open-source private cloud system, and it is expected that a 5TB file size limit will be
available with the “Bexar” release of OpenStack early in 2011.

A file size of 5TB is a massive amount of data, but it is clear that the cloud storage limits are going to
increase even further in the near future. It is only a matter of time before the maximum object size is
in the petabyte or even exabyte range.

The only question then is exactly how you can generate enough data to need that amount of storage!
Amazon Web Services Blog.

By CloudTweaks

Follow Us!

CloudTweaks

Established in 2009, CloudTweaks.com is recognized as one of the leading authorities in cloud computing information. Most of the excellent CloudTweaks articles are provided by our own paid writers, with a small percentage provided by guest authors from around the globe, including CEOs, CIOs, Technology bloggers and Cloud enthusiasts. Our goal is to continue to build a growing community offering the best in-depth articles, interviews, event listings, whitepapers, infographics and much more...
Follow Us!

Sorry, comments are closed for this post.

Comics

At CloudTweaks, we're plugged into the cloud, the internet of things and all that the web has to offer. From wearable technology, to mobile computing, cloud computing and big data, CloudTweaks is your source for updates and news on the most innovative technology.

Popular

Top Viral Impact

The Industries That The Cloud Will Change The Most

The Industries That The Cloud Will Change The Most

The Industries That The Cloud Will Change The Most Cloud computing is rapidly revolutionizing the way we do business. Instead of being a blurry buzzword, it has become a facet of everyday life. Most people may not quite understand how the cloud works, but electricity is quite difficult to fathom as well. Anyway, regardless of…

Using Big Data To Make Cities Smarter

Using Big Data To Make Cities Smarter

Using Big Data To Make Cities Smarter The city of the future is impeccably documented. Sensors are used to measure air quality, traffic patterns, and crowd movement. Emerging neighborhoods are quickly recognized, public safety threats are found via social networks, and emergencies are dealt with quicklier. Crowdsourcing reduces commuting times, provides people with better transportation…

Cloud Infographic – The Future (IoT)

Cloud Infographic – The Future (IoT)

The Future (IoT) By the year 2020, it is being predicted that 40 to 80 billion connected devices will be in use. The Internet of Things or IoT will transform your business and home in many truly unbelievable ways. The types of products and services that we can expect to see in the next decade…

IBM and SAP Announce Industry’s Largest Cloud Deal

IBM and SAP Announce Industry’s Largest Cloud Deal

IBM and SAP Announce Industry’s Largest Cloud Deal IBM and SAP have shaken the cloud computing world this afternoon with the announcement of one of the largest cloud deals in the industry’s history – bringing together two of the largest technology companies in a bid to offer a more holistic service to their clients. SAP…

5 Ways The Internet of Things Will Drive Cloud Growth

5 Ways The Internet of Things Will Drive Cloud Growth

5 Ways The Internet of Things Will Drive Cloud Growth The Internet of Things is the latest term to describe the interconnectivity of all our devices and home appliances. The goal of the internet of things is to create universal applications that are connected to all of the lights, TVs, door locks, air conditioning, and…

Featured Sponsors

Moving From Email Into The Cloud

Moving From Email Into The Cloud

Mobile Collaboration In The Cloud Imagine that you, as a manager, are told by the powers that be that you have to find “efficiencies” within your department that will result in one million dollars of savings annually. You struggle with this. You send an email to everyone on your senior team. “Where can we save…

Salesforce Service Cloud: Air Traffic Control For Your Customer

Salesforce Service Cloud: Air Traffic Control For Your Customer

Salesforce Service Cloud One of the greatest benefits of the increasingly reliable and ubiquitous state of cloud technology is the removal of business silos and the consolidation of information flow, both in-house and on the road. This is of particular importance to the many different types of professionals whose work involves customer relationship management (CRM).…

2015 Advertising Opportunities - Find Out More!

Cloud Logo Sponsors

hp Logo CityCloud-PoweredByOpenstack-Blue square_logo_100x100-01
cisco_logo_100x100 vmware citrix100

Established in 2009, CloudTweaks is recognized as one of the leading influencers in cloud computing, big data and internet of things (IoT) information. Our goal is to continue to build our growing information portal, by providing the best in-depth articles, interviews, event listings, whitepapers, infographics and much more.

Contact

CloudTweaks Media
Phone: 1 (212) 763-0021

Join Our Newsletter