Storing Massive Files On The Cloud

Storing Massive Files On The Cloud

On December 9th, Amazon announced, in a blog post, an increase in the maximum size of
file that could be stored on the Amazon Simple Storage Service (S3). Before the increase, the largest
file that could be stored was 5GB – anything larger had to be split into chunks and put back together
within an application, using an intermediate server or on the client device. Now, the limit has been
increased to 5TB, an increase of approximately 1000 times.

This new limit allows the storage of large databases, video files or scientific data, but to make use of
it properly, the object needs to be uploaded using Amazon’s new Multipart Upload API. The original
objects are split into smaller parts and then uploaded separately before being stitched together on
the Amazon S3 cloud. Objects can then be worked on either as a whole, or in part, on the Amazon
Elastic Compute Cloud (EC2) before storing the finished product back on S3.

Data stored on Amazon S3 is designed to have 99.99% annual availability and to withstand the
concurrent failure of data in two facilities, so you can be certain your data would be available at all
times. For less critical data, Amazon’s Reduced Redundancy Storage (RRS) offers a cheaper way of
storing your non-critical data, by utilizing less storage facilities. There is an associated loss of data
durability, but the quoted availability is still 99.99%.

OpenStack have also announced that they are working on increasing the size of files that can be
stored on its open-source private cloud system, and it is expected that a 5TB file size limit will be
available with the “Bexar” release of OpenStack early in 2011.

A file size of 5TB is a massive amount of data, but it is clear that the cloud storage limits are going to
increase even further in the near future. It is only a matter of time before the maximum object size is
in the petabyte or even exabyte range.

The only question then is exactly how you can generate enough data to need that amount of storage!
Amazon Web Services Blog.

By CloudTweaks

Follow Us!

CloudTweaks

Established in 2009, CloudTweaks.com is recognized as one of the leading authorities in cloud computing information. Most of the excellent CloudTweaks articles are provided by our own paid writers, with a small percentage provided by guest authors from around the globe, including CEOs, CIOs, Technology bloggers and Cloud enthusiasts. Our goal is to continue to build a growing community offering the best in-depth articles, interviews, event listings, whitepapers, infographics and much more...
Follow Us!

Sorry, comments are closed for this post.

Comics

At CloudTweaks, we're plugged into the cloud, the internet of things and all that the web has to offer. From wearable technology, to mobile computing, cloud computing and big data, CloudTweaks is your source for updates and news on the most innovative technology.

Popular

Top Viral Impact

Cloud Infographic – The Future Of Big Data

Cloud Infographic – The Future Of Big Data

Cloud Infographic – The Future Of Big Data Big Data is BIG business and will continue to be one of the more predominant areas of focus in the coming years from small startups to large scale corporations. We’ve already covered on CloudTweaks how Big Data can be utilized in a number of interesting ways from preventing world hunger to…

The Lighter Side Of The Cloud – Holiday Photos

The Lighter Side Of The Cloud – Holiday Photos

The Lighter Side Of The Cloud – Holiday Photos Enjoy our weekly comics provided by our talented cartoonists. By David Fletcher About Latest Posts Follow Us!CloudTweaksEstablished in 2009, CloudTweaks.com is recognized as one of the leading authorities in cloud computing information. Most of the excellent CloudTweaks articles are provided by our own paid writers, with…

Cloud Computing Offers Key Benefits For Small, Medium Businesses

Cloud Computing Offers Key Benefits For Small, Medium Businesses

A growing number of small and medium businesses in the United States rely on as a means of deploying mission-critical software products. Prior to the advent of cloud-based products — software solutions delivered over the Internet – companies were often forced to invest in servers and other products to run software and store data. The…

Cloud Infographic – Cyber Security And The New Frontier

Cloud Infographic – Cyber Security And The New Frontier

Cyber Security: The New Frontier The security environment of the 21st century is constantly evolving, and it’s difficult to predict where the next threats and dangers will come from. But one thing is clear: the ever-expanding frontier of digital space will continue to present firms and governments with security challenges. From politically-motivated Denial-of-Service attacks to…

Featured Sponsors

Salesforce Service Cloud: Air Traffic Control For Your Customer

Salesforce Service Cloud: Air Traffic Control For Your Customer

Salesforce Service Cloud One of the greatest benefits of the increasingly reliable and ubiquitous state of cloud technology is the removal of business silos and the consolidation of information flow, both in-house and on the road. This is of particular importance to the many different types of professionals whose work involves customer relationship management (CRM).…

Watching You Shop: Stores And Mannequins “Read” Their Customers And Respond

Watching You Shop: Stores And Mannequins “Read” Their Customers And Respond

Watching You Shop The mannequin in the store window stares blankly ahead as shoppers look at the clothes it is dressed in and contemplate a purchase. One shopper makes some comments. “It’s nice. I wonder if they have my size.” Another takes a few steps inside the store to see where these particular clothes might…

Sponsors

What To Do When You’ve Outgrown Your Basic Business Software

What To Do When You’ve Outgrown Your Basic Business Software

What To Do When You’ve Outgrown Your Basic Business Software You know that feeling. You have multiple business products that aren’t communicating, meaning your employees are doing more work, uploading redundant data into different systems. The software is sluggish and can’t be accessed via browser. You’re paying per user, which is starting to get painful.…

Placement Opportunities - Find Out!

Established in 2009, CloudTweaks is recognized as one of the leading influencers in cloud computing, big data and internet of things (IoT) information. Our goal is to continue to build our growing information portal, by providing the best in-depth articles, interviews, event listings, whitepapers, infographics and much more.

You can help continue to support our community by social sharing, sponsoring, partnering or contributing to this great educational resource.

Contact

CloudTweaks Media
Phone: 1 (212) 763-0021

Join Our Newsletter