Storing Massive Files On The Cloud

Storing Massive Files On The Cloud

On December 9th, Amazon announced, in a blog post, an increase in the maximum size of
file that could be stored on the Amazon Simple Storage Service (S3). Before the increase, the largest
file that could be stored was 5GB – anything larger had to be split into chunks and put back together
within an application, using an intermediate server or on the client device. Now, the limit has been
increased to 5TB, an increase of approximately 1000 times.

This new limit allows the storage of large databases, video files or scientific data, but to make use of
it properly, the object needs to be uploaded using Amazon’s new Multipart Upload API. The original
objects are split into smaller parts and then uploaded separately before being stitched together on
the Amazon S3 cloud. Objects can then be worked on either as a whole, or in part, on the Amazon
Elastic Compute Cloud (EC2) before storing the finished product back on S3.

Data stored on Amazon S3 is designed to have 99.99% annual availability and to withstand the
concurrent failure of data in two facilities, so you can be certain your data would be available at all
times. For less critical data, Amazon’s Reduced Redundancy Storage (RRS) offers a cheaper way of
storing your non-critical data, by utilizing less storage facilities. There is an associated loss of data
durability, but the quoted availability is still 99.99%.

OpenStack have also announced that they are working on increasing the size of files that can be
stored on its open-source private cloud system, and it is expected that a 5TB file size limit will be
available with the “Bexar” release of OpenStack early in 2011.

A file size of 5TB is a massive amount of data, but it is clear that the cloud storage limits are going to
increase even further in the near future. It is only a matter of time before the maximum object size is
in the petabyte or even exabyte range.

The only question then is exactly how you can generate enough data to need that amount of storage!
Amazon Web Services Blog.

By CloudTweaks

Follow Us!

CloudTweaks

Established in 2009, CloudTweaks.com is recognized as one of the leading authorities in cloud computing information. Most of the excellent CloudTweaks articles are provided by our own paid writers, with a small percentage provided by guest authors from around the globe, including CEOs, CIOs, Technology bloggers and Cloud enthusiasts. Our goal is to continue to build a growing community offering the best in-depth articles, interviews, event listings, whitepapers, infographics and much more...
Follow Us!

Sorry, comments are closed for this post.

Using Big Data To Make Cities Smarter

Using Big Data To Make Cities Smarter

Using Big Data To Make Cities Smarter The city of the future is impeccably documented. Sensors are used to measure air quality, traffic patterns, and crowd movement. Emerging neighborhoods are quickly recognized, public safety threats are found via social networks, and emergencies are dealt with quicklier. Crowdsourcing reduces commuting times, provides people with better transportation…

Cloud Infographic – The Internet Of Things In 2020

Cloud Infographic – The Internet Of Things In 2020

The Internet Of Things In 2020 The growing interest in the Internet of Things is amongst us and there is much discussion. Attached is an archived but still relevant infographic by Intel which has produced a memorizing snapshot at how the number of connected devices have exploded since the birth of the Internet and PC.…

Cloud Computing and Social Networks

Cloud Computing and Social Networks

Pushing Innovation With Social Networks There was a time when storage space was a precious commodity. A few videos and photographs and the system would be gasping for breath in old PCs’ limited storage space. 10 Years ago, today’s top social networking sites would have broken down under the burden of the heavily visual traffic…

Cloud Infographic – The Future Of Big Data

Cloud Infographic – The Future Of Big Data

The Future Of Big Data Big Data is BIG business and will continue to be one of the more predominant areas of focus in the coming years from small startups to large scale corporations. We’ve already covered on CloudTweaks how Big Data can be utilized in a number of interesting ways from preventing world hunger to helping teams win…

5 Considerations You Need To Review Before Investing In Data Analytics

5 Considerations You Need To Review Before Investing In Data Analytics

Review Before Investing In Data Analytics Big data, when handled properly, can lead to big change. Companies in a wide variety of industries are partnering with data analytics companies to increase operational efficiency and make evidence-based business decisions. From Kraft Foods using business intelligence (BI) to cut customer satisfaction analysis time in half, to a…

CONNECT TO THE CLOUD

 

Cloud Logo Sponsors

hp Logo CityCloud-PoweredByOpenstack-Bluesquare_logo_100x100-01
cisco_logo_100x100 vmware citrix100
Site 24x7


Contributor Spotlight

Established in 2009, CloudTweaks is recognized as one of the leading influencers in cloud computing, big data and internet of things (IoT) information. Our goal is to continue to build our growing information portal, by providing the best in-depth articles, interviews, event listings, whitepapers, infographics and much more.

Contact

CloudTweaks Media
Phone: 1 (212) 763-0021

Branded Content Programs

Advertising