Recent problems experienced with Ransomware are evident from infections, which have occurred in 99 countries including China and Russia. The organization that was worst hit by the attack was the National Health Service in England. It was reported that there was a WannaCry programme that demanded...

Storing Massive Files On The Cloud

Storing Massive Files On The Cloud

On December 9th, Amazon announced, in a blog post, an increase in the maximum size of file that could be stored on the Amazon Simple Storage Service (S3). Before the increase, the largest file that could be stored was 5GB – anything larger had to be split into chunks and put back together within an application, using an intermediate server or on the client device. Now, the limit has been increased to 5TB, an increase of approximately 1000 times.

This new limit allows the storage of large databases, video files or scientific data, but to make use of it properly, the object needs to be uploaded using Amazon’s new Multipart Upload API. The original objects are split into smaller parts and then uploaded separately before being stitched together on the Amazon S3 cloud. Objects can then be worked on either as a whole, or in part, on the Amazon Elastic Compute Cloud (EC2) before storing the finished product back on S3.

Data stored on Amazon S3 is designed to have 99.99% annual availability and to withstand the concurrent failure of data in two facilities, so you can be certain your data would be available at all times. For less critical data, Amazon’s Reduced Redundancy Storage (RRS) offers a cheaper way of storing your non-critical data, by utilizing less storage facilities. There is an associated loss of data durability, but the quoted availability is still 99.99%.

OpenStack have also announced that they are working on increasing the size of files that can be stored on its open-source private cloud system, and it is expected that a 5TB file size limit will be available with the “Bexar” release of OpenStack early in 2011.

A file size of 5TB is a massive amount of data, but it is clear that the cloud storage limits are going to increase even further in the near future. It is only a matter of time before the maximum object size is in the petabyte or even exabyte range.

The only question then is exactly how you can generate enough data to need that amount of storage! Amazon Web Services Blog

By Glenn Blake

About Glenn Blake

Glenn Blake is a writer for CloudTweaks and has been writing about technology trends for over 25 years.

View Website


(ISC)2 and Cloud Security Alliance Host Cloud Security Summit to Help Cybersecurity Pros Securely Harness Cloud Technologies

By CloudBuzz | September 22, 2017

(ISC)2 and Cloud Security Alliance Host Cloud Security Summit Research cites strengthening of cloud security skills top priority over next three years CLEARWATER, Fla. ,Sept. 22, 2017 /PRNewswire-USNewswire/ — (ISC)² today announced it’s partnering with the Cloud Security Alliance (CSA) for the CSA…

Exclusive: T-Mobile, Sprint close to agreeing deal terms – Sources

By CloudBuzz | September 22, 2017

(Reuters) – T-Mobile US Inc (TMUS.O) is close to agreeing tentative terms on a deal to merge with peer Sprint Corp (S.N), people familiar with the matter said, a major breakthrough in efforts to merge the third and fourth largest…

Hack of U.S. securities regulator rattles investors, stirs doubts

By CloudBuzz | September 21, 2017

WASHINGTON/NEW YORK (Reuters) – Wall Street’s top regulator faced questions on Thursday about its defenses against cyber criminals after admitting hackers breached its electronic database of corporate announcements and may have used it for insider trading. The incursion at the…

Leaking Cloud Databases and Servers Expose Over 1 Billion Records

By CloudBuzz | September 21, 2017

Servers Expose Over 1 Billion Records As The Wall Street Journal recently pointed out, some clients of cloud service providers such as Amazon and Microsoft are accidentally leaving their cloud databases exposed due to misconfigurations of their services. Coupled with recent headline-making…

Thales Joins the Microsoft Enterprise Cloud Alliance

By CloudBuzz | September 21, 2017

SAN JOSE, Calif., Sept. 21, 2017 /PRNewswire/ — Thales, a leader in critical information systems, cybersecurity and data security, is now a member of the Microsoft Enterprise Cloud Alliance (ECA). Designed to foster innovation and promote awareness of partner solutions, the ECA membership…

Addressing the UK NCSC’s Cloud Security Principles

By CloudBuzz | September 20, 2017

As your organization adopts more cloud services, it’s essential to get a clear picture of how sensitive data will be protected. Many authorities, from government regulators, to industry standards bodies and consortia, have provided guidance on how to evaluate cloud…