Browsed by
Tag: Storage

Amazon S3 | Batch processing of image files

Amazon S3 | Batch processing of image files

At work, we store all of our images on Amazon S3.  This allows us to have a reliable storage point where we can host millions of photos.

We store a couple different sizes of images, so the Amazon S3 buckets (folders) are named by the different image sizes we keep.  As a new feature, we started saving a larger image size whenever anyone uploaded a new photo through our site so that new bucket (1680px wide) needed to be back filled with the largest image size we stored for all previous images (1200px wide).

As I looked into this, there is no batch process on Amazon that allows you to send them a large list of keys and a command to run on those keys.  A program must be written to do this instead.

Read More Read More