Skip to main content

Versioning static files with S3 buckets

Although there's a trend of making single-page applications with frontend static files managed separately from backend api, the need of managing static files haven't gone away just yet. And everyone who does web is aware of common issues with it. Probably the most common one is browser cache. Files get cached in user's browser and are used even after you changed and deployed them. Cache-controlling headers can help somewhat but not much. That's my cache-boosting techniques are usually a must.

There are many ways to do cache boosting. Usually, it involves adding some version info into all static urls (e.g. /style.css becomes /style-13.29.css or /13.29/style.css), hence it's often called "versioning". If you use some Django app to manage your static files (compress, combine, minify, etc.) it often can provide you with some solution. Use it, it's probably reliable and easy. This proposal, however, is cool if you happen to use Amazon's S3 for your static hosting (directly, not behind cloudfront). Yeah, I'm mostly referring to Django in this post because that's what I use, but general principles apply anywhere.

What exactly I'm proposing? Url has two elements: domain and path, since you can create unlimited number of S3 buckets for free, you can create a bucket for every deployment with version in its name. So your static url will look like

I won't include particular examples in this post but basic workflow is like this: always populate AWS_STATIC_STORAGE_BUCKET_NAME setting from environment or some similar source, create a bucket when a deployment starts, make sure new version (it could be git hash or anything, just like with any other tools) is available as environment variable and previous one is somehow available too, run collectstatic (it will be using new bucket but currently-running application will still use the old one), reload application when it's done, destroy the old bucket once every host is reloaded (if running on more than one server). Multi-server environments will probably need some way of communicating for destroying old buckets effectively, but it's beyond this short post. Other than that, all you need is some way of shuffling two environment variables (or something), a couple settings, and two very short custom management commands (for creating and destroying buckets), and IAM role for the instance it's running on with appropriate policy.

Is this much better than using a directory in a single S3 bucket? No, not much. Url could be somewhat shorter (bucket names must be unique and by adding version to them it could be easier not to clash with other users), garbage collection is easier (you just remove the whole bucket, no need to do any file operations), but that's about it. IAM policy will be a bit more complicated, you'll need a little bit of additional code, no way to use the same bucket for static and media files (which might be a bad idea anyway but still). Overall, I do not recommend this way for anyone who doesn't understand everything in my post, use it on your own risk, but I personally find this idea pretty neat.

Popular posts from this blog

UK plug and wall socket is probably the best there is

While we are on the topic of sockets and chargers. UK three-prong plug (type G) allows you to put heavier things into the socket and they will sit there securely. Also it's rated 250V/13A allowing more powerful appliances and probably safer than any other plug type  (not mentioned there that almost every socket has an off switch, as an additional safety feature, apart from all the shutters, insulation, and built-in fuses). (Photo from Wikimedia Commons) If the world ever comes to a standard AC plug type, I hope it will be either that or at least something as robust and safe. And, let's face it, the only two alternatives (EU and US plugs) are not that universal, there are dozens of variations (where to put the ground prong, how to ensure polarity, even how big the prongs themselves must be) that makes either harder to adopt as a single universal standard.

ORICO — small things that stand out

More than once I've written about stuff made by Big Brand manufacturers so I decided to balance things out by writing about a mostly-unknown but good things. In short, if you need something small like HDD enclosure or some cable or USB hub or a dongle of some kind, check if ORICO makes it and chances are you won't be disappointed. It's usually slightly more expensive than low-end stuff from aliexpress but, in my opinion, it's usually worth it. Disclaimer. This post is a shameless promotion of their brand although, unfortunately, nobody paid me for it or even sent me any free stuff. Guys from ORICO and not, if you read this, I'll gladly review whatever free goodies you send me :) I only do honest reviews so better make sure to send the good stuff. At this point I own a USB hub, three different 2.5" HDD enclosures, and a precision screwdriver set from them. And probably something else I don't remember. Here are the pics: USB hub in its natural habitat

Terrible experience with bluetooth headsets and what to do about it

If you own a Bluetooth A2DP headset and any kind of personal computer, good chance is you tried connecting them together. Isn't it nice to watch movies and listen to music free from cluttering wires? Well, good chance is you hated that experience and if that was the reason for buying the headset, you might've even considered throwing it away (or using it only with your phone). Why do they suck so much? Everything points to the fact that it's not a hardware problem. The same headset most likely will work flawlessly paired with an android device or even the same computer under a different operating system (windows users report huge difference between, like, 8.0 and 8.1, and not in favor of the latter, surprisingly, I think only mac users report good stuff about their macbooks and beats although I didn't try it myself so they might all be faking it :) ). And most likely it's not even the drivers or other low-level stuff, android and desktop linux has mostly the same