How S3 Signed URLs work
Signed URLs provide secure a way to distribute private content without streaming them through the backend. Learn how they work and how to use them.
Let's say you have an object in an S3 bucket which is set to be private (i.e. no anonymous access). Then you want to share it with people who have no AWS accounts, for example, subscribed visitors to your website. This can be a video course that only paying users can access, or an EBook that requires subscription.
Making the object public would open it up to anybody, but keeping it private denies all non-AWS users. Bucket policies do not help either, as the users are not IAM users.
How to solve this problem?
Signed URLs
This is where signed URLs come into play. You sign the access URL on your backend, and distribute that only to the subscribed users. By making the signature valid for only a short period of time, you can also prevent reusing the same URL.
This way, you have explicit control over who has access to the resource.
Signing a URL
Let's see how this works code-wise!
As a test setup, I created a bucket called pres-url-test and uploaded a test.txt object to it. I also made it
private (no anonymous access) but added an IAM user that has access via an IAM policy.
As everything is set up, I grabbed the access and the secret key of the IAM user and used this code to generate a signature:
const s3 = new AWS.S3({
signatureVersion: 'v4'
});
const params = {Bucket: 'pres-url-test', Key: 'test.txt'};
const url = s3.getSignedUrl('getObject', params);
This code uses the AWS SDK, which works from both the browser and NodeJS.
In the former case, you can include the SDK via a script tag, like this: <script src="https://sdk.amazonaws.com/js/aws-sdk-2.1.12.min.js"></script>.
In case of NodeJS, use NPM and install from here: https://www.npmjs.com/package/aws-sdk.
One common misconception about signed URLs is making them requires a request to AWS. In reality, they work fully offline, so you can sign any bucket and object with any access and secret keys, there is no check that the resulting URL will work.
You need to provide the region, the keys, the bucket, and the object key for the signer. Optionally, you can also provide an expiration time, where the default is 15 minutes.
A resulting URL looks like this:
https://pres-url-test.s3-eu-west-1.amazonaws.com/test.txt
?X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Credential=AKIAJQ6UAEQOACU54C3A%2F20180927%2Feu-west-1%2Fs3%2Faws4_request
&X-Amz-Date=20180927T100139Z
&X-Amz-Expires=900
&X-Amz-Signature=f6fa35129753e7626c850a531379436a555447bfbd597c19e3177ae3d2c48387
&X-Amz-SignedHeaders=host
Using that URL opens the file even for anonymous users.
How does it work?
This works by signing an operation, in this case, this is the S3 getObject with the bucket and the object key as
parameters. You can sign other operations too, for example PUT allows uploading new objects.
If you look at the URL, you can find the Access key, but the Secret key is only used to generate the Signature part. In this regard, the access/secret key par is not like a username/password.
The signed URL on itself does not give access to the object in the S3 bucket, but it sends the request as the signer user. Effectively, it works the same as if the signer issued it.
So the presigned URL is only effective if the signer user has access to the action (getObject) and the resource
(bucket + key), therefore, if the permission is revoken later, the signed URL stops working. But if the permission is
restored after that, it will be working again.
In short, the signed URL is not guaranteed to work until it expires, and in fact, there are no guarantees that it works at all. It is all dependent on the permissions of the signer user.
How to revoke?
This brings us to the problem of revoking signed URLs.
The signature itself can not be invalidated, you need to wait until it expires. But you can revoke the signer user's permission to the object.
If you find yourself in the situation when a signed URL is compromised, revoke the permission. But this will prevent all signed URLs from working that were issued by that user for that resource. Which is suboptimal, but at least you can mitigate further damage.
What can you set?
For the getObject operation, which is what you'll use for distributing files, you can set the bucket and the object,
along with the expiration time (defaults to 15 minutes).
I couldn't find reference what is the maximum expiration time, but you shouldn't use long ones.
Best practices
Because of the way permissions are checked, you should use a separate IAM user that only has permissions to view the files you want to share. Usually, you'll have a bucket with all the protected files, so the IAM user will have access to read the whole bucket. Use the least privilige.
You need to protect the secret key. You can sign an URL in the browser, but sending the access+secret keys to the users defeats the whole process. Use a server-side signing endpoint, like a Lambda function that checks if the user has access to the resource (i.e. he is a subscriber).
URLs should expire as soon as possible. 15 minutes is the default, and it should be more than enough for most use cases. Sign the URL just before the user needs it, and that way you don't need to worry about expiration. For example, when the user wants to download a protected resource, generate the URL when he clicks on the Download button.
Conclusion
Presigned URLs provide a solution to constraining access to private content contained in S3 without the need to stream it through your backend. Using them relieves your backend from having to distribute large files.
But this is essentially a security feature, and as such, you need to be careful how you implement it. Know how they work and follow the best practices.
