How to solve CORS problems when redirecting to S3 signed URLs

Background

The implementation of signed URLs on the frontend usually uses a 2-phase fetch. First, there is a request to the backend, asking to sign an S3 URL. Then a separate request is sent to the bucket to fetch the file.

const response = await fetch("<backend url>"); // 1
if (!response.ok) {
	throw new Error();
}
const url = await response.text();

const fileResponse = await fetch(url); // 2
// use the fileResponse

When I implement a solution like this, I always have a strange feeling that something is not right. Implementing signed URLs should be transparent to the frontend, but with a 2-phase fetch, it is not. Also, separating the signing and the usage encourages bad practices, like signing ahead of time and increasing expiration time.

Why can't the backend respond with a temporary redirect, such as a 303, with the location of the file? The browser would automatically follow the link and it would be transparent to the frontend code.

const fileResponse = await fetch("<backend url>"); // 1
// use the fileResponse

I did some investigation, and the reason is CORS (Cross-Origin Resource Sharing), which is a security mechanism that comes into play when the frontend makes requests to a different domain. But it is configurable, it just requires some planning.

3-domains setup

A common architecture uses 3 domains: one for the frontend, one for the backend, and one for the bucket the files are served from. This setup requires setting up for CORS, which means some headers must be returned from the backend and from the bucket.

The 2-phase fetch makes this process simple. The first fetch is affected only by its config and the headers return from the backend. Then the second fetch is only by its config and the headers returned from the bucket.

The usual case for the first fetch is to set credentials: "include" as without it there will be no cookies sent which makes it hard to check access. In this case, the backend has to respond with both Access-Control-Allow-Origin: <domain> and Access-Control-Allow-Credentials: true, which makes it fully CORS-enabled and lets the frontend to read the body of the response.

Note that the Origin can not be * when Allow-Credentials is true. This is according to the standard. It is vital to check the Origin header as failing to do so would let any domain to get access to the private files!

Then the second fetch does not need credentials and only needs to get Access-Control-Allow-Origin: * from the bucket. Even this is not required for opaque responses, such as showing an image. But if you need to get programmatic access to the contents, you need to enable CORS.

This is a straightforward setup CORS-wise as there is no need to consider how the backend and the bucket respond to a redirected request.

Let's see how instead of sending the URL in the body, sending a redirect would work!

302, 303, or 307 for redirects?

There are three options to send a temporary redirect: either a 302, a 303, or a 307 status code would do it. According to MDN, the difference is how they handle redirecting non-GET requests:

302 can change it to a GET, though there are no guarantees.

303 forces the redirected request to be a GET.

307 does not change it.

In most cases, either of them could be good as the backend gets a GET request. But what if you want to download something using a POST? For example, a tracked download which is conceptionally a non-idempotent operation. But S3 requires a GET request, so in this case, a 303 status code has to be used.

As I find it more robust, I'll use 303 in the examples below.

credentials: "include"

Let's start with the most common scenario, which is to include the credentials in the request to the backend. It then signs a URL and sends a redirect. The call in this case is:

fetch(<backend URL>, {credentials: "include"})

Since this is a CORS request with credentials included, the backend has to respond with two headers:

Access-Control-Allow-Origin: <frontend URL>
Access-Control-Allow-Credentials: true

If either of them is missing or different, the browser won't allow access to the response body.

The bucket can be configured in one of three ways:

• it does not send back any CORS headers
• it sends an Access-Control-Allow-Origin: *
• it sends an Access-Control-Allow-Origin: null

The bucket can not send back Access-Control-Allow-Credentials, which is a limitation of S3, but fortunately, it is not needed.

The following table shows whether the request body could be read for every configuration.

The rows show what headers the API sends: it does not send any CORS-related headers, on the second row it sends Access-Control-Allow-Origin: *, while on the last row it sends Access-Control-Allow-Origin: <domain> and Access-Control-Allow-Credentials: true.

The columns correspond to the bucket CORS configurations. On the first column, there are no CORS-related headers configured for the bucket. Then on the next column, it sends back Access-Control-Allow-Origin: *, while on the third it's Access-Control-Allow-Origin: null.

	Bucket: No CORS	Bucket: *	Bucket: null
API: No CORS	-	-	-
API: *	-	-	-
API: Allow-Credentials	-	-	✓

 

No surprise on the API-side, both Access-Control-Allow-Credentials: true and Access-Control-Allow-Origin: <frontend URL> is required. But on the bucket-side, only Access-Control-Allow-Origin: null works.

Why the null origin?

It turns out that redirecting to a different domain is a privacy-sensitive operation and as such the Origin header is not sent.

Allowing the null origin seems to be opening an attack vector, but it is not;

A Lambda backend implementation:

return {
	statusCode: 303,
	headers: {
		Location: signedUrl,
		"Access-Control-Allow-Origin": origin,
		"Access-Control-Allow-Credentials": "true",
		"Vary": "Origin",
	}
}

And the bucket CORS configuration, managed with Terraform:

resource "aws_s3_bucket" "bucket_cors_null" {
	cors_rule {
		allowed_methods = ["GET"]
		allowed_origins = ["null"]
	}
}

No credentials

Let's see what changes when credentials are not included in the request! While I don't recommend this setup as access control is the most important part of a backend that provides signed URLs, it's important to see how things change in this case.

In this case, the API still needs to return CORS headers, but a simple Access-Control-Allow-Origin: * would suffice.

After running the tests, these are the results:

	Bucket: No CORS	Bucket: *	Bucket: null
API: No CORS	-	-	-
API: *	-	✓	✓
API: Allow-Credentials	-	✓	✓

 

No surprise on the API-side, when CORS header allows the frontend origin the request works. But on the bucket side, both * and null works! But even if null works, I wouldn't use that.

A Lambda function:

return {
	statusCode: 303,
	headers: {
		Location: signedUrl,
		"Access-Control-Allow-Origin": "*",
	}
}

And the bucket config:

resource "aws_s3_bucket" "bucket_cors_star" {
	cors_rule {
		allowed_methods = ["GET"]
		allowed_origins = ["*"]
	}
}

2-domains setup

A step towards simplifying infrastructure and CORS is to use fewer domains. Let's investigate a setup where the frontend and the backend are on the same domain but the private bucket is on a different one. This can be configured with CloudFront.

In this case, there is no need to specify credentials: "include" as the request to the backend will be same-origin. Also, the backend does not need to send back any CORS-related headers.

But the redirected request to the bucket still needs some configuration.

The following table shows the results for this configuration:

	Bucket: No CORS	Bucket: *	Bucket: null
API: No CORS	-	✓	-
API: *	-	✓	-
API: Allow-Credentials	-	✓	-

 

No surprise on the API-side, no headers are required. On the bucket configuration, a simple Access-Control-Allow-Origin: * is sufficient. The null origin does not work in this case.

The Lambda code:

return {
	statusCode: 303,
	headers: {
		Location: signedUrl,
	}
}

And the bucket config:

resource "aws_s3_bucket" "bucket_cors_star" {
	cors_rule {
		allowed_methods = ["GET"]
		allowed_origins = ["*"]
	}
}

1-domain setup

But this config can be simplified even further. The private bucket can be served via CloudFront also which means everything is under a single domain.

No cross-domain requests, no CORS-related problems.

Conclusion

Cross-domain requests require some planning to prevent CORS-related errors, but ultimately it is a matter of sending back the right headers. I was surprised to see that the null origin is the only one to work in the 3-domains scenario, but that is a valid configuration also.

My recommendation is to use CloudFront to bring the different services under one domain and that, among bringing other benefits, solves CORS.