Large files can play well with AWS Lambda

Serverless computing is about short functions that run fast and only for a short duration. On the other hand, files can be arbitrarily large, and sending them through ephemeral functions can easily hit a limit of the runtime.

This is the problem S3 signed URLs solve.

Learn how to implement a serverless-friendly, robust, and secure solution to handle files using signed URLs.

Want a sneak peek? Sign up for free chapters here:


Handling files in a serverless environment

Serverless functions are designed to be run for a short duration, handle small amounts of data, then terminate quickly. This is great to provide an HTTP API that users can call to interact with your app but is terrible for large downloads or uploads.

S3 on the other hand is to handle arbitrarily large files and can handle long downloads and uploads and terabytes of data.

URL signing is a mechanism to combine the best parts of the two services. The Lambda function decides who and how can download and upload files, but the actual transfer is handled by the S3 service.


"But a signed URL is just a function call!"

When I started working with signed URLs I quickly realized how easy it is to end up with a solution that is not reliable and not secure. This prompted me to learn how it works and what services are taking part.

Signed URLs change how you think about files and access control.

And the best part is that this knowledge is transferrable between clouds. The services and the details are different, but the underlying mechanism is the same for all providers that offer this functionality.


Learn how to implement a secure and robust file-handling solution

This book is a thorough introduction to signed URLs. You'll learn:

  • Why serverless needs a new model to handle files
  • What services work together when you sign a URL
  • Backend and frontend implementations
  • How to secure your implementation and what are the usual problems
  • Solutions to common use-cases

...and more!


Screenshots

Table of contents

Chapter 1: Overview
 Use cases
 From servers to serverless
  3-tier architecture
  Serverless architecture
 Signed URLs for S3
  Credentials
  URL structure
  Expiration time
 Security of S3 signed URLs
  Algorithm
  Bandwidth control
  Implementation disclosure
  Revocation

Chapter 2: Implementation
 Sample code
 Infrastructure
  Bucket
  Object
  Function
  Execution role
 Backend with Node.js
  S3 service
  Sign URLs
 Frontend

Chapter 3: Specific use-cases
 Least privilege with dedicated roles
 CORS
 Using HTTP redirects
 How to check if a file exists before signing
 How to set the filename
 Integrate with CloudFront
 Caching
 Uploading files
 Handling encrypted data
 Permanent URLs

Troubleshooting
 AccessDenied
 NoSuchKey
 ExpiredToken
 PermanentRedirect
 InvalidRequest

Security checklist


About the author
Tamás Sallai

I'm a software developer focusing mostly on cloud computing and web technologies. I'm especially interested in how to handle edge cases to end up with dependable software.

One of my main focus is security and how each part affects the whole system. I'm an AWS-certified security specialist.


Get the book

The book is available from these stores: