Allowing users to upload files on consumer applications is a common requirement. Services allow images, videos, CSV, and other file formats to be uploaded for further processing and usage. While designing the file upload solution, we should take into consideration the file content type, its size, caching strategy, and other security-related considerations.
While working on a client project, we had to allow users to upload their profile pictures from web and mobile applications. As the application was hosted on the AWS cloud, Amazon S3 was our storage solution. While designing the upload service, we cautiously decided to upload the files directly from the user's device to S3. And that’s exactly what the presigned URL is designed for. It’s also worth mentioning that the application was built using Golang.
After looking at AWS Go SDK documentation we discovered it only supports presigned PUT URLs and had a couple of restrictions for our usage. In this blog, we'll explore the limitations of presigned PUT requests, the need for a more flexible solution, and how we overcome these challenges by implementing a secure solution using presigned POST requests in Go.
What were we trying to achieve?
The goal was to enable seamless file uploads from the client device to Amazon S3, ensuring file uploads are secure and controlled.
Presigned POST requests provide a way to generate URLs and fields for file uploads, allowing us to set restrictions such as file size limits and content type, while still ensuring that only authorized users can perform the upload.
What roadblock did we hit?
Presigned PUT URLs, while useful, pose some limitations. They include essential information like the signature and expiration date but lack built-in mechanisms for setting limits on the upload size. This limitation becomes apparent when trying to impose restrictions on the file size or specify multiple content types for a single presigned URL.
In contrast, the presigned POST approach offers greater flexibility by leveraging a policy mechanism. This mechanism empowers the signer to define rules, such as specifying supported file sizes or multiple content types. Unfortunately, while supporting presigned GET, PUT, and DELETE URLs, the AWS Go SDK lacks support for presigned POST URLs.
We found presigned POST implementations in AWS Ruby and Javascript SDKs. But very limited help was available for Go.
How did we solve it?
To overcome the limitations and implement a secure solution, we developed and released an open-source go-presigned-post package. This package provides a convenient way to generate presigned POST URLs and fields for file uploads to Amazon S3 using HTTP POST requests. It includes features such as specifying expiration times, file content types, ACLs, cache control, and more.
Key Features:
Presigned POST Support: Generate presigned POST URLs and fields for secure file uploads.
Flexible Policy Mechanism: Leverage a policy mechanism for setting rules such as file size limits and content types.
AWS SDK Integration: Seamlessly integrate with the AWS Go SDK for a comprehensive solution.
Here's a simplified example of how to use the go-presigned-post package:
The go-presigned-post package offers a flexible and secure solution for presigned POST URLs in Go. With its support for policy mechanisms, developers can define and enforce rules for file uploads. Its seamless integration with AWS Go SDK makes it simple enough to be integrated with any Golang application.
Looking forward to your feedback.
