Create a Serverless AWS S3 API Gateway Upload Portal with Frontend and Hosting Options

Author

Posted Oct 27, 2024

Reads 610

Computer server in data center room
Credit: pexels.com, Computer server in data center room

Creating a serverless AWS S3 API Gateway upload portal is a great way to simplify file uploads, and it's surprisingly easy to do.

You'll need to start by setting up an API Gateway, which can be done in just a few clicks.

This will provide a REST API that you can use to upload files to S3, and it's fully managed by AWS, so you don't need to worry about scaling or maintenance.

With API Gateway, you can also set up authentication and authorization, so only authorized users can upload files.

To host your frontend, you can use Amazon S3 Static Website Hosting, which is a cost-effective and easy-to-use option.

This will allow you to host your frontend files, such as HTML, CSS, and JavaScript, and make them accessible to users.

Alternatively, you can also use Amazon CloudFront, which provides a global content delivery network that can improve performance and reduce latency.

CloudFront can be used in conjunction with S3 Static Website Hosting to provide a fast and reliable hosting solution.

Serverless Setup

Credit: youtube.com, How to upload files to amazon S3 using API Gateway and Serverless Framework

Setting up a serverless environment on AWS is a breeze, thanks to the API Gateway and Lambda functions. You can create a new function and declare an "HttpAPI" event with the wanted path and method, and a reference to the API Gateway.

API Gateway acts as a proxy between the web and your code, exposing endpoints and triggering your Lambda function when called. From a Lambda function's perspective, it's just another event with a specific event payload.

To create an API Gateway, you can use the AWS::Serverless::HttpApi type in your SAM/CloudFormation template. This type is simpler and more performant than the AWS::Serverless::Api type, but still provides the necessary features for your API.

Here are the key parameters you need to configure when setting up API Gateway:

You can also use the AWS Serverless Application Repository to deploy API Gateway directly. This is an easy way to get started, but make sure to configure and verify the necessary parameters, such as replacing static files in your project with the ones you want to use.

Credit: youtube.com, Serverless Web Application on AWS [S3, Lambda, SQS, DynamoDB and API Gateway]

Using AWS SAM is another option for setting up API Gateway. You'll need to ensure you have the latest AWS CLI and AWS SAM CLI installed and configured, then download or clone the API Gateway Serverless Developer Portal repository. Update the Cloudformation template file, and configure the necessary parameters, such as ArtifactsS3BucketName and DevPortalSiteS3BucketName.

Once you've set up API Gateway, you can use it to trigger your Lambda function and upload files to S3. This is a powerful combination that allows you to create a serverless upload portal on AWS.

API Configuration

API Configuration is a crucial step in setting up your serverless AWS S3 API Gateway upload portal. You can create an API using API Gateway, which acts as a proxy between the web and your code.

To expose an API on the web, you don't need an always available server, thanks to AWS. API Gateway can trigger a Lambda function, forwarding the call's parameters and returning the service's response.

Credit: youtube.com, Create a REST API with API Gateway and Lambda | AWS Cloud Computing Tutorials for Beginners

The API Gateway is created using the AWS::Serverless::HttpApi type in the SAM/CloudFormation template. This type is the most barebone but also the most performant and cheapest option, about three times cheaper than the AWS::Serverless::Api type.

The Lambda function is triggered by the API Gateway and takes a specific event type as an argument. It returns a structured object with the status code, body, and potential headers needed for the REST response.

The event payload for the Lambda function includes contextual information about the API call, such as the path and query parameters used.

File Upload

The method of uploading files to S3 is not set in stone, and you have options to choose from.

You can use the pre-signed URL approach, but it's less customizable than the method shown here, which consists of two simple HTTP POST requests.

The pre-signed URL approach lacks the ability to specify the minimum and maximum file size, which is a must-have in some cases.

Using Sar

Credit: youtube.com, ChatGPT NEW Feature: Upload and Analyze Files (Full Tutorial)

Using SAR for file upload in your developer portal is a great option. You can deploy api-gateway-dev-portal directly from AWS Serverless Application Repository.

To configure the static files, you'll need to replace the existing files in your project with the ones you want to use. This includes custom content, logos, and other static assets. You can find these files in dev-portal/public/custom-content and lambdas/static-asset-uploader/build.

The static assets you'll need to configure include:

  • api-logo: contains the logos you'd like to show on the API page (in png format)
  • content-fragments: includes various markdown files comprising the content of the different pages in the portal
  • Other static assets: including favicon.ico, home-image.png, and nav-logo.png

To verify the configuration, open Amazon Cognito User Pool console and select the User Pool created for your developer portal. From the General Settings > Users and Groups page, select the User you want to promote as Administrator and add them to the Admin group.

Alternative File Upload Methods

You've got options when it comes to uploading files, and it's worth considering them. S3 offers multiple ways to accomplish the same task, so choose the one that fits your needs and environment.

Credit: youtube.com, Every FastAPI File Upload Method | Working and Best Practices!

The method we've shown here, on the client side, involves two simple HTTP POST requests, which can be done without using a whole framework or package. This approach keeps your client app build light and efficient.

You might've also heard about the pre-signed URL approach, but it's less customizable than the pre-signed POST data method. Note that not all operation parameters are supported when using pre-signed URLs.

The pre-signed URL approach lacks the ability to specify minimum and maximum file size, which is a must-have in some cases. This is a notable feature that's missing in the pre-signed URL method.

Portal Setup

To set up a portal with AWS API Gateway, you'll need to create a custom domain name for professional projects. This will provide a simpler and more intuitive URL that you can share with API users.

Make sure to update your API Gateway domain name in the Route53 record set created after setting up your custom domain name.

Credit: youtube.com, Upload large files to S3 with API Gateway and Lambda: Overcoming Size Limitations using Signed URLs

Using AWS SAM is another option for setting up a portal. You'll need to ensure you have the latest AWS CLI and AWS SAM CLI installed and configured.

The AWS SAM CLI will guide you through the setup process, which involves downloading or cloning the API Gateway Serverless Developer Portal repository.

Here are the key parameters you'll need to update in the CloudFormation template file:

Once you've updated the template file, you can create a ZIP file of your code and dependencies and upload it to Amazon S3. This will generate an AWS SAM template packaged.yaml that references the S3 location where the artifacts were uploaded.

Credit: youtube.com, Amazon API Gateway | S3 Bucket | AWS Lambda | API Endpoint for File Upload to Amazon S3 Bucket | AWS

To deploy your portal, run the following command from the project root, replacing the placeholders with your actual values:

  • {your-template-bucket-name} with the name of your Amazon S3 bucket
  • {custom-prefix} with a globally unique prefix
  • {cognito-domain-or-prefix} with a unique string

Note that you'll need to have the required privileges to make deployments, as the process will create various resources such as AWS Lambda, Cognito User Pool, IAM roles, API Gateway, and Cloudfront Distribution.

After deploying your portal, you can open the AWS CloudFormation console, select your stack, and open the Outputs section to find the URL for the developer portal specified in the WebSiteURL property.

Security and Permissions

We create a new IAM role and policy that grants the API Gateway the necessary permissions to upload objects to our S3 bucket.

This policy emphasizes the principle of least privilege, ensuring the API Gateway has no more access than necessary. We define the policy, then attach it to our newly created IAM role, empowering our API Gateway to interact with the specified S3 bucket.

The policy is attached to the IAM role, giving the API Gateway the required permissions to interact with the S3 bucket.

Enable CORS

Credit: youtube.com, Learn CORS In 6 Minutes

To enable CORS on an API resource, you can follow these steps. There are two ways to do it, but the easiest way is through the API Gateway console.

You can enable CORS for all methods under a resource or just for a specific method. To do this, open the API Gateway console, select the API Gateway for your API, and choose the resource you want to enable CORS for.

Select Enable CORS from the Actions drop-down menu and fill out the Enable CORS form. Leave Access-Control-Allow-Headers and Access-Control-Allow-Origin headers to their default values and click Enable CORS and replace existing CORS headers.

You'll need to review the changes in the Confirm method changes popup and choose Yes, overwrite existing values to apply your CORS settings. This will enable CORS for the selected resource and you'll see a mock integration on the OPTIONS method.

You must also enable CORS for ${proxy} resources. To verify that CORS is enabled, try using curl on the OPTIONS method.

Setting Up IAM Roles and Policies

Credit: youtube.com, Overview of Users, Groups, Roles and Policies | AWS IAM

When setting up IAM roles and policies, it's essential to follow the principle of least privilege. This means granting the API Gateway only the necessary permissions to perform its tasks.

First, create a new IAM role and policy that grants the API Gateway the necessary permissions to upload objects to an S3 bucket.

To do this, define a policy that outlines the specific permissions required, such as the ability to upload objects to a specified S3 bucket.

Attach this policy to the newly created IAM role, empowering the API Gateway to interact with the specified S3 bucket.

This setup ensures the API Gateway has no more access than necessary, which is a key aspect of security and permissions.

By following these steps, you can establish a secure and controlled environment for your API Gateway to operate within.

Testing and Deployment

Testing the setup involves sending a PUT request to the invoke URL, which serves as the gateway for uploading images to the S3 bucket. This can be done using Postman or a similar tool.

To confirm a successful upload, look for a 200 OK response, which indicates that the image has been successfully uploaded to the S3 bucket. The image should also be present in the bucket.

Deployment and Testing

Credit: youtube.com, Deployment And Testing Process - Functional Tests

You've completed the configuration, now it's time to deploy the API to a 'dev' stage. This will give you an invoke URL that serves as the gateway for uploading images to the S3 bucket.

The invoke URL is crucial for testing the setup, and you can use Postman or a similar tool to send a PUT request to it. Include the bucket name and object key in the URL path, and you'll be able to confirm a successful upload by a 200 OK response and the presence of the image in the S3 bucket.

To test the file size condition, try uploading files that don't comply with it. If the file is smaller than 100B, you should receive a specific error message. On the other hand, if the file is larger than 10MB, you should also receive a specific error message.

Uploading a file that's in the allowed range should result in a 204 No content HTTP response, and you should be able to see the file in your S3 bucket.

Evaluating Potential Solutions

Credit: youtube.com, The Evolution of Testing: Evaluating the Crucial Role of Testing in Technology

As we consider the best approach for uploading images, we must weigh the trade-offs between cost, latency, and functionality.

API Gateway can be linked to a Lambda function to manage the file transfer, but this comes with added complexity and cost.

In contrast, direct integration with S3 offers reduced latency and cost, making it an attractive solution for image uploads.

This approach is particularly viable given the 10MB limit on API Gateway requests and our 4MB image constraint.

Direct integration with S3 also bypasses the need for Lambda, further reducing latency and cost.

Here are the two potential solutions side by side:

  • Linking API Gateway to a Lambda Function
  • Direct API Gateway to S3 Integration

We've decided to opt for the second approach, Direct API Gateway to S3 Integration, as it's the most cost-efficient solution.

Ismael Anderson

Lead Writer

Ismael Anderson is a seasoned writer with a passion for crafting informative and engaging content. With a focus on technical topics, he has established himself as a reliable source for readers seeking in-depth knowledge on complex subjects. His writing portfolio showcases a range of expertise, including articles on cloud computing and storage solutions, such as AWS S3.

Love What You Read? Stay Updated!

Join our community for insights, tips, and more.