S3 Upload | AWS + .NET | Part 1

Fardeen Khan

Published in

Code Panthers

7 min readMar 31, 2022

This is Part 1 of a series. All the links are below

Part 1 — S3 Upload Use Case (You are here)
Part 2 — SNS Notification
Part 3 — CloudFront Signed URLs

In this article, we are going to explore the integration of AWS services, especially S3 in an ASP.NET Core WebApi using the AWS SDK.

We will be developing a simple WebAPi that allows the user to upload a file to S3 and get a Pre Signed URL to access the uploaded file.

Handy Dandy Tools

AWS S3 is the most popular and easy to use Storage Service on AWS, allowing users to upload Objects of up to 5 TB in size while still offering great durability and security features

ASP.NET Core (now replaced with .NET 6) is still one of the most popular cross-platform web solutions, whether your goal is quick and robust API development or Web Application design.

Finally, AWS SDK is the gateway for all the programmer folks to command the power of the AWS services at the binary tips of their code. It allows them to seamlessly integrate AWS services through a well-designed interface.

Prerequisites

Create a new AWS account if you don't have one already. Make sure to watch out for unnoticed charges. A Budget always comes in handy

Have Visual Studio 2019/2022 installed with the ASP.NET and web development module selected. Also, install .NET 3.1 (the latest version supported by AWS SDK)

Create your first S3 bucket

Files that we are going to upload to S3 will have to place in a Bucket that already exists. So let's head over to the S3 service in the AWS console and create our first bucket.

Please note that names of S3 Bucket need to unique globally .i.e., no two buckets will ever have the same name across AWS

Click on the Create Bucket button to get started.

Bucket Name: Choose something unique like sdk-net-upload-service
AWS Region: Select the one closest to your geographical location, for me that would be ap-south-1 (Asia Pacific — Mumbai)
Enable Server Side encryption. For this tutorial use SSE-S3.

Leave the rest of the configuration as it is. By default, your bucket will be public and no one will have access to it.

Once you create the bucket it will appear in the list of buckets on the S3 Homepage

Select the bucket you just created
Go to the Properties section and copy the ARN value
It would look something like this: arn:aws:s3:::sdk-net-upload-service

The ARN will come in handy in the next step

Setup AWS Roles and S3 Bucket

When you create a brand new AWS Account what you get is a Root user account (kinda like the one ring to rule them all), it's powerful and dangerous in the wrong hands. Before I start throwing LOTR memes, let's move on.

So basically what you need is to create a new user with minimal permissions. Our goal is to upload objects to the S3 bucket we created and also download them. So the following are the actions we need permission for

s3:GetObject
s3:PutObject
s3:ListBucket

Also, we need to restrict these actions so that they apply only to the bucket we created.

Create a new policy that allows the user to perform the above actions
Go To IAM > Policies > Create Policy
You can use the Visual Editor and select the actions manually or you can use the JSON editor and paste the following code

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "S3AccessPolicyForUploadService",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::sdk-net-upload-service/*",
                "arn:aws:s3:::sdk-net-upload-service"
            ]
        }
    ]
}

Review the changes and Save the Policy. Lets name it S3AccessPolicyForUploadService

Next, we need to create a User and assign this policy. For IAM best practices, we will create a new User Group and assign the policy to it. By doing so, any users that we add to this group will automatically have the same policy assigned to them.

Let's create a User Group first

Go To IAM > Access Management > User Groups >Create User Group
Set a User Group Name like UploadServiceDev
We will add Users later on, so leave that as it is
Attach the policy that we just created: S3AccessPolicyForUploadService
Create the Group

Let's create a User now

Go to IAM > Users > Add Users
User Name: dev (let's keep it simple)
Access Type: let's select Programmatic access for now
We will receive an AccessId and a SecretKey (Make sure they are safe and secure)
In the next section add the User to the group we created
Skip the remaining steps and Create the User
Download the CSV file for User credentials, this will be very important later on

With these credentials, we can use AWS SDK locally. For that, we need to configure a credentials file. Just follow this guide for now (ignore the Java

Configure AWS credentials — AWS SDK for .NET (amazon.com)

Now let's do some Coding

Create a new ASP.NET Core WebApi Project

Open Up Visual Studio and Create a New Project
Select ASP.NET Core Web API as the project Template
Set the Project Name and Location
Add the following AWS NuGet packages with the latest versions
AWSSDK.S3 (v3.7.8.13)
For calling the S3 API
AWSSDK.Extensions.NETCore.Setup (v3.7.2)
For setting Default AWS Configuration and adding AWS Service Clients to the Project’s DI Container

If you are not familiar with NuGet packages and how to add them to your project head down to this article I have written

With this, the initial project setup is done.

Code It

AWS Configuration

First, we need to set up the credentials so they are picked up by the AWS SDK
Edit the appsettings.json file and add the following block of configuration

{
  ......,
  "AWS": {
    "Profile": "default",
    "Region": "ap-south-1"
  },
  "S3BucketName": "sdk-net-upload-service",
  "S3ObjectExpirationHours": 6,
}

Profile: The AWS credentials profile you have set up in local. Set to default if only one profile is added
Region: AWS region of choice. I am using ap-south-1
S3BucketName: The S3 Bucket we will work with
S3ObjectExpirationHours: Time in Hours after which the Pre-signed URL will expire

Now we need to set up this configuration in the Startup script and add the required AWS Service to the Dependency Injection container.

Edit the Startup.cs file and add the following lines

public void ConfigureServices(IServiceCollection services)
{
    .....
    services.AddDefaultAWSOptions(Configuration.GetAWSOptions());
    services.AddAWSService<IAmazonS3>();
    .....
}

Develop the File Upload Service

Now let's work on creating the Upload Service interface.

Create a Services directory in the project
Add an Interface class with a single methods declaration that returns the Pre-signed URL to access the upload file

namespace CodePanthers.AWS.S3.UploadService.Services
{
    public interface IFileUploadService
    {
       Task<string> GetS3ObjectPresignedUrl(IFormFile file);
    }
}

Now to write the code, create a new Class implementing this service. Here's the logic for uploading the file and generating a Pre-signed URL for it

Finally, we add the Service to the projects DI Container

public void ConfigureServices(IServiceCollection services)
{
    .....
    services.AddTransient<IFileUploadService, FileUploadService>();   
    .....
}

With this done we simply need to expose the Service through a controller and that will be all. So here we go !!!

Exposing the Service

Create a new Controller class UploadController
Add a Single POST that accepts a File from Multipart form data
Call the FileUploadService to transfer the file to S3 and return the Pre-signed URL as the response

Here's the Code

Lastly…..

That was all we needed to do to achieve the simple goal of using AWS SDK in a .NET Core API. To test out the code, simply run the project and do a Postman request