Upload Zip Files to AWS S3 using Boto3 Python library

2 min readSep 14, 2018

Originally published at https://jun711.github.io

Learn how to upload a zip file to AWS Simple Storage Service(S3) using Boto3 Python library.

Boto3

According to boto3 document, these are the methods that are available for uploading.

The managed upload methods are exposed in both the client and resource 
interfaces of boto3:

* S3.Client method to upload a file by name: 
S3.Client.upload_file()

* S3.Client method to upload a readable file-like object: 
S3.Client.upload_fileobj()

* S3.Bucket method to upload a file by name: 
S3.Bucket.upload_file()

* S3.Bucket method to upload a readable file-like object: 
S3.Bucket.upload_fileobj()

* S3.Object method to upload a file by name: 
S3.Object.upload_file()

* S3.Object method to upload a readable file-like object: 
S3.Object.upload_fileobj()

(The above methods and note are taken from boto3 document, and there is a line saying that they are the same methods for different S3 classes.)

Solution

What I used was s3.client.upload_file.

The method definition is


# Upload a file to an S3 object
upload_file(Filename, Bucket, Key, ExtraArgs=None, Callback=None, Config=None)

Example Code

You can use the following code snippet to upload a file to s3.

import boto3
s3Resource = boto3.resource('s3')

try:
    s3Resource.meta.client.upload_file(
        '/path/to/file', 
        'bucketName', 
        'keyName')

except Exception as exp:
    print('exp: ', exp)

You can use ExtraArgs parameter to set ACL, metadata, content-encoding etc.

import boto3

s3Resource = boto3.resource('s3')

try:
    s3Resource.meta.client.upload_file(
        '/path/to/file',
        'bucketName',
        'keyName',
        ExtraArgs={'ACL': 'public-read'})

except Exception as exp:
    print('exp: ', exp)

All the valid extra arguments are listed on this boto3 doc. I have them listed below for easier reference.

ALLOWED_UPLOAD_ARGS = [
'ACL', 'CacheControl', 'ContentDisposition', 'ContentEncoding', 'ContentLanguage', 'ContentType', 'Expires', 'GrantFullControl', 'GrantRead', 'GrantReadACP', 'GrantWriteACP', 'Metadata', 'RequestPayer', 'ServerSideEncryption', 'StorageClass','SSECustomerAlgorithm', 'SSECustomerKey', 'SSECustomerKeyMD5', 'SSEKMSKeyId', 'WebsiteRedirectLocation'
]

If you need help with boto3, you can join their gitter channel.

References

1) What is ExtraArgs for upload_fileobj? boto3 GitHub thread
2) Boto3 not uploading zip file to S3 python StackOverflow thread
3) python: Open file from zip without temporary extracting it StackOverflow thread

Support Jun

Thank you for reading! Support Jun

Support Jun on Amazon Canada

If you are preparing for Software Engineer interviews, I suggest Elements of Programming Interviews in Java for algorithm practice. Good luck!

You can also support me by following me on Medium or Twitter.