How To Upload Files Automatically To Drive with Python

I run a local website for Algerian students to find their documents online, a major issue I ran into is that, a lot of these documents were on Google Drive already and I needed a way to download, re-upload, sorting and sharing them on drive. I also needed to add each link of these files to my database, it was more than 600 documents.

I’m a lazy person who hate doing repetitive tasks, but yeah, all of us hate doing this, it’s just boring, quite boring actually, Imagine if you had to upload images with a certain pattern in their names every hour to Drive, or having to create a new folder every 30 minutes in Drive and so on…

Introducing Google Drive API :

Google has a great API for Drive, it can be quite hard to understand how it works first, but you don’t need to know how, we’re going to use a wrapper to make things much easier: PyDrive .

Before starting to use Google Drive API or PyDrive, you need to create a project in Google Developers Console, this is where Google will link your account with the API, provide you with keys to authenticate and track your usage rate.

Signup to the developers console using your google account, in the toolbar you will find a dropbox to select a project, right now you don’t have any project so it will ask you to create one:

Give it a name, if you have an organization like your company or college select it. Don’t worry if you don’t have one, it won’t change anything for us.

Now you should be redirected to your project dashboard, if there’s an error message telling you that you don’t have any project, just go to the blue toolbar, select your project from there.

Your dashboard should be empty, you need to select the APIs you want to activate for your project, we only need the Google Drive API, but if next time you want to play with Youtube, Maps, Gmail or any other Google App you can activate their APIs the same way.

To activate Drive API, press on the burger menu in the toolbar and select ‘ APIs and Services’ then select Dashboard:

Activating APIs for Google dev console

Search for the APIs you need to activate

Just type in Drive in the search bar, and click on the link to go to the API’s page, once there, enable it :

Once you enable it, you will be taken to a new screen :

Here you will see your API calls stats, it’s time to create our API & client keys, we use those when we make API calls so it can link up the call to our account, thus control our drive from our code.

We did not write any code yet, be we need all this setup before starting, just bear with me for now, the next step should be the last one before getting our hands dirty with code.

Getting API keys:

This is quite important for any Google API you will use, for each API you will need some keys, not just for Google APIs, even for Reddit, Twitter, Facebook and several other products.

To get these keys go to Credentials (it’s on the sidebar), and press “Create credentials” button, you will have a list of 3 options:

1. API key : Choosing this one will only let you access your app drive acount only.
2. Client ID: This one is the one you need in case you’re building a web app where users can link up their drive account, this lets you access other users accounts on their consent.
3. Service account: this one is only in case you have some kind of web service running on a server and communicating with other servers.

We need the 2nd one, even if you’d like to access only your own drive account, because the first one let you access your app drive account which is different from your own account.

But before doing that, we need to give our app a name, go to the ‘OAuth consent screen’ and give it a name in the ‘Product name’ field:

Now, let’s go back to the 1st tab, and create a Client ID, you will have to choose the type of application, just choose other as this will be a script forr our personal use.

Now you will be taken back to the credentials tab, this time you have a client ID and a client secret, both of these are important to authenticate our app through Python.

Note: it’s okay to communicate the client ID, but you always need to keep the client secret key for you, because yes you guessed it: it’s secret.

Like I promised, the setup part is done, it’s time to jump to the code !

Code Baby, Code:

I know it’s been a little bit long to setup our console the right way, but this is due to the complexity and large scale of Drive and the powerfulness of its API.

Let’s get to the code now, as I wrote earlier, we won’t be using the APIs directly because this would take a lot of time and effort, instead we will use a nice wrapper in python: PyDrive

Jargon Bonus: A wrapper just means a library with a set of enhanced function to handle API calls and make it easier to use through a given language.

Installing PyDrive:

To install PyDrive we will use python package manager: pip, to install any python package through pip just type the following in your terminal: pip install module_name , so for PyDrive it’s just: pip install PyDrive

Now you should have the latest stable version of PyDrive installed, it will also install google api client module for python.

Connecting to Google Drive with PyDrive:

First step before using your drive account through your program is to login to that account, PyDrive have made this extremely easy to do :

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
import os
g_login = GoogleAuth()
g_login.LocalWebserverAuth()
drive = GoogleDrive(g_login)

The first 2 lines import GoogleAuth class from the PyDrive.Auth module and GoogleDrive class from pydrive.drive, we create an object to handle authentication. The magic happens in the 5th line.

g_login.LocalWebserverAuth() will fire up your browser and navigate to a google login page, choose the account you want to access in your program, authorize the app, and you will be sent to a page saying that

The last line creates a Google Drive object to handle creating files and uploading them to drive, we need to pass the g_login object to the constructor to check if authentication was successful.

Now you’ve been authenticated successfully, it’s time to start playing with the API.

Uploading Files to a Drive Account:

It’s time now to upload our files to our drive account, to do so we need to create a Google drive file, fill it with our file content and then upload it.

You cannot upload files directly, you need first to get the files you want to upload from your computer using Python, to do so we will use Python’s awesome file IO features, to open a file we will use open(filename,mode) function, call it like this :

with open("path_to_your_file","r") as file:
    #do something here with file

This will open a file in read-only mode (that’s what the “r” stands for in the 2nd parameter) and call it file .

Let’s read its content, create a new drive file and finally upload it:

file_drive = drive.CreateFile({'title':os.path.basename(file.name) })  
file_drive.SetContentString(file.read()) 
file1_drive.Upload()

Make sure to put this inside the with block.

Let’s see what each of these lines do:

1- We are calling the CreateFile function on drive object, this create a google drive file or entity, a google drive file (GDF) is quite different than other normal files, it’s because it can also be a folder, a GDF is a google drive entity that could be either : a folder, a document, an image, a video…

Notice this expression os.path.basename(file.name) this gives the filename of our file without its path, we could have used file.name but this would include also the path:

file = open('path/to/file.txt')
print file.name
# Print path/to/file.txt
print os.path.basename(file.name)
# Print file.txt only 

So back to our topic, we’ve successfully uploaded our file to our drive account, you can get very creative with just this.

For example, here’s a code that gets all PDFs in a subfolder having a number in their filenames:

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
#Login to Google Drive and create drive object
g_login = GoogleAuth()
g_login.LocalWebserverAuth()
drive = GoogleDrive(g_login)
# Importing os and glob to find all PDFs inside subfolder
import glob, os
os.chdir("/docs")
for file in glob.glob("*.pdf"):
    print file
    with open(file,"r") as f:
     fn = os.path.basename(f.name)
     file_drive = drive.CreateFile({'title': fn })  
  file_drive.SetContentString(f.read()) 
  file_drive.Upload()
  print "The file: " + fn + " has been uploaded"
   
print "All files have been uploaded"

That’s all for this post folks, hope it was helpful, I will write another article on how to manage permissions, rename files and handle folders in drive with Python.

If you found this post helpful give it a clap, follow me to receive more in-depth python automation tutorials to turn into an automation wizard !