Google Drive, Node.js, and a custom Electron app
An agency-wide move to Google Workspaces opened up a new set of problems for our team in maintaining project workflow.
Moving to Google Workspaces
Here at Bluecadet we recently upgraded to using Google Workspaces, which included the ability to use Google Drive to store all our project files. Previously, these files were scattered across an internal server, Dropbox, and individual Google Drive accounts. Our team created a flexible file structure template for our projects with directories for supporting our most common workflows and file shortcuts to key documents and files. Unfortunately, testing revealed a problem: a simple copy paste of the parent directory did not preserve those shortcuts (they referenced the original directory in the template project, rather than the new one in the copied project).
Shortcuts
Let’s take a step back and look at what exactly a shortcut is and why it’s a problem. A shortcut is a link that references another file or directory. In a complex file structure it is sometimes handy to have a quick shortcut to another directory multiple levels or directories away. For example, if you have a directory containing notes for developers, it might be nice to have a shortcut to that same folder under your client discovery directory.
However, when you copy the file structure, the shortcut remains pointing to the directory in the template structure and not the new directory in your new project.
Our Google rep suggested that we use the API to help maintain the directories and shortcuts when copying them to a new project folder. And this is where I entered, tasked to make it all work! As a web developer typically deep in PHP code, I was excited to dive into something a little different, the Node.js world.
Breaking down the problem
Here is our little project scope: We had a specific shared drive. In that shared drive we had a directory that contained multiple templates, depending on project type, with each containing about 80 directories and shortcuts. So, I had to be able to copy any of these templates. The templates were a work in progress so it was important to be able to just copy and pate the template structure in Google Drive, instead of hard-coding the templates in a config file. Oh and, of course, it had to be done ASAP so we could get the team using the new structure.
The first step was just getting connected to the Drive API. In the past I’ve always had problems dealing with authentication and Google APIs, however, their basic node.js example got me up and running pretty quickly and to be honest that was not as terrible as I thought. Pleasant surprise!
Directory wrangling
Second step was simply getting a basic understanding of how Google treats files and directories. I played a lot. After I had about 20+ tabs open with pages from different parts of the Drive API documentation, I was getting pretty familiar with it and started to dive into my actual task, copying an entire directory structure. And now we move from the nice little contained Google examples into the fun of a real project. It never works as easy as their examples. I created a test directory structure, all 5 directories, 2 levels deep.
The first hurdle was efficiently getting the directory structure. There is no way to call the API and say give me all the children of a directory. I can query for EVERYTHING on a shared drive and extract what I need, or recursively work through the hierarchy. In order to limit my total API calls I chose the first option.
After that hurdle, it was a pretty straight forward process. Find a directory, copy it. And I was sort of done! Or so I thought…
A long way to shortcuts
The next step was to create the shortcuts. This is also pretty simple, except for the fact that the directories need to already be created. Now I had to manage ids; ids from the original template structure and the ids of the new structure and their relationships. Small hurdle, but added complication. But note that Google does provide a handy method to generate IDs so I can build this relationship structure before actually calling all the API endpoints to create directories and shortcuts.
At this point we are talking about a few hours of work, nothing crazy, and I am feeling really good about it and I take a step back. All of a sudden it hits me — since this is a Node script, how are project managers, the ones creating new projects, going to do this? I don’t want to be bugged every time we start a new project. And I definitely don’t want to have to try and train project managers on installing and running a NPM package. I can’t even imagine. Next hurdle.
Creating a tool for the team
There are a few options out there that I could have done, however I’ve been really interested in Electron.js apps and its potential. So not getting any advice or discussing it with anyone, and thinking it would only take an hour or two, I went right in.
Needless to say, I went from dealing with hurdles to hitting a number of walls… but it was so much fun. I do like a challenge. And, if you are asking, of course I underestimated the 1 or 2 hours to convert it.
First major wall was getting Google authenticated within the Electron App. Electron is a browser but also, it’s not. And it doesn’t like playing with things outside of its little world. You know, security and all. I’m sure there was a better way to do it, but I eventually got it working. I do think this is what took me the most amount of time transferring everything to Electron.
The next wall for me was fully understanding how Electron’s main.js file interacts with the window you are actually seeing. Initially it was so perplexing. It also could have been that it was midnight and I was on hour 6 of what I was sure should have taken me an hour. Probably not though. Needless to say, when I took a break, let my brain recoup and actually read the documentation, it all started to become clear.
The next steps were to build out the UI a little bit, and how a user would choose a template and enter the new project title. Nothing Earth shattering here, but it was really fulfilling to see this “thing” come to life. Command line tools are really cool and powerful, but all of it is so hidden. It is very underwhelming at times. I did all this coding, I run the process and all I get is a new blank command prompt. Yeah! It wasn’t much but adding the UI felt good.
Now, of course, I jumped the gun and told the higher ups it was done and working. They did want it ASAP. Also, do you remember I said my test was 5 directories? And the actual template structure was 80+? Well, 10 minutes before I go to show it off, I switch it over to the real template directory structure. And it stalls… nothing. No output (I threw in a lot of debug messages as it is creating the directories) after the first 10 directories.
Well, after my “oh crap” moment, and rescheduling some meetings, I went back in. So, of course, I tried creating way too many new projects to debug what was going on: Will this work_001, Will this work_002, How about this one_003, please God let this one work_004, WHY WONT THIS WORK_005, did that actually work or was I not paying attention_006. Since this was in the shared public drive, I did get a few people asking me about my mental health. I was fine, just tired of naming things Test ###.
Error handling
Unfortunately, I didn’t take as close a look at the error messages as I should have. Another note, set proper reject methods and messages for your js Promises. It could save you time when debugging. Apparently copying 5 directories you do not hit any of Google API’s rate limits. When copying 80+ you do hit those limits. That fixed, we were good to go, but just running slower in order to not hit the limit. Also to note, you have to make one call to the API for each directory creation, instead of sending all the data at once. Kind of annoying.
Long story short I had a lot of fun “playing” and building up my js skills and after a solid week’s worth of work, we had a little desktop (Mac only) app to copy directories in Google Drive. We plan on building it out a bit more in the future, and I look forward to the new challenges.
