Creating a basic project roadmap
My SLTP 2020 Walkthrough: Part 2
Continuing from my previous article, at this checkpoint, I have two things with me:
- A primitive idea of what I want to build- Something related to applied AI, a framework that creates content, curates information, and interacts with social media.
- A reference job description that can help me convert my primitive idea into something more solid and industry-relevant.
Now, let’s read through the description of the job profile I chose.
OpenAI - Senior Software Engineer, Applied AI
Join the API team, a core group who work to bring OpenAI's technology to the world in partnership with other…
Some highlights and thoughts-
- From the first image, you can see, we are almost doing what they want- We are self-directed, owning the problem end to end, and acquiring knowledge to get the job done.
- So, OpenAI is building an API platform for its deep learning models. While I am aware of what an API is, even if you are not, you can easily find out via a simple google search.
- From my basic understanding, I can deduce the simple architecture of my project. An application that can access the deep learning models via an API and obtain the results which can be used in our social media interactions. Whenever I build something I look at reusability, hence keeping each component separate would help in that. It is a good practice.
- I finalized that my first step would be in creating my own API service. Using this API service, I should be able to create the content I want. Once the content is created, we can have another module that will do something like posting it on social media, etc.
- So, what tech stack do I use? I know python, but I noticed that OpenAI has a part of its codebase in Go (a programming language). I have never written even a single line of code in Go. So, I just googled, “Python vs GO, building an API”.
- I got a bunch of good articles to read and understand why I should consider using Go.
REST API Performance Comparison Python vs Golang
In this post, we're going to explore building APIs with Python and GO. We're going to put these two languages to the…
Why We Switched from Python to Go - The Stream Blog
Updated May 14th 2019 to better reflect improvements to Go in the last 2 years (package management, better performance…
Golang vs. Python: Which One to Choose? - DZone Open Source
Prior to beginning any project, most development teams go through several meetings to decide the best language for…
- To summarise the above articles, Go is better in terms of scale and performance. So, I have concluded that I will use Go for building the APIs. However, deep learning models may still remain in Python. I will explore how much of it I can make in Go, and how I can use Python functions directly in Go. In your context, at this stage, it is okay even if you do not know any programming language. Just understand the theoretical logic behind your choice.
- Next, it was mentioned that OpenAI’s services run on Kubernetes Clusters and are versioned in Terraform. I have never used them and have no idea about either of these. But I will explore to see if I can use the same tech to run my service.
We now have a basic idea of what tech stack to use. And then I started looking into how exactly I should design my API? I decided to create a RESTful API. I just googled about it and here are some results (All of them from the first page of search results) I referred to.
What is REST
REST is acronym for REpresentational State Transfer. It is architectural style for distributed hypermedia systems and…
How to design a REST API
Learning REST in pieces is one thing, while applying all those learned concepts into real application design is…
Creating a RESTful API With Golang
this tutorial demonstrates how you can create your own simple RESTful JSON api using Go(Lang)
These articles are a great starting point on this journey (especially, the third one). It gave me some basic idea of what I should do next- Creating the API specifications.
But, before that, I thought I will list down all the steps I should explore, and the questions I should ask myself along this journey- our road map.
- What deep learning models will I use to create/curate content- like news, art or poetry, etc?
- How do I create a program that receives input required to be fed to the deep learning model, get the output I want, and send it back to me?
- What are all the different APIs related to deep learning that I need to make? What is the functionality of each one of them?
- What are the APIs required for interacting with social media? For example, If I want to post the created content on Twitter, how do I do it?
- Once all of this is finalized, do the actual coding to build them. Notice that, still step 5, it is just me logically thinking what my product will do. If you are a non-tech person, just creating a solid product idea along with specifications and use cases in itself should give you a lot of experience in building something from scratch. You can actually pitch it to other developers and take the role of a product manager.
- How do I test my APIs after they are built? The debugging and enhancements.
- How do I host my APIs in such a way that other people or applications can use it?
- The finishing touches, and packaging the project into a usable product.
- Added to that, taking inspiration from the job description (Build tools to accelerate your own workflows), I have decided to make a User Interface for my APIs which can be used to configure different parameters and control how the APIs are used. For example, a UI where the user can generate content by simply clicking some buttons and then post it on their twitter from this application itself. This makes it easy to test the APIs as well.
I will think about the tech stack I should use for the UI once I complete building my basic APIs. Overall, this should give some good full stack experience. You might think there is so much complexity here. I’m not going to lie, it can feel quite overwhelming. But stick to your idea. Ask all the hows and whats you can think of. It is okay even if you have a longer list of questions and no clue about the answers to any of them. The best approach for any project you do is to break it down into very small stand-alone tasks, as modular as it can get. Read through this series and you will get a grasp of what I mean.
For people just getting into programming and feel a lot difficult to proceed further, I would suggest you take up something on a much smaller scale. For example, if you are also thinking to make APIs, you should be good even if you create just one-two simple ones.
At this point, it is completely okay if you do not know about any of the programming lingoes. Search on google, one keyword at a time. Even I don’t know a lot of things. Notice that so far, we are just thinking in terms of the product we want to build, and given a particular functionality, I am asking myself, “How do I make it happen?”- And that is exactly what I am googling. I call this question based learning. It works very well for me, and I hope it does for you as well.
If you are following the self-learning technology program along with me, here is a checklist for you at this stage.
- You now know, what exactly your project will do. What is the input? What is the output? Where and how it will be used? In my case, my input is some instruction or data. This could be a prompt to create more content or get some specific content from elsewhere.
- You know your tech stack. If you don’t know how exactly to work with them, at this moment, just knowing the names of these frameworks and the logical reason of why they were chosen should be enough. (For example, my reason for choosing Go)
- You have a basic list of steps to follow or the questions you have to ask yourself to make this idea into a reality. I have listed down my questions above.
While I do have some deep learning experience, let us assume I don’t know anything about it. So, how do I proceed further? I am not going to do any online courses. My approach here is a little different. Moving forward I will be clearly dissecting my thought process to find answers to all the questions one by one and build the project.
Find out more in the next article where I will make a more detailed project road map from this basic understanding.
For more details about SLTP 2020, visit-
Self Learning Technology Program 2020
Join us for an industry-relevant technical adventure with SLTP 2020
I have also created a peer to peer learning group on WhatsApp (Can be migrated to Telegram if more people are interested) to support your self-learning journey and the exchange of ideas and experiences with the community. There is a google form in the above article, which you can fill to get the invite link.