When GPT-3 as a Service, Implication for Jobs and Society

From API to AI as a Service

Cheng He
Cheng He
Aug 4, 2020 · 8 min read
Image for post
Image for post

In the previous article <GPT3 The Dream Machine in Real World>, I covered some highlights about what is GPT-3, why GPT-3 and OpenAI’s API is a big deal and what are the possible use cases with some inspiring examples. I also touched a bit on why it creates a paradigm shift in the future of developing AI products and society. Several people followed up on that topic, so I will share some of my thoughts and further expand it.

The paradigm shift comes from 3 different angles for the possible changes with the debut of OpenAI API or the era of AI as A Service.

  • Changes to ML product development workflow
  • Changes to AI ecosystem, jobs and responsibilities
  • Changes to AI research topic and directions

Designing a novel model architecture, acquiring large volume of training dataset, spend thousands to millions of dollars on building your in house ML models from scratch with TensorFlow or PyTorch sounds familiar to you?

Reading through the papers of “Attention is all you need” (the visualized version), getting small set of task specific data, fine tuning on top of large pre-trained models like BERT, Elmo, GPT-2, RoBERTa, scaling up your infrastructure to server the fine tuned models on production, are these something your team is working on?

Now, these might be changed and accelerated with OpenAI API.

Let’s start with how a team build the products with ML system and workflow nowadays. Here is a simplified process consist of several stages.

  • Identify and define the problem you want to solve with ML system
  • Analyze the data availability, cost for acquisition and annotation
  • Collect, annotate, verify the data you need to train your ML models
  • Understand your data pattern and distribution, choose the ML framework and algorithm, design your model architecture, build initial version
  • Tune the model hyper-parameters, run the evaluation to achieve your acceptable launch metrics
  • Manage to deploy your models within your infrastructure, server the production prediction or batch prediction
  • Once the system is up running in production, you need to monitor the model’s performance, conduct the error analysis, feed the results into the whole process again for retraining and iterations

The following diagram shows a simplified workflow in sequence to compare with the workflow of using GPT-3 API

Image for post
Image for post
Traditional AI product development workflow
Image for post
Image for post
AI product development workflow with GPT-3 api

Now, let’s go through the same process with GPT-3 API in multiple stages.

  • Understand what GPT-3 can and cannot do for your product, identify and define the problem you want to solve, choose the success metrics and evaluation method
  • Prototype through the OpenAI’s playground console, curate examples, design the prompt, tune the API parameters
  • Integrate OpenAI API with your applications, tweak examples, prompt, and API parameters, run the evaluation to achieve your acceptable launch metrics
  • Launch your product, conduct the error analysis, feed the results into the whole process again for retraining and iterations

You probably notice that this workflow is more simple than the traditional ML pipeline, more importantly, it dramatically reduced the demand on ML experts, resources and development cost.

It is like integrating another third party API in your application, you only need to focus on API payloads (in this case will be examples, prompt, and api parameters) to achieve high quality predictions, no need to worry about the expensive, tedious and complicated process of data gathering, labeling, and model architecting, training, tuning, and serving, not even mention the tools and experts behind it.

As long as you have a well defined problem fits into GPT-3’s capabilities and strength, then integrate with the API, iterate to align with your success metrics, you should be able to build AI products even without in house scientists and ML engineers.

To summary, the new development workflow by using GPT-3 and OpenAI’s API has several benefits:

  • Potentially much cheaper (depends on OpenAI’s pricing strategy)
  • AI for anyone! (significantly lowered the barrier of AI adopting)
  • Powerful and super-intelligent (huge model with 175B parameters)

We saw the benefits of transfer learning and using the pre-trained large model in the ML development process, we saw the huge potential and benefits of integrating OpenAI’s API.

But what does it mean to you as a developer, scientist, product manager, entrepreneur, investors, or even other big cloud providers?

“From API to AI as a Service”

Currently, the API is in beta, team at OpenAI is working on defining the rules to prevent misuse, improving the service performance, figuring out the pricing model. If you wonder why they decided to commercialize the technology and release it in the form of API instead of open source models, check out their blog.

If OpenAI proved this new business model works, then it will bring significant changes to the AI community. Let’s go through some of the changes might impact us.

Cloud providers ☁

Instead of providing tools, workflows, and cloud resources to help developers train, evaluate, deploy, and launch theirs in house ML models. It might make sense for cloud providers like AWS, Google Cloud or Microsoft Azure to launch their own version of “GPT-3 API”, which can significantly reduce the developer frictions and time to market.

This cloud provider’s “GPT-3 API” could be another product line on top of their existing ML products/solutions to bring intelligence to everyone, makes AI for everyone possible.

Considering the cost and resources needed for training such a gigantic model, there will be limited companies can afford to do this. Though this may change with the evolving of infrastructure, hardware, algorithms to bring down the cloud computing cost and improve the efficiency.

Entrepreneur, Startups, Investors

Most of the startups do not have the luxury resources and fund to train the gigantic model even OpenAI open sourced it. It make sense to think how to innovate and create more opportunities on top of the API.

This is a great moment for entrepreneur and investors to explore new business opportunities during this paradigm shift. No matter it is creating new products/service to help adopting GPT-3, or leveraging the OpenAI API to improve the quality of existing ML system, launch new features they never thought of before.

Product Manager, AI Scientist, ML Engineers

The role of developers and product manager are critical in developing a successful AI products with GPT-3 API. Traditionally, the product managers identify the problem, define scopes, then help the AI team prioritize the most fruitful ML tasks. While scientist and ML engineers works on the designing the model architecture, developing the system that does well on the dev set to integrate with product. See the following summary from Andrew Ng’s presentation in NIPS.

Image for post
Image for post
Traditional AI PM and Scientist/Engineer Responsibility

What could be the changes of their responsibility in the AI as a Service era?

In the new era, a lot of heavy lifting tasks will be handled by the AI as a Service provider (like OpenAI API). Team can build on top of the giant or focus on other important ML problems that GPT-3 is incompetent.

As product manager

  • Able to identify the meaningful problem to work on, define clear scope about the tasks become more critical
  • Some of the new responsibilities like curating the examples, designing the prompt, work with developers on dynamically managing the prompt in the API requests will become a big part of the product’s success

As scientist

  • Helps team understand the technology boundary, identify the most meaningful problem to work on with the API
  • Work on other ML problems that GPT-3 can not handle with confidence
  • Work on tasks like how to reduce bias and harmfulness, how to bring inclusiveness and fairness to the AI predictions
  • Came up the solutions to post processing or moderate GPT-3’s predictions

As engineers

  • Prototyping to explore the API’s capabilities and other potentials
  • Build software to manage examples, prompt, session, and API parameters in all using scenarios
  • Develop software to wrap up OpenAI API, modularize it to make it more accessible and manageable by other application and teams
  • Built tools and libraries to make it more capable and scalable

As domain experts

People with deep domain knowledge will become popular. They will be the go to person for team in terms of curating examples and prompt design, or brainstorm new ideas the API can help.

There might be some new roles created specific for this type of task considering it requires domain knowledge and some of the creativeness. Looking forward to seeing what will change to the AI ecosystem.

Hate speech, fake news, racism, gender bias all these issues are generating tons of text on internet everyday, and those text could be used for training large unsupervised models like GPT-3. Not even mention someone use the model to generate deceptive, biased, or abusive languages at scale(happened with GPT-2). This is another reason OpenAI released GPT-3 in an API instead of open source is because team want to control the use cases to mitigate the harmful bias and negative effects of the model.

If you wonder what could go wrong with it, check some of the following examples from machine predictions.

Image for post
Image for post
Mexican restaurants end up with low sentiment score from word embeddings.
Image for post
Image for post
Bias towards stereotypical names
Image for post
Image for post
Image for post
Image for post
For more bias and fairness study, check out Vinod Prabhakaran’s research

In the not-too-distant past, there was no TensorFlow, no attention mechanism, no BERT. We saw rapid progress in the last 3 years due to deep learning, larger models, and leveraging unlabeled data.

GPT-3 and OpenAI API opened a new era of AI, not only from technology stand point, but also from how it reshape the ecosystem with new opportunities and bigger social impact. Now, it is time to build.

To learn more about what is GPT-3 and use case demos, check out their blog or my previous summary.

The Startup

Medium's largest active publication, followed by +775K people. Follow to join our community.

Cheng He

Written by

Cheng He

AI evangelist, engineer, entrepreneur, YC alum, clubhouse @satorii

The Startup

Medium's largest active publication, followed by +775K people. Follow to join our community.

Cheng He

Written by

Cheng He

AI evangelist, engineer, entrepreneur, YC alum, clubhouse @satorii

The Startup

Medium's largest active publication, followed by +775K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store