My Experience Completing the deeplearning.ai Short Course “Pair Programming with a Large Language Model”
Follow me on LinkedIn for daily posts.
Summary
Pair Programming with a Large Language Model from deeplearning.ai is a short course that covers use cases and prompt engineering for code creation and editing. Although it has limitations, the course should be beneficial for anyone who writes code and wants to learn how to use an LLM API to generate or edit code (versus a web interface or auto-complete plugin). The entire course can be completed in approximately 90 minutes and will provide a solid understanding of use cases and techniques for programmatically leveraging an LLM within a notebook.
Why I Took This Short Course
There is no shortage of LLM content, and like many I have a personal backlog of articles, lectures, and other content I want to consume. However, once I saw this course announced by deeplearning.ai in late September 2023, I moved it to near the top of my list.
For the last several months Andrew Ng and deeplearning.ai have put out several excellent ‘short courses’ about LLMs and related topics. When I was just getting started with the technology I found the courses he delivered with OpenAI and LangChain to be especially good. These, and others from deeplearning.ai, include the most important topics about a subject and come with notebooks that you can leverage to get started with your own work. On top of that, Laurence Moroney is a fantastic instructor and hearing him introduce this course was like seeing an old friend.
Course Overview
The course has four main parts, each of which is accompanied by a Jupyter Notebook you can edit and run to follow along with Lawerence’s lectures.
Lesson 01: Getting Started with PaLM
This section provides an introduction about how to programmatically interact with an LLM API. In this case, it is the Google PaLM API, but the lessons apply to any other LLM you might interact with via an API. The accompanying notebook demonstrates how to pass a message to a model with parameters, such as temperature, and then display the response. Lawerence then moves to demonstrate how you can pass a prompt to a text generation model, in this case text-bison, and ask it to return code.
Lesson 02: Using a String Template
I thought this section did a great job explaining the elements of a prompt template. The accompanying notebook demonstrates the following approach for using a single string containing three elements:
- priming: background or context to prepare the LLM for the task
- question: the specific task for the LLM
- decorator: how the LLM should provide or format the output
I like understanding an approach to prompt generation that does not rely on a third party library like LangChain (which is fantastic in its own right, but is not always appropriate for production or enterprise use). It was also useful to see how different ‘priming’ and ‘decorator’ text can significantly affect the the responses generated by the models.
Lecture 3: Pair Programming Scenarios
This is the longest section of the course, but also the most rich with content. It digs into several specific code generation or code editing scenarios which gave me new ideas for how to leverage LLMs in my day-to-day including:
- Improving Existing Code: You have an existing piece of code, such as a function or class, and ask the LLM to demonstrate alternate ways to implement it. One example provided was finding ways to make a Python function ‘more Pythonic’.
- Simplifying Code: You have code that has more steps or more functions than necessary, so you ask the LLM to simplify it. While closely related to ‘making code more efficient’ below, I view this as focusing on the overall process and implementation, rather than an efficient runtime.
- Writing Test Cases: This is a great use case that I had not thought about before. When you write a piece of code, you may be too close to it to think about test cases that could expose errors. By passing the code to an LLM, it can objectively devise multiple test cases, including edge cases, that expose issues in the original code.
- Making Code More Efficient: This use case goes hand-in-hand with ‘simplifying code’ but focuses on the computational runtime efficency.
- Debugging Code: Finally, this is a very common use case to identify and fix errors produced by code. I can see it combined with writing test cases (above) to identify potential errors and then updating the code to fix them.
The accompanying notebook demonstrates all of these examples, which are well explained by Lawerence during the lecture. Even if you don’t complete the entire course, I highly recommend this portion of it.
Lesson Four: Technical Debt
Two additional use cases related to code maintenance are introduced in the final section of the course. Like the five pair programming examples, I found these to be informative and applicable, regardless of the API or foundation model you are using.
- Understanding Code: Almost anyone who writes code as part of their job will encounter situations where they inherit a piece of legacy code or need to interpret or update code written by someone else. This can be challenging and time-consuming, especially if the code is complex and part of a larger workflow. In these situations, an LLM can be a very efficient way to gain a high-level understanding of that code. With larger and larger LLM context windows becoming available, entire scripts can now be passed to the model for a comprehensive explanation.
- Documenting Code: Documentation is closely related to understanding code and is an area where LLMs can often be quite helpful. In my current role I often find myself doing this to make code examples easier for customers to understand, even if it feels over-documented at times. Additionally, if you are writing code yourself the LLM can quickly add documentation that might take you several minutes to prepare yourself. Over time this can add up to a considerable time savings, while making your code easier for others to understand and maintain.
Areas for Improvement
While I found the entire course beneficial, there are several areas where I thought it could have been improved or expanded. I am also highlighting these areas as encouragement for those who take the course to continue exploring how to use LLMs to work with code, building upon what Lawerence presented during the course.
Explain and Demonstrate Code-Specific Models
This short course was developed in partnership with Google, which has emphasized deploying LLMs customized for different use cases. In particular, Google has deployed code-specific models for generation, completion, and chat. I was surprised to see that none of these models are mentioned throughout the course. One of my objectives for taking this course was to better understand how Google optimized models for working with code and when I should use code-specific models versus the standard text generation or chat ones. That said, the techniques demonstrated in this course should apply to using any code generation models, including Google’s.
Incorporate Chat Models
At the start of the course Lawerence notes that he is going to focus on text generation models, specifically text-bison. I understand that introducing conversation flows may have complicated the course, but based upon my experience there are situations where having a conversational model is beneficial for working with code. This is especially true with Google’s codechat-bison model. For the ‘debugging code’ as well as ‘understanding’ and ‘documenting’ code use cases, a conversational workflow would have made a lot of sense. I have often needed multiple turns in a conversation with an LLM when debugging my own code.
Combination Workflows
Combining the above, another potential opportunity for the course would have been to introduce combination workflows that start with a code generation model then move to a code conversational model. I have experimented with this in my own workflows, but have not determined when it is best to start with a generation model then move to a chat model versus starting and staying with a chat model. For example, a code generation model could be used to create a piece of code from a natural language prompt. From there, a codechat model could then be used to debug, optimize, and document that same piece of code. The combination of different LLMs, especially Google LLMs, is a topic plan to explore in a future post.
Conclusion
The deeplearning.ai course Pair Programming with a Large Language Model is a good introduction to using LLM APIs to help you generate and work with code. I definitely recommend this course for individuals who write code and are exploring how to use APIs as part of their workflows. I have been doing so for several months and took away several ideas that will help me going forward. While there are a few areas where the course could be improved and expanded, it is well worth a 90 minute investment of your time.
