My experience with OpenDevin

The Setup of a AI Enginner. My $3 AI Engineer Trial.

Since April 2024, I have been following the further development of OpenDevin, the AI coding agent. Having not gained access to its original, Devin (Preview) from Cognition, until today and allegedly being misled by a promising promo video, I looked for alternative coding agents. In the open-source project OpenDevin, I liked the architecture and the initially used technology stack.

OpenDevin: Code Less, Make More

The idea of having an LLM work independently on my laptop excited me but also made me a bit anxious. In the April 2024 version, there was no functioning history or “Pause the current task” button yet. Nevertheless, OpenDevin already created files and folder structures for me or installed a runtime environment for Python. This works much better today.

Setup

Whether on my Windows machine (WSL) or the MacBook, you simply need to have the latest Docker Version (29.0) installed on both operating systems. A stable internet connection is helpful, as constant communication with the chosen LLM provider takes place after the start. With the use of LiteLLM, you have quite a bit of freedom in choosing your model. Whether it’s ChatGPT, Gemini, or Anthropic — all models seem supported by the LLM Backend.

Clone with Git (MacOs Sonoma 14.5)

Create a directory for cloning the Git repositorys. The specified revision number worked well for me.

cd documents
mkdir testing
cd testing
git clone https://github.com/OpenDevin/OpenDevin.git
(optional) git checkout 703b920286d4380551f678f9819de080cbe0cde6

Docker

Start Docker on your device. Pay attention to common issues like permission requirements and ensure that you don’t need to run Docker commands with sudo.

cd OpenDevin
WORKSPACE_BASE=$(pwd)/workspace
docker run -it \
    --pull=always \
    -e SANDBOX_USER_ID=$(id -u) \
    -e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
    -v $WORKSPACE_BASE:/opt/workspace_base \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name opendevin-app-$(date +%Y%m%d%H%M%S) \
    ghcr.io/opendevin/opendevin

Configuration

Configuration — OpenDevin

Agent: By default, CodeActAgent is suggested. For development, CodeActSWEAgent seems more suitable.

Model: I tested with gpt-4o. I was able to create the API key for it at OpenAi (User API Keys).

Testing

As usual, the AI Software Engineer greets you warmly and waits for my commands.

Landing Page — OpenDevin

For the first task, I wanted to have it create an Angular app:

Create a basic Angular App — OpenDevin

Its process could be tracked through 21 messages in the chat and was roughly as follows:

1. Checking and installing Angular CLI
2. Installing nodejs16 and npm
3. Removing nodejs16 and installing nodejs18
4. Installing angular/cli (both with and without sudo permissions)
5. Installing NVM
6. Reinstalling Angular CLI
7. Creating an Angular project
8. Displaying a prompt during the Angular setup.

ng --version
-bash: ng: command not found
npm install -g @angular/cli
-bash: npm: command not found
curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt-get install -y nodejs
sudo npm install -g @angular/cli
ng new my-angular-app --defaults
sudo apt-get remove -y nodejs
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
sudo apt-get install -y nodejs
ng new my-angular-app --defaults
sudo npm install -g @angular/cli
node -v
npm -v
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
sudo apt-get install -y nodejs
node -v
npm -v
sudo npm install -g @angular/cli
npm install -g @angular/cli
exec $SHELL
node -v
npm -v
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash
export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
nvm install --lts
node -v
npm -v
npm install -g @angular/cli
ng new my-angular-app --defaults
Command: "ng new my-angular-app --defaults" timed out. Sent SIGINT to the process: 
 Would you like to enable autocompletion? This will set up your terminal so 
pressing TAB while typing Angular CLI commands will show possible options and 
autocomplete arguments. (Enabling autocompletion will modify configuration files
 in your home directory.) (Y/n)

It doesn’t look too bad, does it?

Unfortunately, the Y/N input did not work, neither in the terminal nor in the prompt window. I also couldn’t manually execute the ng new command through the terminal. What the issue was – I have no idea.

Costs

For my tests, 17 requests were sent to ChatGPT. A total of half a million tokens were used.

Usage — OpenAI

Charges at OpenAI are based on the number of tokens used, which amounts to $3.30. Guess I’ll stick to free models for now and save my pennies for caffeine?

Cost — OpenAI

Conclusion

I was impressed with the AI Engineer’s progress in installing the Angular app, as the approach was understandable and largely correct. The AI Engineer did version checks, troubleshooting, and installations independently. As a software developer, the result is certainly a useful template. The progress over three months is remarkable! When it comes to costs, it’s important not to draw the wrong conclusions. OpenDevin is in a development stage, and it’s likely that resource-efficient prompts have not yet been prioritized. There are already many good, free GPT models available, and I hope this article has made you curious to test them yourself :)