The Top Coding Assistant Platforms of July 2024
Just three months ago I released an overview of coding assistant platforms — The field has changed rapidly since then, and I’ve gained insights that deserve to be included in this list. Please enjoy this updated survey on the field of coding assistant platforms!
While plenty of developers utilize generalist chat windows such as Chat-GPT and Claude Chat, I’ve decided to limit this list to code-purposed services. I’ll be providing details on the nine platforms listed above (in no particular order).
Notable mentions include: CodeGPT, Codebuddy, Aider, Sourcery, Supermaven, Double, and Mentat.
Notes:
- This article does not include tools which are taking an “AI-Software-Engineer” (Beyond inner-loop) approach — in my experience thus far, these tools are not useful to the average professional software team.
- “Justin’s rating” is based off how useful I find each platform currently in my day-to-day work.
- I tested mostly using VSCode and a small amount on Jetbrains IDEs. Most companies seem to invest more effort into VSCode currently.
Github Co-pilot
Takeaway: I won’t go into depth here as there are tons of other Copilot reviews. Copilot is the most mature IDE-based coding assistant available. It is powered by OpenAI models and has a large set of reliable tooling features that capitalize on integration with github products.
Amazon Q
Amazon Q (previously AWS Code Whisperer) refers to both a workplace assistant as well as an IDE plugin.
Q has been designed with intention for serving enterprise AWS customers:
- Their models have been optimized for understanding AWS APIs
- Code generations include references to their likely source (to help avoid license infringement)
- Managers can set policies on risk models take, or fully opt out of any suggestions resembling public code
- Automated Java version upgrade (Amazon Q Code Transformation)
- Semantic-level security scans
The Amazon Q extension is free at the individual tier. The professional tier, at $19/user/month, provides policy management, embedding-based RAG, and ‘Amazon Q Code Transformation’. Amazon’s rights to train on user data can be disabled in both tiers.
Amazon Q Code transformation is intended to automate Java version upgrades. The service is in preview, and AWS has noted that they are in the process of adding more languages/framework capabilities. I did not test this feature, and there is a lack of 3rd party reviews online.
Takeaway: The recent transition to Anthropic models has made this service valuable. Code Whisperer‘s code completions are effective, it has quality security features, and it’s free for individuals! Further, it can be supplemented with customer-specific data in the professional tier. However, it currently lacks some valuable features other platforms have, such as context providers, and is low on customizability.
Tabnine
Tabnine targets a security-focused market — training their models exclusively on permissively licensed open source repositories, offering multiple deployment environments of varying security (including self-hosted on-prem and VPC), and providing admin tools with strict policy management capabilities.
On the downside, users who want to utilize locally-hosted models cannot do so using Tabnine. Further, the product lacks some of the important supplementary features — namely RAG.
Takeaway: Tabnine is a quality product, especially for security-focused enterprises — However, in my experience, is on the expensive end of the spectrum for what it offers.
Sourcegraph Cody
Sourcegraph began as a code/documentation search company, and recently adjusted much of their focus to their coding assistant product, Cody. This history helps explain why Cody has exceptional RAG capabilities.
Cody performs RAG using embeddings, keywords, or a blended solution (recommended). The embeddings are sourced from a pre-built search index that users can initiate at the beginning of their session.
Cody comes with tools such as inline chat, a prompt recipe toolkit, custom recipe generation, and natural language file search.
Takeaway: Cody probably has the best RAG system among all platforms. I found Cody to be easy to use and extremely useful, especially in scenarios requiring effective RAG. While using Cody I realized that not only does high quality RAG improve the quality of some uses cases, it also enables use cases.
Codeium
Note: There are two ‘Cod_ium’ coding assistants (Codeium and CodiumAI)(They are not associated)
Codeium is high-quality all-around. In addition, they probably have the best free tier offering. From my experience, Codeium’s context awareness system seems nearly as effective as Cody’s. They have pin-able contexts, and some @-mentions context providers.
Codeium also has begun work on components outside the IDE, such as an analytics dashboard — which is already quite valuable.
Takeaway: The platform has all the valuable features, access to the best-of-the-best models, top-tier RAG, isn’t buggy, and offers an enterprise tier (although at a slightly higher price). Further, Codeium offers the best free tier — which has contributed to the company building a strong user community.
CodiumAI
CodiumAI has branched out to a few products:
- “Codiumate”: an IDE-based coding assistant.
2. PR-agent: a git extension currently compatible with Gitlab, Github, and Bitbucket which provides auto-documentation, auto-labeling, and more forms of review.
3. Cover Agent: a planning and code completion agent
Previously, Codiumate was heavily focused on test generation — they provided an interesting iterative-generation feature. The feature showed some promise — however I found it not to be practically useful.
However, that whole section appears to be removed in the current version. The current version is built around their planning agent — which upon testing, is very interesting, but not useful for my day-to-day work.
Further, while trying out a few of the versions from previous months, I observe that the focus and design of Codiumate seems to change drastically every few versions. I also encountered multiple bugs in just a short time of testing.
Takeaway: CodiumAI is an innovator among coding assistant platforms — but their products changes so dramatically fast it’s hard to keep up as a user.
RefactAI
Refact is unique in that they provide methods for fine-tuning, as well as an open-sourced basic version of their platform capable of self-hosting. This open-source project allows users to easily setup a local server hosting LLMs, which can be interfaced with through Refact IDE plugins.
The Refact platform allows for model assignment, sharding, and GPU sharing. Refact is compatible with a large selection of LLMs.
Takeaway: Refact offers more configurability/flexibility and an easier way to test the waters of coding assistants than most other platforms. However, the extension lacks some supplementary features, has an unattractive UI, and is buggier that most others. I expect Refact to fill a very important market soon, but as of now, my opinion is that their product needs some refining.
Cursor
Cursor is unique in that its product is not an extension, but rather a whole IDE itself via a fork of VSCodium — The founders note making this decision because they foresaw future features coming in conflict with restrictions IDE-extensions are faced with. Note that your preinstalled VSCode extensions and settings can be automatically imported when you install Cursor.
As a symptom of only having one product (rather than a collection of extensions), Cursor doesn’t deal with the issue of inconsistency across different editors — Most platforms in this list have significant discrepancy between their products among different editors.
Cursor is additionally advertising non-standard products including
- Copilot++ — Autocomplete which predicts your next edit
- Interpreter mode (beta)— Which I couldn’t get to work
- Long Context Chat (beta)— Use Claude’s 200k context window
- AI Review (beta) — Performs a custom review of your code, which I think is quite useful.
Cursor also has a larger number of valuable context providers than many of the other platforms in this list.
Takeaway: Cursor’s product is smooth to interact with, highly customizable, bring your own api-key, and has some interesting non-standard features. Interestingly, they don’t have an enterprise version yet. Note: OpenAI is Cursor’s top investor.
Continue
Continue is distinguished by its emphasis on configurability and customization. The extension is bring-your-own-model and likely has support for the widest array of model providers among all coding assistant platforms. Rather than utilize a graphical settings panel, Continue initializes settings via a configuration file. In combination with a local hosting service, such as Ollama, users can run nearly any popular open source model locally.
Continue has features such as custom docs and custom prompts, which allows users to customize quickly. Further, the platform is open source, and accepts many contributions from users who want to customize further. Continue developers have prioritized VSCode thus far, and built a UI very similar to Cursor’s. Jetbrains extensions are also supported, however lag significantly behind.
Takeaway: Continue provides a smooth experience with autocomplete, chat, RAG, tons of customization options, and the highest amount of flexibility/configurability among all other platforms…. Oh did I mention that it’s free??
Note: Currently, Continue is the platform I use most often— I make use of the Azure OpenAI model endpoints hosted within my company’s cloud, so that their is no risk of exposing IP.
A Brief Commentary on Coding Assistant Trends
1. Specialization
Github Copilot stands out as the most widely adopted IDE-based tool by a significant margin, although certain other platforms are esteemed just as, or even more, highly.
We’re observing the trend that coding assistant platforms are not one-size-fits-all. These separations are caused by security limitations, design preferences, feature specialization, and more.
While each of these platforms has placed much of their focus on the two core products of autocomplete and sidebar chat, it’s thought by many that we may see platforms begin to specialize in certain areas with time.
2. Beyond IDEs
While I mentioned previously that attempts to automate the software engineer job function as a whole don’t appear to be effective currently, there are valuable end-to-end implementations of certain subsets within the software engineering process — examples include PR review and test-specific generation.
Aligned with the hypothesis of increased specialization, it is expected by many that coding assistants will next trend towards becoming experts in individual subsets of software engineering, rather than the field as a whole.
To accomplish these ‘more challenging’ tasks, agent-like behavior is expected to be required — meaning coding assistants will need to become more objective-oriented, create a more complex state, become capable of using a wider array of tools, and more.
In order to engineer such complexity, it may be that development as an IDE-extension is too constraining. Cursor is an interesting platform, in that they have chosen to build their product as a fork of VSCodium. Other companies are choosing to build some of their experimental/beta projects as webapps.
Hopefully this article was useful to you! Please leave a comment if you think I left something important out, find an error/inaccuracy, or have any other feedback.
Youtube: https://www.youtube.com/@aiwithjustin2897
LinkedIn: https://www.linkedin.com/in/justin-milner-b190467b/
