How to Price an AI Project in 2023

What’s new and what stayed the same 6 years after my 2017 article in TDS

Daniel Shapiro, PhD
TDS Archive
Published in
11 min readJan 12, 2023

--

Image by 愚木混株 Cdd20 from Pixabay

This is an article offering hard-to-find insights for other people working in the AI/ML consultancy space. Almost no one is willing to talk about pricing or hiring practices. My 2017 article “How to Price an AI Project” provided insight into how to price a Machine Learning (ML) project for clients. But it’s been a long time since 2017! Before providing a 2023 update on how to price AI/ML projects in 2023, I’ll briefly summarize some key points from my previous article. It can be tricky to provide fixed price estimates for large ML projects, due to factors such as requirements changes, integration issues, data problems, and user acceptance. Executives and project managers should expect requirements and scope changes during the execution of ML projects. The bigger the unknown or requirements risk, the more you should lean toward hourly pricing rather than fixed pricing. Try and get the risks out of the way as early on as possible. Science first, and engineering second. Our standard rate for ML consulting as of August 2017 was $250/hr and projects had to be at least 5K (20 hours) in size. Our goal in 2017 was to hit our hourly rate even on fixed-price projects…. But now it’s 2023. Let’s see what changed and what stayed the same.

What’s The Same?

Surprisingly, our outlook is mostly the same as it was in 2017. It’s been a few years since that article went live, and it was cited by various sources like this and this. The artificial intelligence consulting industry has grown rapidly since 2017, and I’ve noticed that some things have remained consistent. One of the major consistent points is that pandemic pricing is not a thing for us. We have maintained our hourly pricing since 2017 at US$250/hr. Our fixed pricing is built based upon this hourly rate as explained in the 2017 article, and we have found that it works well for both clients and our business. Also, we continue to follow the same approach of being the subcontractor instead of being a prime contractor, as it allows us to scale up and down as needed, and it makes interacting with government clients a lot easier. However, we have found that having a sector-specific sales team (such as defense, product, government, and fintech) has helped us establish long-term pricing for our services.

Image by Clker-Free-Vector-Images from Pixabay

In terms of pricing, we’ve found that the unknown, or what we call requirements risk, continues to be a major factor in determining whether we lean towards hourly or fixed pricing. When the requirements are not well-defined or are subject to change, we lean more toward hourly pricing. On the other hand, where requirements are very tight, or what we call an executable specification, we lean more towards fixed pricing. A big risk in AI/ML projects is access to data, and that has been the source of many of the scope changes and delays we have seen on projects big and small.

We’ve continued to find that providing expert advice for project definition and solution architecture has become an increasingly important aspect of our consulting services. Many clients come to us with poorly defined requirements, and it is critical that we identify any provably impossible-to-achieve requirements or requirements that require new science at the earliest possible stage of discussion.

Also, as we did back in 2017, we continue to prefer to delegate as much lower-end work as possible to the client’s staff to lower the hours required to complete the project.

How we have been hiring in the AI/ML consultancy space

Obviously pricing an AI project is tightly related to compensation. During the pandemic, there were several changes to the AI/ML consulting field, mostly related to staffing. First, there was a rush by large companies to hire new and inexperienced talent at irrational salaries. This drained the talent pool and created some perverse incentives for new graduates in fields like statistics and biology to change paths and move into data science and ML. It also caused turnover as staff jumped around between firms. In the last year, things changed again. As the economy has cooled and layoffs have arrived, I have interviewed a number of people who were let go from large organizations and wanted to move into consulting. Speaking objectively, people leaving six-figure jobs were less qualified than their new-grad peers who expected five-figure salaries. Unfortunately, these supposedly more experienced candidates were less willing to put in the effort required to demonstrate their competency. I was surprised to find that the salary expectation of applicants was highly correlated to the number of years of experience, but the test scores dropped off significantly for the applicants that had about 3 to 5 years of experience. My theory is that after working in very large teams on narrow tasks for a few years, many experienced applicants were just not used to doing the crazy consulting job we do every day. My approach has been to ignore the swings in the market and focus on how much value an individual brings to the team. I try and map this as tightly as possible to the billable work that the candidate will bring to the role. Pricing is therefore tied to compensation by making sure that compensation and profit margins allow us to maintain our pricing structure.

Let me tell you a bit about our hiring process. We do things a little differently around here. We ask each candidate for their salary expectation, and then when the technical interviews have been scored, we graph salary expectation versus the scores on the skills testing. We select who to hire based on what that graph tells us is the best value for skills. We request compensation expectations from pre-screened applicants (technical, geographic, and interpersonal scoring), and then try to answer any questions they may have about the role or the company that was not found on the job posting. My goal is to get the responsibilities and compensation discussion out of the way before the technical interview so that the technical interview can focus entirely on technical merit. Everyone’s time is valuable to me, and so I have created a situation where, at each step before the technical interview, people can apply with very little investment of their time, and then progress to the next step or decline.

What’s New?

Well, first of all, we still exist! That’s a big one. In a field full of startups that either exit or vanish, we stayed independent. We slowly built up our business and our reputation using the basic plan described in my 2017 Towards Data Science article. Our CEO Matt incorporated the business on June 22, 2016, and we are still doing our thing 7 years later.

Sales channels have changed over time as well. We have always provided detailed RFQs, but one change worth mentioning is that we now apply to some RFIs and RFPs. We used to hate RFPs so much that we simply didn’t apply to them. Now, over time, we have planted our flag on vendor lists and when something is juicy enough for us to think we have a good chance to win, we throw our hat in the ring.

Credit: The author, and the staff that got us on these lists through sheer force of will to stay up late and fill in forms.

In terms of the size of projects, we now aim for relationships that are expected to lead to projects that are at least 100K (400 hours) in size. Tiny clients just don’t fit so well anymore. However, if we see a project that has an initial scope of 25K but a follow-on opportunity of 150K, then we sink our teeth in and build up the relationship. Notice the change in our focus from project size to client size. A pipeline of small deals from one client adds up over time, and so we have shifted our concept to represent clients rather than specific deals. If we are working with a partner on various small deals, we can sum up the value of the projects they are bringing to the table so that these smaller deals start to make sense from a “big picture” perspective. As far as how we model our business, we use a tool called floatapp.com that directly feeds our P&L data into our revenue forecasts. We also use HubSpot as a CRM to track deals and opportunities.

For project management, we use a self-hosted GitLab instance for most ticket tracking, but we end up using various tools because we use what our clients want us to use for their projects (everything from JIRA to GitHub). We now charge for project management explicitly, which is a good thing to

Another change is that we now have an expanded set of products that we offer to clients. Our services are organized as SKUs and include AI Strategy, Gap Analysis and AI Roadmap, where the deliverable is a report or advisory services, Solution Architecture, where the deliverable is a requirements document or a scoping document, Virtual CTO, where the service includes deeper interaction with the executive team and investors, and our original technical offerings which are Data Science, AI/ML/DL Model Training, and Deployment and MLOps. The change here is that we get involved earlier in the process (e.g., development of a roadmap) and we stick around for longer (e.g., MLOps and support with SLAs).

Juicy juicy on-prem hardware. Credit: Lemay.ai and the electrician who was super understanding about dropping extra power lines into the server room.

We have also pivoted to using more on-prem hardware than ever before. Our hardware is now 4U rack-based and includes a Titan RTX (24 GB, 4608 CUDA cores, Turing Architecture, CC 7.5), 5 RTX 3090 (24 GB, 10496 CUDA cores, Ampere Architecture, CC 8.6) w/ NVLINK, 5 GeForce GTX 1050 Ti, and a whole lot of Xeon/i9 CPUs, and yes, a stupid amount of RAM. We save a lot of money by running our R&D activities on-prem when it makes sense to do so. As far as disk, we have been using unraid for level-2 backup as well as offsite level-3 backup. FYI you can get unraid installed as the base OS when you order a 45drives storinator server. Another change is that we have been seeing some of our clients who shifted to cloud pivot back to on-premise GPU as well, stored at colocation facilities. We also upgraded to a 10-gigabit ethernet switch and network cards. It’s zoomy when moving around big datasets.

Regarding pricing out hardware, we made a decision a long time ago that we don’t mark up hardware costs. That’s not our business model. Instead, we let the customer know exactly the costs of the hardware and warranty, and we only bill for our time if that’s needed. Customers respect our pricing transparency, and we have been getting really positive feedback from this strategy. The customer pays the colocation fees and we only physically head over there when we need to.

Image by Gerd Altmann from Pixabay

Another change is that in recent years, there has been a significant rise in MLOps (machine learning operations) requirements. The use of Kafka, Terraform, and other tools is sharply on the rise in the field of AI consulting.

MLOps involves automating the process of building, testing, deploying, and monitoring ML models using Infrastructure As Code (IAC). MLOps tools such as Kubernetes and Docker were already popular in the AI consulting industry for their ability to easily manage and scale ML workloads. But now tools like mlflow, DVC, Airflow, and Ansible are also becoming standard components in our statements of work. We also see a lot of CI/CD workflows with container registries in our requirements. Components like Kafka and Terraform are gaining traction simply because speed (Kafka is fast — runs using event-driven real-time data streams — so basically it has high throughput and low latency, given sufficient compute) and scaling (Terraform provisions and manages stuff using code instead of buttons on web pages) are on the minds of clients when they approach us to build ML pipelines. Terraform is a popular tool (along with bicep and others) for its ability to create, manage, and scale infrastructure for ML workloads. Gone are the days when a jupyter notebook and Keras were enough to complete a project.

Collaboration Is A Sales Channel

We’ve also learned that there are many ML teams out there, so we collaborate with them when it makes sense. This allows us to offer a wider range of services to clients and helps us specialize in ML and nothing else. Additionally, we’ve found that there are times when off-the-shelf solutions make more sense, and we have started getting paid to be tool integrators in those situations. We have found that especially for teams located in Canada and the US, we are able to collaborate effectively. I mention this specifically because the aerospace and defense sector makes it difficult to work with third parties outside of those 2 countries. It seems like the best collaborations so far are actually with frontend teams that run into ML requirements they can’t address on their own. We do ML and not frontend, and they do frontend and not ML, so it’s a good fit. The other pattern has been digital transformation teams that sell IT services but don’t have deep expertise in AI/ML. We pop into the relationship with their client as an upsell to their existing relationship. We do our invoicing separately but in a time-synced way so that the partner is in lockstep with us in their relationship with their end client.

What Does It All Mean?

Well, despite the AI consulting industry undergoing some huge growth, our approach to pricing and the factors that we consider have remained pretty consistent. We now offer more stuff on an end-to-end basis, and we continue to provide transparent, honest pricing and aim to strike a balance between meeting our own goals and meeting the needs of our clients. Looking down the road this year, I’m expecting pricing to be stable, and I still think there will be a lot of work offloaded from enterprise clients to consulting teams. With the insane pace of progress in the field (yes, I’m going to say ChatGPT) there is a strong awareness among executives that AI/ML has to be on their agenda in order to stay competitive. The takeaway for me is that we had a small startup in 2017, located in my garage, and now we have a less small startup in a nicer office, with bigger toys and bigger projects. We price with a focus on adding value, and we don’t mark up things like hardware purchases or frontend development because we don’t add value there.

Pricing an AI project is complicated, but the formula works, so we are sticking with it.

If you liked this article, then have a look at some of my most-read past articles, like “Large Language Model Morality” And hey, join the newsletter!

Until next time!

Daniel Shapiro, PhD
CTO, Lemay.ai
linkedin.com/in/dcshapiro
daniel@lemay.ai

--

--

TDS Archive
TDS Archive

Published in TDS Archive

An archive of data science, data analytics, data engineering, machine learning, and artificial intelligence writing from the former Towards Data Science Medium publication.

Daniel Shapiro, PhD
Daniel Shapiro, PhD

Written by Daniel Shapiro, PhD

Passionate About Machine Learning R&D and Value Creation. ✍ daniel@lemay.aihttps://lemay.ai

Responses (1)