Bringing AI to Your Team as an Engineering Leader
As Director of Software Engineering for an incredible team, working to deliver best in class solutions for a well-loved global brand, my job is to bring forward my own years of engineering and management experience to help guide my team through a complex digital landscape. It’s incredibly rewarding, especially the times where I recognize some situation I probably stumbled through at some point in my career and can now successfully steer my team through. In those moments I feel so lucky for my career and all of the amazing opportunities I’ve had to be a part of so many different situations that all add up to where I am today. If you’re a senior leader for your engineering team, I’m sure you feel the same!
Which is why perhaps you and I have both been feeling some creeping existential dread over the last few months as generative AI has roared its way into seemingly every conversation, with its promise of completely upending all those years of experience, potentially changing everything you’ve ever known about leading a software engineering team.
Having spent the last few months (like so many people) immersing myself in AI so that I can better understand what may be on the horizon and how to embrace it, I’ve gathered a few insights and recommendations that I want to share with anyone else who may be in a similar situation but unsure on how to move forward for themselves and their team. I am still very much on the journey along with my team, but I am feeling more optimistic than ever about the role that generative AI will play. I hope after reading the below recommendations, you are too.
Recommendation 1: Avoid Analysis Paralysis
You’ve almost certainly played with ChatGPT, Bard or any of the other LLM’s that have emerged, but may be having a difficult time seeing how/where to bridge from a chat tool that can give you great NYE champagne cocktail ideas into something that your organization can use to drive business value. As a result, you may be waiting for that first great business use case to come along and illuminate the path.
Don’t do it!
Generative AI is a vast new collection of technologies, terms, ways of working, and more. If you wait for that first project to come to your team, you’ll leave them flat-footed and scrambling to learn while they build — which as you know rarely results in success.
My recommendation is to immerse yourself and your team in these new concepts soon as possible. Start simple by having everyone follow IBM’s AI Academy series, and then discuss what they’re learning. Encourage them to take LangChain courses and learn what vector databases are. Start conversations around how to form better prompts or where RAG fits in. Getting your managers and their teams engaged at this level to start will help establish a common understanding, and a foundation on which you can continue to collectively grow.
Recommendation 2: Create Time And Space For Your Team
It’s a near certainty that more than one person on your team has already taken it upon themselves to learn as much as they can about generative AI in their own time. That’s excellent! However, if you want your team to be competent and versed in a technology that will almost certainly impact every person on your team in some way, you must create opportunities for them to learn during business hours.
For my team, we introduced the opportunity by reserving Fridays for a period as a time when anyone (we opened it to not just engineering, which worked out amazingly) could ‘check in’ on Slack and share what they were going to set out to do with generative AI. Whenever they were done (by end of day) they would ‘check out’ and share what they had covered/learned.
The level of collaboration and growth was amazing to watch, as small groups came together to work through various engineering challenges and share resources or ideas. The amount of collective progress in a short time was outstanding, and we’ve now moved on to more focused exercises like competing (for prizes) to see who can write the best prompts for use in code generation (which enables us to standardize and form templates), as well as scheduling hackathons.
Recommendation 3: Pierce The Veil Across your Organization
“It’s coaxing, not coding” is something I find myself repeating quite often, as my organization works to bring AI-powered features to our customers. For years, software engineers have formed their identity around the idea that once they are in the code, they can make anything happen, even if they have to use occasional unwieldy ‘if…then…else’ statements to get it done.
With generative AI, that same level of control becomes practically impossible. When you are dealing with a generative tool that leverages probabilities to produce its output, success can mean less hitting the bulls-eye, and more just making sure it even hits the target. This can be a lot for an organization to wrap its head around, so it’s important that you take the time to help your partners and stakeholders become familiar with the constraints of this new technology- along with its opportunities.
It’s also a tremendous way for you to help your peers bridge the divide that they may be feeling, and showing them how these tools, when properly prompted, can become a force-multiplier for their own operations and processes. There are a lot of great resources and books out there that can help many different aspects and teams within an organization learn to make the most of generative AI.
Recommendation 4: Earn Your Tools, But Be Prepared to Accelerate
Nearly every vendor I’ve talked to over the last few months has unveiled ‘their’ AI feature that has been integrated, refactored, or just plain bolted on to their product. A lot of them scream ‘pay no attention to that ChatGPT behind the curtain!’ but there are also some that are very well thought out and make very compelling cases for why they should be a part of your organization (a tip of the cap to you Vercel v0). Point is- be prepared for AI’s to be everywhere.
I’ve written about it before, but I’m generally very cautious about the idea of bringing in a high-powered tool for a team or organization that hopes it will solve all its problems. Tools like Github CoPilot are going to be game-changers, but only if engineering teams take the time to understand what their needs are, and how to best utilize such a powerful tool to solve those needs for them.
Suffice to say generative AI is going to be an LLM horse-race for a while, and who knows which will be in the lead this time next year (or a few weeks after that). It can be hard to know where to invest your team’s time, and how to pull the ‘right’ tools into your tech stack. That is one reason I appreciate AWS Bedrock. As an LLM marketplace, my team can focus on a set of services that work across multiple models, while allowing us to easily tie in to the right model for the right task without major disruption.
Regardless of which tools you choose, If you proceed deliberately and with a plan in place, it’s highly likely that you’ll see positive adoption, followed by an explosion in productivity!
Recommendation 5: All Your Data Will Someday be an AI’s Data
It’s highly probable that in the near future your organization will have several different AI tools deployed, both internally and externally, and that each one will have a particular domain or area of focus that it excels in. It’s also entirely possible that those AI’s will occasionally coordinate to execute cross-functional tasks that would otherwise take a team of humans weeks to do. It’s gonna be wild. But until then, it will be important that you consider every data conversation you’re having right now through the lens of how it may ultimately be consumed by whichever AI is responsible for it. All of your team’s architectural decisions should consider how to best use the data passing through it for use in vector databases, or fine-tuning jobs, or accessible via RAG. By anticipating it now, you’ll have an easier time bringing on those tools later, which will allow your team to bring value to the organization that much faster.
All that said, you have to be cautious about feeding an LLM your data without fully understanding what is happening with that data. These LLM’s are made inherently better the more data they consume, but you obviously don’t want that information becoming part of a response to other consumers of that LLM. Take care to understand your agreements with LLM’s and AI tools before you begin feeding it all that delicious data, because once it’s in there you may not be able to fully retrieve it.
— — — -
Hopefully these recommendations resonate with you, or validate where you currently are in your journey. This is certainly going to be an interesting transition for everyone. The feeling of ‘this will eliminate my job’ will probably permeate every aspect of how we grow with these tools for a while- it’s only natural. However you’ve hopefully now seen that while the executed code may shift from developer to AI in small ways, there is a lot of expertise required to optimize that code as an output through effective prompting and a considerable amount of work involved in forming and optimizing AI systems on the whole. There is still much work for you and your team to do before there is any practical time horizon on which the role of a software engineer (and its leadership) begins to meaningfully contract, and speaking plainly — it will likely start with those who choose not to adopt these tools.
For my own part, I see that the role engineering has traditionally played as being in the distance between an idea’s inception and its realization, but that gap narrowing significantly (from weeks/months to hours/days) as a result of generative AI. That feels a bit scary for all the implications I just mentioned, but I think it will actually be a great thing, since it’s those ideas that have been the scaffolding on which everything digital has grown. We’re on to bigger and better things, and you have an opportunity to lead your team there- I wish you the best of luck!
