As an engineer, are you aware of the implications and risks of using generative AI?

A pencil drawing of a software engineer robot working at the computer by DALL-E

We don’t need any explicit references to say that, as of today, using generative AI and large language models (LLMs) is becoming commonplace. Every day, new products are announced (Roth, 2023; Vincent, 2023), new researches are published (Zhu et al., 2023), and AI-enabled software is released (‘Open-Assistant’, 2023), all with the common topic of exploring these artificial intelligence paradigms.

As the world embraces using generative AI, we’re as well asking ourselves the same question: what is our position towards this disruptive technology at Think-it? More generally, what should the position of anyone doing contract engineering work on behalf of another organization be? What are the legal, intellectual, and ethical risks of using AI code assistants to create value for contracted partners?

We attempt in the next sections to share insights and takeaways from ongoing research at Think-it, w.r.t, to generative AI and its usage.

LLMs at Think-it

Currently, the use of LLMs within the Think-it collective is relatively limited to documentation tasks. The Growth team has already been using it to tailor messages to specific users for some time now. Our Ops team has been taking advantage of these generative tools to produce qualitative content, helping them complete their tasks. Our team of Technical Owners got help from machine learning algorithms by expanding and rephrasing content, being more productive overall.

Within the Engineering domain, we have started exploring how suitable LLMs are in supporting personal and professional growth as an omnipresent coach. Meanwhile, our Data engineering, Data science and AI Chapter is diving deeper into the implications of these algorithms in our environment with the goal of bringing together more formal usage policies that can provide new perspectives to Software Quality Assurance (SQA), namely automation tasks, documentation tasks and code reviews. Finally, some of our SDEs got curious as well and asked AI models with reasoning capabilities for quick help on engineering-related questions and tedious transactional tasks.

It’s already very interesting to see our collective members taking proactive initiative in exploring the potential applications of machine learning in our day to day. It’s important to understand ChatGPT and fiends capabilities early on, as various opinions say it will enhance and evolve many industries which are close to us; customer service, project management, and sales will all face the revolution that OpenAI unleashed (Daugherty, Wilson and Narain, 2023; Nieto-Rodriguez and Vargas, 2023; Sinha, Shastri and Lorimer, 2023).

Surprisingly, though, a very limited number of our engineers are using it for what developers worldwide use: AI-powered code assistants. As the result of a quick poll survey, only 3% of our Think-iteers are using code-enhancing tools such as Copilot.

AI code assistants

In the past year or so, AI code assistants have boomed. The GitHub Copilot release to the general public (Dohmke, 2022) has taken the industry by storm. At a similar time, Amazon announced CodeWhisperer (AWS announces Amazon CodeWhisperer (Preview), 2022), and Google announced that Googlers benefit from code assistant software developed in-house (ML-Enhanced Code Completion Improves Developer Productivity, 2022). But luckily, we don’t only have the big three wondering about pair programming with a machine learning algorithm; a long time before big names announced their initiatives, there were already a few visionaries. Tabnine, one of the first AI code assistants on the market, has survived until today (AI Assistant for software developers, no date). Funded in 2013, it predates by far even Think-it’s own Predictive Coding initiative, and now has raised a total of $32.1m (Tabnine — Crunchbase Company Profile & Funding, no date), highlighting the vivid attention and interest to the market.

After all, the benefits are palpable. In a marketing effort, GitHub led research (Ziegler, 2022) on their Copilot tool. The study explored how AI-enabled code completion systems can improve developer productivity by generating helpful code suggestions based on the context of their current activity; the outcome result suggests that the quality of suggestions is more important than their correctness (Ziegler et al., 2022). This result is interesting indeed: developers are more productive doing “small code reviews” than writing all the code on their own.

In a later blog post, Kalliamvakou (2022) went deeper into the initial research, focusing on GitHub’s meaning of developer productivity and the impact of Copilot on users. Overall, survey responders reported less frustration doing their job, the ability to focus on more important–and fun–tasks, and an increase in speed.

Biases aside, the research highlights the benefits of AI code assistants for both developers’ productivity and job satisfaction. And what strengthens GitHub’s conclusion is Accenture’s experience with Amazon CodeWhisperer; Accenture has been using CodeWhisperer to reduce development efforts by 30%, improving developer productivity and accelerating onboarding for novice developers while detecting security threats early in the development process and empowering developers to responsibly use AI to create secure and syntactically correct applications for AI and machine learning projects (Viswanathan et al., 2023).

But nothing is as simple as some results in a chart. At Think-it, we often work on multiple and diverse partners’ projects, where we handle intellectual property for their benefit. We are the medium for partners’ success and we’re good at it. To keep up with the time, it may feel obvious that we could aim even higher by subscribing our SDEs to start using generative AI and running with it. But that’s not as easy as it seems.

A pencil drawing of a robot helping a software engineer working at the computer by DALL-E

Legal, intellectual and ethical issues

Current LLMs are trained in the wild. As days pass, the field’s exploration evolves, and with it, also legal ones. The most striking news came out recently; ‘ChatGPT banned in Italy over privacy concerns’ (McCallum, 2023) is the title of the BBC article, which appeared like an April Fools joke — but it wasn’t. The OpenAI chatbot was allowed to operate in the Italian network (Mukherjee and Vagnoni, 2023), but the event still highlights the sensitivity of the problem. Privacy concerns are real, as it is currently impossible to make the OpenAI chat assistant forget anything it was told. An example of this is Samsung’s latest disaster (Lewis, 2023), in which employees leaked company secrets to the bot, which are now accessible to the world with no recourse.

This is already enough to stop and reflect; what does this mean for contracted engineers working with partners? As it stands, only one Think-it collective engineer is using tools such as Copilot–which uses OpenAI’s Codex model (GitHub Copilot · Your AI pair programmer, no date)–and is working on internal projects. We can all agree though it may be less consequential than working on a partner project, it would be detrimental if the code were leaked to the general public. This reason alone would call for a clear and agreed policy among our engineers and our partners, but there’s more.

Copilot and other generative AI software have a bigger, wider issue: they possibly infringe on copyright, and the user won’t even know (Appel, Neelbauer and Schweidel, 2023). GitHub’s Copilot specifically is a liability, as the company hasn’t yet revealed the material used for the training of the AI model (Gingerich and Kuhn, 2022). Hence, the problem is even bigger; even if a partner would agree to us using tools like Copilot, the code we produce may be classified as plagiarism, and they could be liable in case of an audit. Sure, it’s rare to hear about software code auditing for plagiarism, but given the above concerns, it may become more common in the future.

Even if it would be the partner’s problem to have taken the final decision and allowed us to work with a generative AI tool, our brand would take a hit. Moreover, ethically speaking simply washing our hands wouldn’t deliver the best experience to our partners, and it’s a selfish act going against our values of integrity and collectivism.

How to safely explore the possibilities

From an individual perspective, exploring AI is fairly straightforward. As previously suggested, different service providers offer proprietary solutions; once you agree to third-party terms of use and privacy policy, you are all set. Moreover, the open-source community is quickly growing, providing a vast range of tools, ideas and proof of concepts. But for an organization’s perspective, the exploratory investigation becomes more complicated. Hugging Face (Hugging Face — The AI community building the future., no date) is a great starting place to discover and keep updated with the latest machine learning trends. Unfortunately, models’ licensing is not always transparent, and not all the available ones are accessible for commercial use (Large Language Models for Commercial Use, 2023). If you’re an individual that’s performing work on behalf of an organization, those aspects can be troublesome.

AI is snowballing, and a pressing need for responsible AI (RAI) comes with it. Renieris, Kiron and Mills (2022) express the same feeling but also highlight the implementation’s difficulties; pursuing RAI isn’t a predefined process that should be followed, but it must be an integral part of an organization’s beliefs and behaviors. It includes top-management involvement, training where is necessary, and considering society as a stakeholder. It takes investment but gives the organization a more holistic view of the technology and its broad impact (Renieris, Kiron and Mills, 2022).

While we start to explore AI and its use, it’s therefore imperative that we keep the conversation alive among all stakeholders. Our decisions must be well aligned, and every study we make should help reinforce our understanding of what it means for us to be responsible with using generative AI. We must allow these initiatives to guide us according to our values and follow the impact we want to have on the wider society.

At the same time, we cannot ignore the benefits of using AI code assistants shown earlier (Kalliamvakou, 2022; Ziegler, 2022; Ziegler et al., 2022); but at what cost? How was the model trained? What are the terms of use? Is the developer’s code used to re-train the model? Tabnine (AI Assistant for software developers, no date) and Amazon CodeWhisperer (AI Code Generator — Amazon CodeWhisperer, no date) look to address these concerns better, but understanding their position goes behind the purpose of this research. Furthermore, while answering these questions, we as engineers must ensure that any work done on a partner’s behalf is done with awareness and agreement with using these tools. Failing to do so may put people in the unfortunate position of justifying themselves and could have future bigger legal consequences.

Different communities and groups now compare the emergence of generative AI models to previous historical disruptive inventions, such as the printing press, with the potential to initiate a new industrial revolution (Candelon et al., 2023; Cremer, Bianzino and Falk, 2023; Devlin, 2023). As time goes by and using generative AI only increases, we as engineers need to continue to understand and reflect on what responsible AI really means. At Think-it we’re taking a proactive position towards this new technology, and we’ll continue sharing our insights as we go.

References

• AI Assistant for software developers (no date) Tabnine. Available at: https://www.tabnine.com/ (Accessed: 17 April 2023).
• AI Code Generator — Amazon CodeWhisperer (no date) Amazon Web Services, Inc. Available at: https://aws.amazon.com/codewhisperer/ (Accessed: 17 April 2023).
• Appel, G., Neelbauer, J. and Schweidel, D.A. (2023) ‘Generative AI Has an Intellectual Property Problem’, Harvard Business Review, 7 April. Available at: https://hbr.org/2023/04/generative-ai-has-an-intellectual-property-problem (Accessed: 10 April 2023).
• AWS announces Amazon CodeWhisperer (Preview) (2022) Amazon Web Services, Inc. Available at: https://aws.amazon.com/about-aws/whats-new/2022/06/aws-announces-amazon-codewhisperer-preview/ (Accessed: 17 April 2023).
• Candelon, F. et al. (2023) The CEO’s Guide to the Generative AI Revolution, BCG Global. Available at: https://www.bcg.com/publications/2023/ceo-guide-to-ai-revolution (Accessed: 11 May 2023).
• Cremer, D.D., Bianzino, N.M. and Falk, B. (2023) ‘How Generative AI Could Disrupt Creative Work’, Harvard Business Review, 13 April. Available at: https://hbr.org/2023/04/how-generative-ai-could-disrupt-creative-work (Accessed: 11 May 2023).
• Daugherty, P.R., Wilson, H.J. and Narain, K. (2023) ‘Generative AI Will Enhance — Not Erase — Customer Service Jobs’, Harvard Business Review, 30 March. Available at: https://hbr.org/2023/03/generative-ai-will-enhance-not-erase-customer-service-jobs (Accessed: 10 April 2023).
• Devlin, H. (2023) ‘AI “could be as transformative as Industrial Revolution”’, The Guardian, 3 May. Available at: https://www.theguardian.com/technology/2023/may/03/ai-could-be-as-transformative-as-industrial-revolution-patrick-vallance (Accessed: 11 May 2023).
• Dohmke, T. (2022) ‘GitHub Copilot is generally available to all developers’, The GitHub Blog, 21 June. Available at: https://github.blog/2022-06-21-github-copilot-is-generally-available-to-all-developers/ (Accessed: 17 April 2023).
• ‘FauxPilot’ (no date). fauxpilot. Available at: https://github.com/fauxpilot/fauxpilot (Accessed: 17 April 2023).
• Gingerich, D. and Kuhn, B.M. (2022) Give Up GitHub: The Time Has Come!, Software Freedom Conservancy. Available at: https://sfconservancy.org/blog/2022/jun/30/give-up-github-launch/ (Accessed: 17 April 2023).
• GitHub Copilot · Your AI pair programmer (no date) GitHub. Available at: https://github.com/features/copilot (Accessed: 17 April 2023).
• Hugging Face — The AI community building the future. (no date). Available at: https://huggingface.co/ (Accessed: 17 May 2023).
• Kalliamvakou, E. (2022) ‘Research: quantifying GitHub Copilot’s impact on developer productivity and happiness’, The GitHub Blog, 7 September. Available at: https://github.blog/2022-09-07-research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/ (Accessed: 17 April 2023).
• Large Language Models for Commercial Use (2023) TrueFoundry Blog. Available at: https://blog.truefoundry.com/all-about-license-for-llm-models/ (Accessed: 17 May 2023).
• Lewis, M. (2023) Samsung workers made a major error by using ChatGPT, TechRadar. Available at: https://www.techradar.com/news/samsung-workers-leaked-company-secrets-by-using-chatgpt (Accessed: 17 April 2023).
• McCallum, S. (2023) ‘ChatGPT banned in Italy over privacy concerns’, BBC News, 31 March. Available at: https://www.bbc.com/news/technology-65139406 (Accessed: 8 April 2023).
• ML-Enhanced Code Completion Improves Developer Productivity (2022). Available at: https://ai.googleblog.com/2022/07/ml-enhanced-code-completion-improves.html (Accessed: 17 April 2023).
• Mukherjee, S. and Vagnoni, G. (2023) ‘Italy restores ChatGPT after OpenAI responds to regulator’, Reuters, 28 April. Available at: https://www.reuters.com/technology/chatgpt-is-available-again-users-italy-spokesperson-says-2023-04-28/ (Accessed: 17 May 2023).
• Nieto-Rodriguez, A. and Vargas, R.V. (2023) ‘How AI Will Transform Project Management’, Harvard Business Review, 2 February. Available at: https://hbr.org/2023/02/how-ai-will-transform-project-management (Accessed: 14 April 2023).
• Nijkamp, E. et al. (2023) ‘CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis’. arXiv. Available at: https://doi.org/10.48550/arXiv.2203.13474.
• ‘Open-Assistant’ (2023). LAION AI. Available at: https://github.com/LAION-AI/Open-Assistant (Accessed: 17 April 2023).
• Renieris, E.M., Kiron, D. and Mills, S.D. (2022) ‘To Be a Responsible AI Leader, Focus on Being Responsible’, MIT Sloan Management Review [Preprint]. Available at: https://sloanreview.mit.edu/projects/to-be-a-responsible-ai-leader-focus-on-being-responsible/ (Accessed: 10 April 2023).
• Roth, E. (2023) Elon Musk founds new AI company called X.AI, The Verge. Available at: https://www.theverge.com/2023/4/14/23684005/elon-musk-new-ai-company-x (Accessed: 17 April 2023).
• Sinha, P., Shastri, A. and Lorimer, S.E. (2023) ‘How Generative AI Will Change Sales’, Harvard Business Review, 31 March. Available at: https://hbr.org/2023/03/how-generative-ai-will-change-sales (Accessed: 10 April 2023).
• Tabnine — Crunchbase Company Profile & Funding (no date) Crunchbase. Available at: https://www.crunchbase.com/organization/tabnine (Accessed: 17 April 2023).
• Vincent, J. (2023) Google will reportedly release new AI powered search tools next month, The Verge. Available at: https://www.theverge.com/2023/4/17/23686108/google-ai-search-tools-magi-chatgpt-bing-samsung-deal (Accessed: 17 April 2023).
• Viswanathan, B. et al. (2023) How Accenture is using Amazon CodeWhisperer to improve developer productivity, AWS Machine Learning Blog. Available at: https://aws.amazon.com/blogs/machine-learning/how-accenture-is-using-amazon-codewhisperer-to-improve-developer-productivity/ (Accessed: 17 April 2023).
• Zhu, D. et al. (2023) ‘MiniGPT-4: Enhancing Vision-language Understanding with Advanced Large Language Models’. Vision CAIR Research Group, KAUST. Available at: https://github.com/Vision-CAIR/MiniGPT-4 (Accessed: 17 April 2023).
• Ziegler, A. (2022) ‘Research: How GitHub Copilot helps improve developer productivity’, The GitHub Blog, 14 July. Available at: https://github.blog/2022-07-14-research-how-github-copilot-helps-improve-developer-productivity/ (Accessed: 17 April 2023).
• Ziegler, A. et al. (2022) ‘Productivity Assessment of Neural Code Completion’. arXiv. Available at: https://doi.org/10.48550/arXiv.2205.06537.