The Importance of Feedback for Employees and AI
As a CEO of an AI company, I constantly find myself at the intersection of technology and human resources. One cannot operate without the other. Being on the cutting edge of progressing and evolving the framework of Artificial Intelligence, I often come back to human intelligence to compare and correlate. One such correlation I would like to explore is the question of feedback, how to approach it with employees, and how to use it to try and explain the complex process through which AI feedback works in an approachable and simple way.
Feedback as a crucial element in manager/employee relationship
When we think of personal growth, feedback is always described as an essential component of the process. Feedback is the basic mechanism that enables us to learn, and to understand how others view us and our work. It is fairly straightforward to comprehend, yet not trivial to master. When I imagine a good manager, I think of someone who can tell me what he thinks in a candor way, without crushing my ego, without hammering away at my mistakes — but still saying it loud and clear.
A common challenge for the technology industry in 2022 is employee attrition and engagement. Feedback is a critical aspect to cherish, empower and adopt organization-wide to combat this, both as a tool and as a value. However, many experienced managers that I have met in the last few months struggle to give direct feedback. It turns out that building a culture of ‘radical candor’ (see book and website by Kim Scott) is not an easy task. It requires setting solid foundations of trust where feedback, even a candor one, is the norm and not just an arbitrary quarterly practice. Some call it ‘continuous feedback’, others focus on being open with the people you work with during any meeting and internal communication. What is clear is that closing the feedback loop is needed because working without proper feedback isn’t working.
After my PhD in AI, as a young manager who wanted to learn more about feedback in general and specifically about feedback in a startup environment, I began to read everything I could on the subject. I’ve come across many fascinating approaches and points of view; however, my take has been focused on asking the right questions:
What is feedback?
How to give feedback?
When to give feedback?
Why give feedback?
These are not easy questions to answer, and I’ve found that the answers differ from one organization to another, from one manager to another and from one employee to another. Yet, the consensus remains — feedback is crucial for us as humans. In a world where personal growth is so significant to employees and professional development is a leading force for employee satisfaction, it is not surprising that feedback is so valuable.
Intelligence Systems are not Different
After spending a significant amount of time exploring the idea of feedback concerning employees, I came back to technology, particularly AI/ML models, who like humans, need feedback to improve. The information given about a model’s performance is essential for learning. Models can leverage such knowledge to achieve a better generalization of a problem.
Let’s look at a simple scenario where a basic model was trained (the task of the initial build of a new model) using 1000 data samples. This is version 1.0 of the model which was then deployed in production and processed data to predict something. For example, a model was trained to predict whether a sales opportunity will close or not (using data from a CRM system). A few weeks later, some of the opportunities managed to convert and some were lost. Expectedly, the model was right in most cases, but was wrong in some. Closing the feedback loop will mean taking this new data, specifically, where the model was wrong and using it to improve the first model, version 1.0. Instead of a 1000 samples dataset, we can now use 1000 plus 100 additional samples to train version 2.0 of the model.
Simple as it may sound, holding such a process for an AI system is far from trivial and requires heavy development. The four questions noted above in the context of feedback to employees are relevant here as well.
What is Feedback?
This is not an easy question to answer. I gave a scenario where feedback is the new labeled data available for the model to use during training. How do we sample it? How should it be annotated? What is enough feedback data? The answers may vary.
How to Give Feedback?
In a good process, the responsibility of feedback is given to a domain expert, who is usually not a technical expert. Therefore, the way the feedback is given needs to be determined in collaboration between the data science team and the business unit. Noisy data is a big challenge when it comes to training model processes, thus, having a clear definition of the annotation is important.
When to Give Feedback?
As a rule of thumb, frequently and regularly. Besides that, a retraining of the model should be initiated when drifting occurs, i.e., data is changing or the distribution is shifting — basic monitoring capabilities can help and alert to the ‘when’ feedback is especially needed.
Why Give Feedback?
A fairly straightforward answer: we give feedback so the model will be relevant and effective over time. The feedback does not only improve the model but keeps it relevant in an ever-changing environment. By laying the foundation of feedback mechanisms, the business can “keep the models on the rails” and tackle the tedious maintenance activities more effectively. Doing so will reduce the total cost (TCO) and increase the ROI from AI throughout the organization.
Becoming Data-Centric for Production Sustainability
Closing the loop between the users and the (AI) model is, in essence, bringing feedback into action. That is, using the critical information from the field to boost the system by focusing on the new data. As a byproduct of this process, something magical happens — your production environment becomes data-centric. How? With a suitable feedback mechanism, you can bring a simple model to production and improve it with better data gained from the production process. This constant feedback and growth process saves investing research time and money into trying to create an ideal and foolproof model. Just like the perfect employee doesn’t exist, neither does the perfect model.
Conclusion
From employees to AI models — feedback is a powerful tool that moves your business forward. On the human level, mastering it and building the proper feedback process will empower your people across the organization. On the digital transformation level, this user experience factor will boost your productionization journey and, therefore, a wide adoption, scale, and ROI.
