People change teams all the time. There are many reasons including changing jobs, internal migration, personal time off, etc. Gone are the days when people stayed with company long time, let alone with one team. Embracing this fact and being prepared enables a team robust to changes. A big part of the preparation includes a solid onboarding plan. Machine Learning (ML) teams are different since they include many different techniques and skill dimensions compared to typical software products. Onboarding in such teams, therefore, brings in some unique challenges. With this article, we show an onboarding process as part of handling team changes in the ML product area. We will share our view on the offboarding process in a different article.
ML product teams are unique in the sense that it typically deals with a wide variety of dimensions:
- data-driven business process
- (agile) ways of working that are open to exploration
- data and data platforms
- public/private cloud
- ML techniques
- backend/frontend software systems
Supporting these dimensions requires a range of skilled people, such as product owner, business analyst, data analyst, data scientist, ML engineer, data engineer, software engineer, DevOps engineer, etc. People in these roles in such product teams are opting for role/team changes at a rate faster than other roles in another type of software product team due to heavy shortage in the market, fast-changing technical landscape, opportunities to blend multiple roles in one, etc. In our personal observation, we have seen that happening at least once every three to six months. Anticipating a similar rate, we have seen that the best strategy is to focus on shared knowledge and ownership combined with a solid onboarding/offboarding process. Here, we will focus on the onboarding process in such a setting.
A Process for Onboarding
The goal of a solid onboarding process is to make a new recruit an effective collaborator in the team. An additional goal is to make this process fast, without compromising the quality.
We see that onboarding as a four-stage process (see Figure 2). The stages are as follows:
- Recruitment: Starts when the decision has been made that someone will start with the team at a given date. In this stage, a plan is made on how to onboard the new hire.
- Orientation: Starts when someone joins the team. It typically persists a few weeks, during which that person gets all sorts of high-level information and training to understand how to contribute to the product.
- Collaboration: Starts within a couple of weeks. It typically persists a couple of months, during which, the new hire gets different challenges to apply the orientation information to contribute in different dimensions. These challenges are meant to be solved in collaboration with other members of the team.
- Contribution: Starts after a couple of months, since the new hire. At this stage, it is expected that the person knows how to contribute to the product independently and can choose to collaborate with others from time to time.
For onboarding, we recommend the following practices.
- Identify contribution area: The Product Owner (PO) should initiate the onboarding process by informing the team about the new hire. The team should assess the new hire’s profile and pinpoint three to five core areas of contribution relevant to the product. For the case of a data scientist, it could be model training, data analysis, model run analysis, operation support, and documentation.
- Identify/select challenges: The team should find a set of relevant challenges of different levels of complexity that can be assigned to the new hire during onboarding. The challenges can be as simple as writing a unit test for a data wrangling function, moving a hardcoded parameter to a variable, etc. It can be more complex and should take several days to solve, such as investigating the impact of decreasing sample size of training features, implementing a new plot that helps to investigate bias, etc. It can be much more complex and should take several sprints, such as implementing a new feature selection method, automating model selection, optimizing data wrangling, etc.
- Assign onboarding buddy: The team should pick an onboarding buddy, who would list, schedule, and follow up orientation sessions for the new hire. The buddy is not expected to do everything, but should know most of the details of the onboarding and can support the new hire with the bulk of the (development) environment setup. Typically the selection should follow a fair policy unless special circumstances demand a specific person. For example, if the new hire is a DevOps engineer, it is beneficial to go with some type of engineer, even though it was the time for a person with a different role.
- Ensure onboarding check-ins: The product owner should schedule and conduct regular check-ins with the new hire to stay up to date on the progress. The first check-in should be about understanding the expectations both for the new hire and the team against each other. The subsequent ones should be about assessing the expectation fulfillment.
- Setup work environment: The new hire should set up the environment to work with the product using a self-guided process. It could include access to code, data, and model version control systems, setting up licenses for different systems, access to local and remote working environments, data platforms, etc. The person should reach out to the buddy if issues occur. The whole process should not take more than a couple of days.
- Participate in orientation sessions: The new hire should participate in the orientation sessions, which can spread over multiple days over multiple weeks so that the person has plenty of time to grasp the details. Orientation sessions should be on a high level. During this time, the buddy becomes the go-to person for clarifying questions and issues. The buddy can escalate the discussion with someone else should the need arise.
- Engage in learning: The new hire should start reading the documentation on the design, ways of working, technical design, etc., of the product. The person should also start the (self-guided) training on technical components of the product.
- Practice ways of working: The new hire should participate in all the scheduled meetings that the team participates in. The person should also be invited to the ad-hoc discussions. In all these meetings and discussions, the person is welcomed, but not expected to contribute. Furthermore, the person should get a set of very simple challenges that can be used to practice contributing to the product. Writing a unit test, dealing with a hardcoded parameter, improving documentation, etc., are good examples of such challenges. The last interview should be about collecting feedback to improve the onboarding process.
- Carry out the first ticket: The new hire gets assigned to a ticket, which is expected to finish within a sprint. The candidate should get a co-developer in the ticket who shares the responsibility of closing the ticket. The two should engage in pair programming sessions on a regular basis. The co-developer should take a more active role during the sessions. This practice sets the tone of the ways of working in the team — being collaborative. Furthermore, it is preferred that the delivery time of the ticket is a bit relaxed.
- Continue training: The first few sprints of the new hire should not be too occupied. A good amount of time should be left free for the person to continue the training, catch up on documentation, and engage in discussion with the team.
- Carry out more complex ticket(s): After closing the first ticket, the new hire should get a set of tickets, which are collectively a bit more complicated and would consume most of a sprint. Investigating the impact of sample sizes on model training, implementing new straightforward ways to qualify a model, etc., are good examples of such tickets. The person should get one co-developer for each ticket. However, in this case, the person is the main responsible for closing the tickets and is expected to take the lead role during the pair programming sessions. However, it is preferred that the delivery time of the tickets is a bit relaxed to support learning while contributing.
- Plan an epic: The team should plan an epic that the new hire will work on in the last stage of onboarding. While the initial planning can be done without the involvement of the new hire, but the refinement of the work must be done together with the new hire. The work should be much more complex compared to that of the earlier challenges. The epic may be about implementing a new feature selection method, automating model selection process, and optimizing data wrangling, etc.
- Carry out the first major epic: The new hire should be assigned to an epic encompassing two-three sprints. The person should plan and execute the individual tickets within the epic. The person should actively pair the program with other developers to ensure alignment of thinking. However, it is preferred that the delivery time of the epic is a bit relaxed.
- Contribute to the product backlog: While working on the epic, the new hire should start contributing to the product backlog by adding and refining a few tickets.
- Collect feedback on improving the onboarding process: As the last part of the onboarding, the new hire should be interviewed by the product owner on the overall onboarding experience, which becomes the basis for improving the process for the next person.
Best practices that influence onboarding
Here are a few suggestions that not only support a good onboarding process but also improve the way of working of the team in other ways. In our opinion, the following practices will make onboarding effective and faster.
- The team identifies core contribution dimensions to the product. The dimensions should not only touch upon technical aspects, e.g., data wrangling, model management, hypothesis testing, etc. but also on ways of working, e.g., scrum/kanban activity, documentation, etc.
- Some tickets in the backlog should be tagged for training. The idea is that those tickets can be used during the onboarding process. They can also be used in upskilling a team member. The tickets should also be tagged with the contribution dimensions. The tickets should typically have low to mid-level difficulty and relaxed deadlines.
- Templates should be developed to standardize routine work, e.g., onboarding plan, syncs during onboarding, onboarding feedback, etc.
- The team should adopt a solid strategy on documentation and archiving. It is not necessary to record everything in great detail. Good documentation must be easy to update as part of most tickets picked up during the sprints. It can include a few diagrams on business process, infrastructure design, code design, data lineage, etc., API documentation, how-to’s, code documentation, exploration summaries, etc. Good documentation ensures faster onboarding without too much handholding from the team.
- A special section in the documentation should include a guide on setting up the development and operational environment. Some technologies may be new for the candidate. Therefore, for all the technical components, e.g., Python, Docker, sklearn, etc., (publically) available tutorials must be included. Good setup documentation and tutorials ensure faster and easier onboarding.
- The presentation material for the orientation session should be prepared in such a way that it can be deemed as tutorials. For example, a data introduction can be designed as a self-guided notebook with examples and exercises, model management introduction can be designed as a programming challenge, etc. The sessions should be recorded and reused if the presentation materials remain mostly unchanged. In the case of recorded sessions, a short Q&A session should be arranged for further clarification after each session. The tutorial-style onboarding material ensures an engaging onboarding experience. Recording and archiving sessions allow running onboarding faster since it takes away scheduling conflicts.
- The onboarding process should be owned by the whole team, not a few selected ones. The overall process should be standardized and well-understood by everyone. While orientation sessions should be distributed across different team members, most should be able to discuss common high-level points. Training on the onboarding process should be organized for new team members so that they can own the process in the future. Sharing ownership on onboarding makes the team resilient to team change.
- A team should adopt a highly collaborative work style, where pair programming, brainstorming, engaged discussions, etc., is adopted heavily. A high degree of collaboration makes the onboarding process more personalized and efficient.
- A team should strive for a high level of automation for routine work, where very little change occurs in every iteration. A high degree of automation reduces the amount of knowledge that needs to explain to the team member.
- A team should be hypothesis-driven in all ideas. In ML work, surprises are quite common. A good way to deal with surprises is treating every unknown step as an exploration that tests an idea. Findings from the exploration indicates should inspire the next-step implementation or exploration. This ensures giving tasks to the new hire significantly easier due to reduced uncertainties.
- The team should strive for a process where the share of active learning versus contributing changes in a pragmatic way. Figure 4 illustrates such a change. Typically active learning, which includes reading product documentation, going through the technical training, etc., should be very high in the first few weeks. It should never be very low, even when the onboarding process is finished to stay up to date with the state of the art. On the other hand, contributing, which includes, technical work, documentation, planning, discussion, etc., should start from the very beginning. It should start at a low share in the beginning. However, it should reach a high percentage in the last part of the onboarding process and remain at that level afterward. Of course, there can be exceptions, e. g., new technology needs to be learned to support an exploratory, innovative work.
- Different product teams should opt for standardizing technical components and reuse solutions for common problems. This is time-consuming in the short run but pays dividends in the future. It not only allows smooth mobility across team members but also makes onboarding significantly easier.
The longer it takes to onboard a person, the clearer it is that something is going wrong. It could be that the team is going through some issues that make it difficult to focus on the onboarding or the expected and the actual skills of the person do not match. A much worse case would be that the team does not pay attention to the improvement of the onboarding process or the team inherited a high technical debt that involves a huge amount of time getting used to.
This article presents an approach for onboarding an ML team. However, some of the principles apply to software engineering as well. I am sure this approach may not work for you as is. Please provide feedback on points you beg to differ.