Tech Apprenticeship Series #3: Program Structure & Timeline

6 Month Apprenticeship — 2 Months of Group Project Work and 4 Months of Team Placement

TLDR; Twilio’s Hatch Program is broken into sections and uses a ‘tiered step learning model’ to ensure the apprentice built skills before moving to the next level. The focus was not only on apprentices, but also enabling and supporting managers and mentors.

This is article #3 of the “Tech Apprenticeship Series”

Table of Contents

· Introduction
· Basic Program Timeline
· Tiered Step Learning Model
· Progression of Projects
· Twilio Specific Structure
· Key Success Factors
· Apprentice Support Team
· Program Timeline
· Evaluations and Conversion Decision
· Apprentice Support
· Engineering Manager Support
· Mentor Support
· Condensed Metrics
· In Conclusion

Introduction

Twilio’s Hatch Software Engineering Apprenticeship Program was a 6 month apprenticeship broken into incremental sections. The goal was to convert apprentices into successful, long term Twilio engineers.

The structure of the program was intentionally designed to meet the apprentices where they were in their learning journey. We wanted them to feel supported but also to stretch and grow both technically and professionally.

This article gives an overview of the program structure, and the “why” behind its design. To prevent the article from becoming a novel, I tried to keep it high level (I may have failed…). I do hope to double-click into specific parts of the program later. If you are interested in more information/specifics, please send me a message or ask in the comments.

We took an intentional, data driven, user centered approach to our program design. We considered the audience we were working with, our goals, the company needs, and our capacity to support the program. I mention this because it is important for any company interested in starting a program to consider these areas in order to be successful. And feedback is a must!

A large part of the program focused on supporting and enabling the team managers and mentors, as they spent the most time with the apprentices. The manager and mentor were the most impactful people for the experience (and success) of an apprentice. It is important to support and enable them to ensure a consistent and high quality program experience.

Basic Program Timeline

1. Pre-Apprenticeship
   ⇒ Yearly: Engineering Team Selection
   ⇒ -2 Months: Sourcing and Interviewing
   ⇒ -1 Month: Apprentice Offers Made
2. Week 1: Initial Onboarding
3. Week 2: Language Deep Dive Training
4. Weeks 3–9: Group Onboarding Project
5. Weeks 10–26: Team Placement
   ⇒ Week 21: Conversion Decision
   ⇒ Week 22: Offers Made
6. Week 26: End of Program

Tiered Step Learning Model

With Hatch, we utilized a “tiered step learning model”, where we had progressive projects to ensure apprentices learned skills gradually. We wanted to avoid making the next step so high that progress stalled.

An apprenticeship program is a form of onboarding an employee. When support is not provided to a new employee, the steps required to being successful can be very large. The pressure will be on them to figure everything out, and they may fail. If they do, unfortunately that new employee is often blamed, and not the team who should have enabled that person to succeed.

When you plant lettuce, if it does not grow well, you don’t blame the lettuce. You look for reasons it is not doing well. It may need fertilizer, or more water, or less sun. You never blame the lettuce.
— Thich Nhat Hanh

We took the approach where we did everything we could to create an environment to succeed. It still required the apprentice to work hard, be motivated, learn, take feedback well, and build their skills. In the end, it was them who achieved the success, we just tried to remove unnecessary barriers.

At each level, the apprentice’s progress gave them momentum and prepared them for the next level.

Progression of Projects

1. Week 1 Fun Twilio App Project — A fun project to introduce Twilio as a company and the products, and to give apprentices momentum quickly.
2. Language Deep Dive — A guided, accelerated project to introduce apprentice to their team’s language. They work independently to be able to learn at their own pace.
3. Group Onboarding Project — A professional team environment where apprentices work together using Twilio’s engineering processes and tools. This gives them the foundational skills to be successful working in their team placement.
4. Team Placement — Real work focusing on the teams deliverables. Tightly scoped to the apprentice’s level, the work is done with the support of a mentor. This gives the apprentice specific and focused work to ramp up with the teams processes, systems and technology.
5. Full Time L1 Engineer — After converting to an L1, the apprentices continues to grow outside of the program with the support of their manager and the engineering organization.

Twilio Specific Structure

This structure worked at Twilio, and it was an evolution. We felt the results validated that we were on the right track.

• 95% of apprentices converted to full time engineers
• ~90% of converted engineers were still with the company after 2 years
• 70% of the apprentices identified as women or non-binary
• 94% of Apprentices were satisfied with their experience as an apprentice at Twilio, and would recommend the program to a peer
• 92% of all managers would recommend the program to a peer
• For more stats, please refer to Tech Apprenticeship Series #2: Metrics

What will work at another company will probably be different than Twilio. However, others can learn from our experiences.

Key Success Factors

Two areas vital to the success of the apprentices were the cohort model and the group onboarding project. Apprentices often gave feedback on the importance these played in their development and preparation to join their teams.

1. The cohort model, where the apprentices started together and could build community and rely on each other.
2. The group onboarding project, where apprentices worked on a professional engineering team in an environment conducive to their comfort, learning and growth.

If you are interested to discuss your challenges/needs to start a program, I would be happy to discuss more and can be found on LinkedIn or via email.

Apprentice Support Team

The Apprentice had a team of people supporting in many areas

There was a diverse team of people working together to support the apprentice. This team focused on empowering the apprentice, providing autonomy and space to learn, while also providing guidance and support.

• Early in Career (EIC) Team Lead
  ⇒ The EIC Team Lead was the expert in sourcing, recruiting, hiring and making offers. They acted as the apprentice’s support for anything HR related, and organized programmatic events such as socials and Q&A’s with executives. The EIC team also led the internship and returnship at Twilio, so they had a wealth of experience in supporting EIC talent.
• Hatch Team Lead
  ⇒ The Hatch Team Lead (also referred to as the “Hatch Lead”) was an EM that co-lead the cohorts with the EIC Team Lead. This person needed to have a strong technical foundation, and be skilled at communication, teaching, project scoping, and leading engineering teams. The Hatch Lead led the “Group Onboarding Project”, facilitated and led trainings, was available for technical support, and met 1:1 with the apprentice weekly. We believed that the Hatch Lead’s technical experience also built credibility for their work with the mentors and managers, where they providing support and training when necessary.
• Host Team Engineering Manager
  ⇒ The Host Team Engineering Manager was responsible for setting the apprentices up for success on the team. This person scoped the apprentice’s work, and partnered with the mentor to ensure the experience on the team was inclusive and supportive. They also met weekly with the apprentice, completed evaluations, and provided regular feedback. They were responsible for the conversion decision.
• Host Team Mentor
  ⇒ Probably the most important role, the Host Team Mentor was an L2+ software engineer* on the team the apprentice joined. The mentor helped the apprentice understand the team’s products, systems, processes and technologies. The apprentice often worked with the mentor on feature work. The mentor checked in daily with the apprentices, paired with them, provided code reviews, and helped the manager identify valuable work.
• Hatch Alumni Buddy
  ⇒ A special role was that of the alumni buddy, a former apprentice to support the apprentice. By having gone through the program, they were able to relate and empathize with the apprentice, and provide support in ways that the others could not. The buddy was often there for emotional support, to answer questions the apprentice did not feel comfortable asking others, and to help them find resources at Twilio.

* The mentor was not necessarily the most senior engineer. Morst important was their motivation to mentor and communication skills.

Program Timeline

-2 Months — Start: Pre-Cohort

• Team Applications, Interviews & Selection (Yearly Nov — Jan, Managers)
  ⇒ We had a selection process to ensure the teams were stable and had a culture of mentorship. In 2022 we had ~36 teams apply for ~30 roles
• Applications Open & Initial Steps (-2.5 months, Candidates)
• Interview Training (-2 months, Managers & Mentors)
• Virtual Onsite Interviews & Apprentices Selected (-1.5 months, Candidates)
• Offers Made (-5 weeks, Candidates)
• Pre-Cohort Meeting (-2 weeks, Managers & Mentors)
  ⇒ To ensure aligned expectations and to answer questions, we met with managers and mentors before the start. We also gave feedback on planned work for the apprentice.
• Welcome Email & Introductions (-1 week, Apprentices)

Weeks 1 & 2: Initial Onboarding

• Welcome and Introductions (Day 1, Meeting)
• General Twilio Onboarding (Days 1 — 3, Meetings)
• Mentor, Manager and Buddy Coffee Chats (Days 1 & 2, Social)
• Program Meet & Greet (Day 3, Social)
• Intro App Project: Build with Twilio API (Week 1, Learning Project)
• Intro App Project Demo (Day 5, Social & Event)
• Alumni Meet & Greet (Day 8, Social)
• Language Deep Dive (Week 2, Learning Project)

Weeks 3–9: Group Onboarding Project

The apprentices worked together on a group project for ~2 months before joining their teams. These projects were run by Hatch Lead and were set up like a real engineering team engineering environment.

I appreciate that I was able to work on a project with the other apprentices before joining my team. I was able to learn how to work with others without the anxiety of making a mistake in front of my team. Since it was just the other apprentices, I felt more comfortable.
— Apprentice

Project Details

• Timeframe: 7 weeks (Three 2 week sprints, 1 wrap up week)
• Manager: Hatch EM
• Team Size: 3 to 6 Apprentices, working together
• Project Language: Apprentice’s team language (Java/Scala, Python, GoLang or React/Javascript)
• Project Focus: Building a service or frontend the “Twilio Way”. The project focus was developed by the Hatch EM and was a training project, not one that would be used in production.
• Project Docs: “Hatch Blueprint” doc, similar to blueprints that are created by engineering teams at Twilio. Our blueprint combined product requirements and technical requirements into one doc.
• Skills Learned: Understanding a PRD, understanding a Blueprint, sprint planning, time estimation, code reviews, testing, mocking, retrospectives, deployment processes and deploying, debugging, working collaboratively, etc.
• Supplemental Trainings
  ⇒ Agile Best Practices
  ⇒ Introduction to Retros
  ⇒ Effective Code Reviews
  ⇒ Git 101: Branching, Rebasing, Cherry-Picking & a Clean History
  ⇒ Testing 101: Unit Testing
  ⇒ Testing 102: Mocking Fundamentals
  ⇒ PLACEHOLDER: PUSHING BACK TRAINING
  ⇒ Product Management at Twilio
  ⇒ Introduction to Docker & Containers
  ⇒ Deploying at Twilio
• Project Demo (End of Month 2, Social & Event)

Month 2.5–6: Apprentice Team Placement

For the final 4 months of the apprenticeship, each apprentice became a team member, working on real features and tickets that were central to the goals of their team.

Project Details

• Timeframe: 16 weeks
• Manager: Placement Team EM
• Project Language: Apprentice’s team language (Java/Scala, Python, GoLang or React/Javascript)
• Work Focus: Tickets and work central to the goals of the team. The manager assigned work similar to any other engineer on the team, though properly scoped for an apprentice (L0 Engineer Level). We did not allow the apprentice to do a “pet project” or project by themselves (which is often what interns do), as we wanted them to work on things valuable to the team, alongside the other engineers, and have ongoing focused work (and not just one project).
• Skills Learned: While the apprentice continued to learn how to best work in a team environment, they also began to specialize based on the needs of the team. They could expand their initial understanding to do things like more advanced testing (integration testing, load testing, chaos testing), working with production build systems, participate in game days, focusing on a specific technology or part of the stack, etc.
• Conversion Decision (End of Month 5, Managers)
• Full Time Offers Made (Beginning of Month 6)
• Apprenticeship Demo & Celebration (End of Month 6, Social & Event)

Evaluations and Conversion Decision

All apprentices continuously received feedback in both areas of growth and positive reinforcement. To reduce bias in this process, the Hatch Team guided managers to use a standardized rubric, with input from the mentors. The conversion decision was based on the final evaluation, making this decision consistent and straightforward.

3 Evaluations

• Evaluation 1: Week 9, Completed by Hatch Lead
• Evaluation 2: Week 16, Completed by Manager
• Evaluation 3: Week 22, Completed by Manager, Conversion Decision

Apprentice Support

It was important to build community amongst apprentices. We also focused on empowering them to find and use the resources available at Twilio.

It was nice to be able to get together and talk about things outside of work. Joining my team was a big adjustment as I no longer saw the other apprentices daily. I looked forward to our coffee chats every week.
— Apprentice

Regular Support and Resources

• Apprentice Roundtables — Every Month, 6 Total
  ⇒ A discussion amongst the cohort, facilitated by the Hatch Lead. The meeting was a safe space to speak openly, discuss learnings, failures and successes, build camaraderie amongst the apprentices, and to get feedback. This also allowed the program to understand areas where we could support the apprentices better.
• Hatch Socials
  ⇒ The EIC Lead organized fun ways to connect, such as “Two Lies and a Truth” (including personal and Twilio trivia), virtual escape rooms, collaborative games over zoom, etc. It was important to intentionally foster a sense of community as we were all WFH and distributed.
• Apprentice Q&A with Leadership
  ⇒ Small Q&A sessions with different members of Twilio Leadership, to better understand their roles, the overall goals of the company, and to get to know each other. The leaders often changed, but included the CEO, CPO, CTO, CDO, CImpactO, CInformationOfficer, VPs, and GMs.
• Apprentice Onboarding Packet
  ⇒ One stop shop with important information such as an onboarding checklist (apps to install, etc), Hatch calendar (planned meetings, orientation schedule, tech talks/trainings, events), people they can reach out to for support, important slack channels, team information, tips on how to be a successful apprentice, etc.
• Apprentice Only Slack Channel
  ⇒ An apprentice-only slack channel for the apprentices and alumni (the Hatch Lead was not in this channel). This provided a safe space to ask questions, get support, discuss challenging issues, celebrate success, etc — By it being private, a lot was shared openly and discussed, which really helped the apprentices.
• Hatch Wiki
  ⇒ Consistently evolving wiki with helpful tips, where to find resources, information on how to get started with Twilio specific technologies, Apprenticeship Guide (for apprentices, by apprentices), links to helpful Twilio wiki articles.
• LinkedIn Learning
  ⇒ LinkedIn Learning was available to all apprentices, where they could take online based courses in any topic that they wanted.

Ad Hoc Support

The support given changed depending on the needs of the cohort. Some cohorts needed and wanted more instruction, while others needed more time to connect. Below some things we did depending on the cohort.

• Additional Checkins with Hatch Lead — As needed
  ⇒ Some apprentices needed more 1:1 time with the Hatch Lead (more than the weekly or biweekly regular checkins), for example to talk through a challenge they were facing with a team member, or discuss an anxiety they were experiencing, or to go over a technical concept. These were always available to the apprentices.
• Coffee Chat — Weekly
  ⇒ Weekly 30 minute, optional virtual coffee chat with all members of the cohort including the EIC Lead and Hatch Lead. We often reconnected, checked in, and shared about our week and personal lives to build trust and feel a sense of community amongst the group.
• Study Hour — Weekly
  ⇒ The structure of this varied. One of the more successful approaches was if an apprentice learned an interesting new technical topic, process or approach, they would present about it, show examples and answer questions. Apprentices could also request deeper dives into a technical topic, and the Hatch Lead would develop a quick lesson (presented as a conversation with visual aids).
• Additional Technical Mentors — As needed
  ⇒ At times, an apprentice lacked a team member who could help them in a specific technical area. When this happened, the Hatch Lead would find a mentor from the greater Twilio community.
• Apprentice Organized Checkins — Weekly
  ⇒ The program encouraged the apprentices to self organize in any way that they wanted to support each other, with the Hatch Lead not present. A meeting without a manager/member of the program had very different levels of openness and communication. We often saw the apprentices get together each Friday and do “Highs and Lows” for the week, or similar prompts. More details about these, we do not know, as we were not invited :).

Engineering Manager Support

The Hatch Program was fortunate to work with EMs across many different business units, countries, and backgrounds. In order to ensure consistency in the expectations of the apprentices, we invested time to align with the managers of each cohort. We wanted them to have everything they needed to be successful in converting their apprentice.

At times, an EM would ask Hatch for additional technical trainings for their apprentice. The program also often supported the EMs in properly scoping the apprentice’s work.

Manager Support & Resources

• Manager Playbook
  ⇒ A collection of templates (such as an onboarding checklist), guides (Hatch Mentor Selection for Managers, Scoping Apprentice Work, etc), information (such as “Hatch Values”), and recommendations for managers based on previous experiences.
• Interview Pre-Cap & Hatch Interview Prep Kit
  ⇒ The program provided an interview guide, a standardized interview rubric and a list of questions for the managers to select from. We wanted to make it straightforward for the managers, while also reducing bias and ensuring an equitable interview process for all candidates.
• Performance Calibrations — Months 3 and 4.5
  ⇒ We hosted two facilitated group calibrations during the apprenticeship. The goal was to understand the apprentice performance, level set expectations, answer questions, share best practices, and identify apprentices in need of more support.

Mentor Support

The mentor was highly influential in the success of the apprentice, as they worked most closely with the apprentice. We looked for empathetic and knowledgeable engineers with strong communication skills and a desire to mentor. We already knew any engineer at Twilio that had good performance reviews had the technical skills needed. We allowed all engineers L2 and above to be a mentor — we’d rather have a less experienced engineer with strong communication than a more experienced one that struggled to convey technical concepts.

In terms of support, I enjoyed the mentor roundtables the most. It was a great opportunity to hear how others were approaching mentoring and to borrow ideas. I also appreciated the feedback given by the Hatch Manager!
— Mentor

Mentor Resources

• Interview Training
  ⇒ The mentors paired with the candidates during the technical portion of the interview (building a service together). We provided interview training so they felt prepared, had an opportunity to build the service themselves, and had their questions answered. We also reinforced the need to create a welcoming and safe environment for the candidates.
• Toolkit for Being a Hatch Mentor
  ⇒ We provided a toolkit with guidance, resources and mentorship best practices. This included ideas on how to build trust in the beginning of the relationship, what the mentor expectations were, and where they could reach out for support.
• Mentor Roundtables — Every Month, 6 Total
  ⇒ We hosted 6 facilitated discussions with the mentors. We discussed how their apprentice was doing, any successful moments as a mentor, and what challenges they were facing. It provided the opportunity for mentors to support each other, build a community and ask questions. The Hatch Lead also gained insight into how the apprentice was performing. These were popular with the mentors, and we moved from having 3 to 6 based on mentors asking for more.

In Conclusion

Hopefully this article provided insight into the how and why we implemented different parts of the Hatch Software Engineering Apprenticeship.

As mentioned before, our approach evolved, and while we were confident in the program, there were still many things we hoped to improve. In the end, we were able to consistently achieve our goal of converting the apprentices into full time software engineers.

Please comment below if you have any questions!