The MLOps Community’s Engineering Labs
A collaborative place for solving ML Engineering challenges
Premises
Since I joined the MLOps Community, many things have changed.
People like Demetrios Brinkmann, David Aponte and many others work hard together spreading knowledge about MLOps and ModelOps disciplines. They promote several initiatives like Slack channel, Coffee Sessions, Meetups, and Talks. With experts and practitioners who join from 32 different countries. All of that with the aim of creating a place where you can learn about experiences and best practices around ML and Model operations.
So when I start thinking about how I can contribute, I ask myself:
…What another way to support people…?
…For a person who joins the community, how can I turn this knowledge in something can help him/her solving most common ModelOps challenges in a concrete way?
These were the questions I asked myself at the very beginning.
And the unique answer I find to all them is…
Then, here I am so eager to talk about that. Indeed, this is the opening article of a Medium’s series the MLOps community is going to have about the initiative.
With it, I will introduce to the Engineering Labs and its guiding principles.
In particular, I will give you an overall picture about the typical lab journey and I will describe each phases in details. To make the telling point, I will use the 1st Lab as an example of what I’m referring at. Finally I provide contact information in case I success to convince you join us =)
Below you can see the Table of Content of the rest of the article:
- The Initiative and its guiding principles
- The Engineering Lab’s journey
∘ Step 1: Fill the form
∘ Step 2: Join the team
∘ Step 3: Solve the challenge in 6 weeks
∘ Step 4: Merge your project and provide feedback
∘ Step 5: Coffee session
∘ Step 6: Medium Article and the Final Technical Discussion - The 1st Engineering Lab: PyTorch meets MLflow
∘ Content
∘ Teams
∘ The 1st Journey and its Results - How you can join us
- Summary
The Initiative and its guiding principles
Originally we announce the initiative in August. And now, as then, Engineering Labs is a place where you as a person have the opportunity to join a team and work together on solving a particular Machine Learning Engineering challenge.
No matter how tough challenge will be, we firmly believe that collaboration is the key factor to take down it. Why? Because it is in the DNA of the MLOps. If you want to put machine learning systems in production you need to develop solutions through the collaborative effort of self-organising and cross-functional team.
In a team, each person has the chance to learn from other members. Indeed, it is one of our main concern combining heterogeneous skills. Both technical and soft ones. In this way, you improve at the same time you have fun.
Of course, each challenge has many possible ways to be solved. So we decide not to impose strict constraints. Once we clearly define the content of the lab, you are free to experiment and test each combination of solution and technology. But because we want to create value for our end users, your solution needs to be as much replicable as possible. That’s why we create a GitHub repository as source of truth.
About the content of labs, depending on the participation and how the initiative will evolve as the community changes, we will ask you and anyone who wants to propose ML challenge to solve. We will collect them and we will let the community cast its vote on it.
And, in the end, once you and your team solved the challenge, we will ask you to share your experience and what you learn. Because we commonly agree that the true knowledge is the one people share with feelings and not only with words.
Now that you have an idea about the initiative and its guiding principles, let provide you an overall picture about the Engineering Lab’s journey.
The Engineering Lab’s journey
I always thought that a picture is worth a thousand words. So there you go.
As you can see, the Engineering Lab’s journey consists in 6 steps:
Step 1: Fill the form
We provide a subscription form to collect contact info, background and everything is useful to assign you to the right team. Also you can propose a challenge you would like to solve.
Then, we review the info and we announce the lab. In particular, we let you know the lab challenge (with a community vote as optional step), which is your team and all the information you need to success.
Step 2: Join the team
Once you know your team, we have an onboarding call where you present yourself to the entire team. And remember that it is the right time to ask all questions you have and solve all doubts about what’s next.
Step 3: Solve the challenge in 6 weeks
From now on, you have all you need to start solving the challenge. You and your team are free to plan your 6 weeks as you prefer. Just to be sure everything is going well, the entire team will have 30 min stand up meeting to check how lab is going on.
Step 4: Merge your project and provide feedback
At the end of the 6 weeks, you should be ready to merge your project to the official Engineering Labs’ GitHub repo under the MLOps.community organisation. Then, we will ask you to fill a feedback form in order to tell us what you think about the experience.
For the rest of the journey, we you will present what you and your team have done.
Step 5: Coffee session
In the Coffee session, you attend a general discussion about your lab experience as a team. You talk about aspects like project management, the solution you propose, all technical issues with infrastructure and tools you use to solve the challenge. And because it is an informal session, whatever you like to share.
Step 6: Medium Article and the Final Technical Discussion
In this last step, you have the chance to go in depth of your solution describing it in an article series we will publish on the official MLOps Community Medium page. Then, because each team has its own experience, one representative for each team prepare a number of questions based on the article and we set up a final talk discussing about consensus.
Since this moment, the lab is considered as finished. All teams learned from each other, the challenge is fixed with a proper solution (given time limits), and we can move on to new lab.
That’s it. Probably, at that point, you’re wondering questions like
Is it too much? Will I able to go thought it?
Honestly, we were not sure about that as well at the beginning.
But you know what?
Two out of four teams who joined the 1st lab did it.
And actually they did it really well.
To give you an idea, I will use the 1st Engineering Lab as an example of what I’m referring at. But I let each teams present to you their solutions with articles are coming in the series.
The 1st Engineering Lab: PyTorch meets MLflow
This summer, PyTorch community announced a number of technical contributions to enable end-to-end support for MLflow usage with PyTorch promising to accelerate the workflow from research to production.
As MLOps Community, we know from discussions we had that operationalizing Pytorch models could be tough. That’s why we decide to test these new capabilities and provide a possible end-to-end model lifecycle of Pytorch model using Mlflow in the 1st official engineering labs.
Content
As I mentioned before, we decide not to impose constraints. We just asked to use PyTorch and MLflow. We suggest to have a look at the full MLOps workflow for PyTorch models building and serving using MLflow in the original article. But then we let teams to figure it out.
Teams
After collecting the answer to the form, we decide to have 4x4 teams. In particular, community members involved in this scenario were:
- Team 1 : Michel Vasconcelos, Alexey Naiden, Varuna Jayasiri, John Savage
- Team 2 : Itirell, Mark Peters, Carlos Leyson, Semendiak
- Team 3 : Laszlo Sragner, Artem Yushkovsky, Dimi, Paulo Maia
- Team 4 : Shadab Hussain, Susana Brandao, Godwin Ekainu, Josh
The 1st Journey and its Results
We officially announce the lab in December. And teams start solving the challenge during the month of January.
It was not easy. In the first week, two teams gave up. Indeed, we realised later that set only one constraint (MLflow and Pytorch technologies) was not the best way of doing it because we didn’t ask them to solve a specific problem. That’s why 2 groups had a hard time deciding on a problem. And they folded.
But who remains collaborate a lot and worked hard on it. And at the end, they deliver two final projects:
- Team 1, the News Classification project
- Team 3, the Yelp Review project
In the meanwhile, I’d got contact with the AI&PIE community in Milan and together we agreed to have a meetup where teams were presented what they achieved. If you are interesting in how the meetup went, you can find the recording here. Also both teams shared their experience in dedicated Coffee sessions. Feel free to watch them. Both teams had frank talks about their overall experience. Finally, at the time I’m writing they are working on Medium articles and we are going to have the Final Technical Discussion soon.
By the way, doing the math I can count
- 1 lab about how put in production Pytorch model with MLflow
- 4x4 teams
- 2 final projects
- 1 meetup in the AI&PIE community
- 2 insightful coffee sessions
- (coming) 2 medium articles and 1 final tech talk
Not so bad to be your 1st experience, isn’t it?
Of course, this would like to be a quick overview about what awaits you in case you decide to join the initiative. And with that in mind let me close this article telling you how you can reach us.
How you can join us
As lead of the initiative, I would recommend to
- Join the slack channel: https://go.mlops.community/slack
- Join the #engineeringlabs channel
- Candidate yourself for the next lab with this form
- Feel free to reach me out both in the community (@ilnardo92) and on Linkedin for clarifications
Then, STAY TUNE!
Summary
To sum up, this article is the official reference about everything you need to know about the MLOps Community’s Engineering Labs.
I push myself to provide a concise and clear picture about the journey describing each phases in details.
To go straight to the point, I used the 1st Engineering Lab as an example. And I did some math to show you how valuable is this initiative.
Finally, I provided contact information in case I convinced you to join us. Indeed, as Uncle Sam would say
