Multi-task learning is a machine learning method in which a model learns to solve multiple tasks simultaneously. The assumption is that by learning to complete multiple correlated tasks with the same model, that the performance of each task will be higher than if we trained individual models on each task.
However, this assumption does not always hold true. Naïve multi-task learning approaches do not consider the relationships between tasks and trade-offs involved in learning to complete all of the tasks.
Google’s multi-gate mixture-of-experts model (MMoE) attempts to improve upon the baseline multi-task learning methods by explicitly learning relationships between tasks.
In most machine learning contexts, we are concerned with solving a single task at a time. Regardless of what that task is, the problem is typically framed as using data to solve a single task or optimize a single metric at a time. However, this approach will eventually hit a performance ceiling, oftentimes due to the size of the data-set or the ability of the model to learn meaningful representations from it.
Multi-task learning, on the other hand, is a machine learning approach in which we try to learn multiple tasks simultaneously, optimizing multiple loss functions at once. Rather than…
No matter how experienced you are, everyone eventually has to ask someone else a question at work.
However, not all questions are made equal.
The thing that separates people is how they ask their questions.
By following the tips below, you can ensure that you are being respectful of others’ time and maximizing your chance of getting the answer you need.
Before asking for help, try to find a solution yourself. …
It’s that time of the week again — your recurring hour-long Zoom meeting.
As you join, you hear people already in the middle of a conversation. Was it just small talk? Was it work-related? You’ll probably never know.
The meeting officially starts. The person running the meeting unsuccessfully tries to share their screen for the first five minutes, but does eventually manage it, and begins going over the agenda items for the meeting.
Halfway through the meeting, someone finally brings up a topic relevant to your work. You lean in, determined to get something out of this meeting. As the…
When you want to give several variables the same values, you will often see code that repeats the assignment statement for each variable.
It may look something like the following code, which uses one separate assignment per variable:
a = 1
b = 1
c = 1
In Python, we can simplify this, and assign to every variable at once.
a = b = c = 1
# a == 1
# b == 1
# c == 1
After doing this, every variable was assigned to the right-most value in the chain. …
There are several ways to remove an element from a list in Python.
Let’s look at the four main ones: the
remove methods, and the
In the following examples, I’ll use a common list to demonstrate each method. For simplicity, let’s assume that the list used is
l = [2, 3, 4, 5].
clear method removes all elements from the list. This is useful when you no longer need any of the list’s elements and just want to re-use the same list object for new data.
After we call
clear , the list is empty…
If you’ve ever programmed before, you have almost definitely used some kind of numeric data type, such as an integer.
But have you ever thought about how your computer actually stores these numbers?
Instead of using powers of 10 as we normally would to represent numbers, computers store values using powers of 2.
This means that numbers are represented using only strings of zeros and ones — a binary system. Each digit in that binary string is called a bit, and each group of eight bits is one byte.
Typically, we read these binary strings from right to left in…
In languages that allow
null values in variables, developers must explicitly null-check to prevent runtime errors in their programs.
While null-checking is important, it can become a very tedious process, and oftentimes will hurt readability. It often requires heavily-nested code, and gets in the way of business logic.
Typical null-checking code may look like the following snippet:
In this simple example, we see that a basic method to process the
name field from a
Person object turns into a convoluted, nested block of code that requires several stages of null-checking.
The business logic of the snippet, which is just…
You’re working on a project and it uses Git for version control.
You’ve just finished making a change, and you want to quickly update your branch.
So, you open up your terminal, and with a few quick commands, you update your remote branch with your changes.
git add .
git commit -m "added new feature"
But then you do a bit of testing and find that you have a bug in your implementation.
No worries — you quickly find a fix and make another commit to fix the problem.
git add . …
To understand the relationship between math and software engineering, it is first important to understand what the average software engineer actually does.
Most engineers end up working on web or business applications, either on the front end or back end (or both).