Machine learning (Part 12)
Matrix-Vector Multiplication
Sections
- Matrix-vector multiplication
- Matrix-Vector Multiplication in Python
Section 1- Matrix-vector multiplication (Dr Andrew)
In this tutorial, I’d like to start talking about how to multiply together two matrices. We’ll start with a special case of that, of matrix vector multiplication — multiplying a matrix together with a vector. Let’s start with an example.
Here is a matrix, and here is a vector, and let’s say we want to multiply together this matrix with this vector, what’s the result? Let me just work through this example and then we can step back and look at just what the steps were. It turns out the result of this multiplication process is going to be, itself, a vector. And I’m just going work with this first and later we’ll come back and see just what I did here. To get the first element of this vector I am going to take these two numbers and multiply them with the first row of the matrix and add up the corresponding numbers. Take one multiplied by one, and take three and multiply it by five, and that’s what, that’s one plus fifteen so that gives me sixteen. I’m going to write sixteen here. then for the second row, second element, I am going to take the second row and multiply it by this vector, so I have four times one, plus zero times five, which is equal to four, so you’ll have four there. And finally for the last one I have two one times one five, so two by one, plus one by 5, which is equal to a 7, and so I get a 7 over there. It turns out that the results of multiplying that’s a 3x2 matrix by a 2x1 matrix is also just a two-dimensional vector. The result of this is going to be a 3x1 matrix, so that’s why three by one 3x1 matrix, in other words a 3x1 matrix is just a three dimensional vector. So I realize that I did that pretty quickly, and you’re probably not sure that you can repeat this process yourself,
but let’s look in more detail at what just happened and what this process of multiplying a matrix by a vector looks like. Here’s the details of how to multiply a matrix by a vector. Let’s say I have a matrix A and want to multiply it by a vector x. The result is going to be some vector y. So the matrix A is a m by n dimensional matrix, so m rows and n columns and we are going to multiply that by a n by 1 matrix, in other words an n dimensional vector. It turns out this “n” here has to match this “n” here. In other words, the number of columns in this matrix, so it’s the number of n columns. The number of columns here has to match the number of rows here. It has to match the dimension of this vector. And the result of this product is going to be an n-dimensional vector y. Rows here. “M” is going to be equal to the number of rows in this matrix “A”. So how do you actually compute this vector “Y”? Well it turns out to compute this vector “Y”, the process is to get “Y””I”, multiply “A’s” “I’th” row with the elements of the vector “X” and add them up. So here’s what I mean. In order to get the first element of “Y”, that first number — whatever that turns out to be — we’re gonna take the first row of the matrix “A” and multiply them one at a time with the elements of this vector “X”. So I take this first number multiply it by this first number. Then take the second number multiply it by this second number. Take this third number whatever that is, multiply it the third number and so on until you get to the end. And I’m gonna add up the results of these products and the result of paying that out is going to give us this first element of “Y”. Then when we want to get the second element of “Y”, let’s say this element. The way we do that is we take the second row of A and we repeat the whole thing. So we take the second row of A, and multiply it elements-wise, so the elements of X and add up the results of the products and that would give me the second element of Y. And you keep going to get and we going to take the third row of A, multiply element Ys with the vector x, sum up the results and then I get the third element and so on, until I get down to the last row like so, okay? So that’s the procedure.
Let’s do one more example. Here’s the example: So let’s look at the dimensions. Here, this is a three by four dimensional matrix. This is a four-dimensional vector, or a 4 x 1 matrix, and so the result of this, the result of this product is going to be a three-dimensional vector. Write, you know, the vector, with room for three elements. Let’s do the, let’s carry out the products. So for the first element, I’m going to take these four numbers and multiply them with the vector X. So I have 1x1, plus 2x3, plus 1x2, plus 5x1, which is equal to — that’s 1+6, plus 2+6, which gives me 14. And then for the second element, I’m going to take this row now and multiply it with this vector (0x1)+3. All right, so 0x1+ 3x3 plus 0x2 plus 4x1, which is equal to, let’s see that’s 9+4, which is 13. And finally, for the last element, I’m going to take this last row, so I have minus one times one. You have minus two, or really there’s a plus next to a two I guess. Times three plus zero times two plus zero times one, and so that’s going to be minus one minus six, which is going to make this seven, and so that’s vector seven. Okay? So my final answer is this vector fourteen, just to write to that without the colors, fourteen, thirteen, negative seven. And as promised, the result here is a three by one matrix. So that’s how you multiply a matrix and a vector. I know that a lot just happened on this slide, so if you’re not quite sure where all these numbers went, you know, feel free to pause the tutorial you know, and so take a slow careful look at this big calculation that we just did and try to make sure that you understand the steps of what just happened to get us these numbers,fourteen, thirteen and eleven.
Finally, let me show you a neat trick. Let’s say we have a set of four houses so 4 houses with 4 sizes like these. And let’s say I have a hypotheses for predicting what is the price of a house, and let’s say I want to compute, you know, H of X for each of my 4 houses here. It turns out there’s neat way of posing this, applying this hypothesis to all of my houses at the same time. It turns out there’s a neat way to pose this as a Matrix Vector multiplication. So, here’s how I’m going to do it. I am going to construct a matrix as follows. My matrix is going to be 1111 times, and I’m going to write down the sizes of my four houses here and I’m going to construct a vector as well, And my vector is going to this vector of two elements, that’s minus 40 and 0.25. That’s these two co-efficients; data 0 and data 1. And what I am going to do is to take matrix and that vector and multiply them together, that times is that multiplication symbol. So what do I get? Well this is a four by two matrix. This is a two by one matrix. So the outcome is going to be a four by one vector, all right. So, let me, so this is going to be a 4 by 1 matrix is the outcome or really a four diminsonal vector, so let me write it as one of my four elements in my four real numbers here. Now it turns out and so this first element of this result, the way I am going to get that is, I am going to take this and multiply it by the vector. And so this is going to be -40 x 1 + 4.25 x 2104. By the way, on the earlier slides I was writing 1 x -40 and 2104 x 0.25, but the order doesn’t matter, right? -40 x 1 is the same as 1 x -40. And this first element, of course, is “H” applied to 2104. So it’s really the predicted price of my first house. Well, how about the second element? Hope you can see where I am going to get the second element. Right? I’m gonna take this and multiply it by my vector. And so that’s gonna be -40 x 1 + 0.25 x 1416. And so this is going be “H” of 1416. Right? And so on for the third and the fourth elements of this 4 x 1 vector. And just there, right? This thing here that I just drew the green box around, that’s a real number, OK? That’s a single real number, and this thing here that I drew the magenta box around — the purple, magenta color box around — that’s a real number, right? And so this thing on the right — this thing on the right overall, this is a 4 by 1 dimensional matrix, was a 4 dimensional vector. And, the neat thing about this is that when you’re actually implementing this in software — so when you have four houses and when you want to use your hypothesis to predict the prices, predict the price “Y” of all of these four houses. What this means is that, you know, you can write this in one line of code. When we talk about octave and program languages later, you can actually, you’ll actually write this in one line of code. You write prediction equals my, you know, data matrix times parameters, right? Where data matrix is this thing here, and parameters is this thing here, and this times is a matrix vector multiplication. And if you just do this then this variable prediction — sorry for my bad handwriting — then just implement this one line of code assuming you have an appropriate library to do matrix vector multiplication. If you just do this, then prediction becomes this 4 by 1 dimensional vector, on the right, that just gives you all the predicted prices. And your alternative to doing this as a matrix vector multiplication would be to write eomething like , you know, for I equals 1 to 4, right? And you have say a thousand houses it would be for I equals 1 to a thousand or whatever. And then you have to write a prediction, you know, if I equals. and then do a bunch more work over there and it turns out that When you have a large number of houses, if you’re trying to predict the prices of not just four but maybe of a thousand houses then it turns out that when you implement this in the computer, implementing it like this, in any of the various languages. This is not only true for Octave, but for Supra Server Java or Python, other high-level, other languages as well. It turns out, that, by writing code in this style on the left, it allows you to not only simplify the code, because, now, you’re just writing one line of code rather than the form of a bunch of things inside. But, for subtle reasons, that we will see later, it turns out to be much more computationally efficient to make predictions on all of the prices of all of your houses doing it the way on the left than the way on the right than if you were to write your own formula. I’ll say more about this later when we talk about vectorization, but, so, by posing a prediction this way, you get not only a simpler piece of code, but a more efficient one. So, that’s it for matrix vector multiplication and we’ll make good use of these sorts of operations as we develop the living regression in other models further. But, in the next video we’re going to take this and generalize this to the case of matrix matrix multiplication.
Section 2: Matrix-Vector Multiplication in Python
Matrix-vector multiplication is a fundamental operation in linear algebra, and it is also widely used in many scientific and engineering applications. In Python, matrix-vector multiplication can be performed using the numpy library.
There are two main ways to perform matrix-vector multiplication in Python using numpy:
import numpy as np
matrix = np.array([[1, 2], [3, 4]])
vector = np.array([5, 6])
# Multiply the matrix and the vector using the dot() method
result = matrix.dot(vector)
# Print the result
print(result)
Output:
[17 39]Please Follow coursesteach to see latest updates on this story
If you want to learn more about these topics: Python, Machine Learning Data Science, Statistic For Machine learning, Linear Algebra for Machine learning Computer Vision and Research
Then Login and Enroll in Coursesteach to get fantastic content in the data field.
Stay tuned for our upcoming articles where we will explore specific topics related to Machine Learning in more detail!
Remember, learning is a continuous process. So keep learning and keep creating and Sharing with others!💻✌️
Note:if you are a Machine Learning export and have some good suggestions to improve this blog to share, you write comments and contribute.
if you need more update about Machine Learning and want to contribute then following and enroll in following
👉Course: Machine Learning (ML)
References
1- Machine Learning — Andrew
