Programming Linear Algebra in Java: Vector Operations
A look at Java through the lens of a simple Vector class
It’s really easy for me to take linear algebra for granted when doing data science. Whether I’m overlooking the specifics of eigendecomposition in Principal Component Analysis, completely forgetting that every weighted sum is actually a dot product between a vector of inputs and a vector of coefficients, or sticking my fingers in my ears to avoid hearing the “tensor” in TensorFlow, I feel like I’ve been shortchanging my degree in math by not getting into the weeds and looking at the implementations.
Although there’s nothing in the definition of a vector space that requires vectors to be represented as lists of numbers, we’ll be focusing on vectors that do take this form. We’re also going to focus only on vector operations involving vectors and scalars––no matrices yet. They’re a can of worms that will not fit in this post.
Before we get going, just a few notes. First, to keep the code as accessible as possible (I’m assuming many people reading this are coming from Python), I’ve called static methods using the notation Vector.method(). This is not strictly necessary, but if you are new to Java, it might help you distinguish between static methods (called from the .class file) and instance methods (called from an object that exists in memory). Second, when talking about vectors as mathematical objects, I will represent them as a bold lowercase letter, e.g. b. Third, when talking about Vector objects in our program, I’ll put the name of the vector in monospace font (so it looks like code).
Finally, I’ll be removing documentation from code examples in this post, in order to keep things readable. If you would like to see the documentation, or you’re interested in skipping straight to the code and avoiding my dazzling commentary on how Java differs from Python, feel free to head over to my github to access the code directly.
Getting started: What data do we need to store?
As discussed above, the vectors we’ll be working with are simply arrays of real numbers, so a skeleton of our Vector class can be seen below:
public class Vector {
private double[] v; // methods not shown
}
Although they have some similarities in terms of syntax for accessing indexed elements, arrays in Java and lists in Python behave in very different ways. Arrays are fixed-length––to append to delete elements, you’d need to use something like an ArrayList . This means our Vector objects are locked into whatever length they’re initialized with, unless we replace the array with a new one of a different length.
Another way this code differs from Python is that keyword private before the variable type. There are ways to hide data in Python, like using the @property annotation on a method with the name of the variable we want to hide), but hiding your instance variables is a core feature that makes Java, Java. To avoid code unintentionally changing the array of numbers scored in the bowels of our Vector object, we make it private , and this means v can only be accessed in ways that we deem to be acceptable.
Now that we have a skeleton to store data, we’ll need to have some way to create a Vector object. The following constructor will do the trick:
public Vector(double ... v) {
this.v = new double[v.length];
for (int i = 0; i < v.length; i++) {
this.v[i] = v[i];
}
}I’m taking advantage of Java’s Varargs feature, which allows us to pass either an array of doubles to the constructor, or simply list them in the call to the constructor:
double[] myArray = {1, 2, 3};
Vector u = new Vector(myArray);
Vector w = new Vector(1, 2, 3);Varargs are a neat feature of Java that I wish I had known about earlier! They don’t work exactly like *args in Python, but if you’re familiar with that notation, you’re close enough for our purposes.
In order to facilitate outputs statements with our Vectorobjects, we’ll write atoString() method, which will allow us to represent our Vector as a String in whatever format we want.
@Override
public String toString() {
String str = "[";
String sep = ",\n ";
for (int i = 0; i < this.v.length; i++) {
str += this.v[i];
if (i < (this.v.length - 1)) {
str += sep;
}
}
return str + "]";
}Notice the @Override annotation before the method signature. In Java, every class extends the Object class, meaning that when you create a new class, such as Vector, you inherit a handful of useful methods from a base class. One of these is the toString() method. By default, this method returns a String representation of the object’s address in memory, but by overriding it, we can customize the way our objects are displayed––for example, when passed to an output statement. If you’re used to object-oriented programming in Python, this is similar giving your class a __str__ method.
I wanted to represent my vector as a column of numbers, so I’ve written this method to iterate over the list, adding the entry, a comma, a line break, and a space (if it is not the final object in the list) to a String that initially contained only an opening square bracket. This results in a tidy representation like the one below:
[1.234,
-7.654,
2.222]Maybe not the best representation for all situations, but it works well for what we’re doing here.
In order to get to the juicy bits, here’s a brief description of a few of the other methods that aren’t specifically linear algebra-related:
get(int position)returns the element at positionpositionlength()returnsv.lengthgetV()makes a copy ofvand returns it (so the original cannot be directly modified)setV(double[] v)replaces the contents ofthis.vwith the argumentvset(int index, double value)setsthis.v[index] = value
Now onto the fun stuff!
Two helper methods we need to discuss before we can do anything too linear algebra-y are isZero() , which returns true if all entries are 0 and false if there are any non-zero entries in the vector. An isZero method is important because some vector operations––like normalizing––will result in division by zero if we don’t check this first. Some operations are perfectly valid with the zero vector, so this check is only done under specific circumstances.
Second, we have checkLengths(Vector u, Vector v). All of the important vector operations––for instance, addition or dot products––are defined only for vectors u and v that have the same number of elements. checklengths compares the two vectors. If they have the same length, nothing happens. If they have different lengths, an IllegalArgumentException is thrown:
public static void checkLengths(Vector u1, Vector u2) {
if (u1.length() != u2.length()) {
throw new IllegalArgumentException(
"Vectors are different lengths");
}
}Basic Operations
Vectors are famous having two primary operations: scalar multiplication (multiplying each entry by the same scalar) and vector addition (adding the corresponding entries of two vectors), so our Vector class would be remiss not to support these. For each, I’ve created a static method that can be called directly from the class, and then an instance method that can be called directly on the Vector itself. Note that all of these operations create a new Vector object, rather than modifying the existing one.
public Vector add(Vector u) {
return Vector.sum(this, u);
}public static Vector sum(Vector u1, Vector u2) {
Vector.checkLengths(u1, u2); // ** see comment
double[] sums = new double[u1.length()];
for (int i = 0; i < sums.length; i++) {
sums[i] = u1.get(i) + u2.get(i);
}
return new Vector(sums);
}
The static method sum does all of the work, and the instance method add simply passes this , which refers to the calling object, and u , a passed vector, to sum. Pythonistas will recognize this as the Java cousin of self. There are some important differences between the two, like how instance methods take self as an argument on Python but simply require us to drop static in Java, but if you understand one, it’s easy to wrap your brain around the other.
I draw attention to this in order to show the contrast between calling a method on an object itself (this.method()) and calling the Vector class’ static method with Vector.sum().
Also notice that in Vector.sum , the very first thing we must do is check to see if the vectors are the same length by calling Vector.checkLengths(u1, u2). Because we’re adding the first element of u1 onto the first element of u2 , then their second elements, then their third elements, and so on, it’s important that they actually have the same number of elements.
There’s actually a bit of redundancy built into this code––because checkLengths is inherently a method of the Vector class, we could have simply written the line marked with // ** see comment as
checkLengths(u1, u2);
To make it clear that we’re not calling any method that specifically requires an instance’s data, I’ve written it explicitly as Vector.checkLengths(u1, u2); Because this method simply performs a check and throws an IllegalArgumentException if the condition is not met, we don’t have any return value, and we can think of it almost as an assert statement.
We can take a similar approach with scalar multiplication:
public Vector multiply(double scalar) {
return Vector.product(this, scalar);
}public static Vector product(Vector u, double scalar) {
double[] products = new double[u.length()];
for (int i = 0; i < products.length; i++) {
products[i] = scalar * u.get(i);
}
return new Vector(products);
}
As well as dot products:
public double dot(Vector u) {
return Vector.dotProduct(this, u);
}
public static double dotProduct(Vector u1, Vector u2) {
Vector.checkLengths(u1, u2);
double sum = 0;
for (int i = 0; i < u1.length(); i++) {
sum += (u1.get(i) * u2.get(i));
}
return sum;
}We can apply this same logic to cross products, but we need to make sure the cross product is actually defined first––that means verifying that both vectors are actually of length 3:
public Vector cross(Vector u) {
return Vector.crossProduct(this, u);
}
public static Vector crossProduct(Vector a, Vector b) {
// check to make sure both vectors are the right length
if (a.length() != 3) {
throw new IllegalArgumentException("Invalid vector length (first vector)");
}
if (a.length() != 3) {
throw new IllegalArgumentException("Invalid vector length (second vector)");
}
Vector.checkLengths(a, b); // just in case
double[] entries = new double[] {
a.v[1] * b.v[2] - a.v[2] * b.v[1],
a.v[2] * b.v[0] - a.v[0] * b.v[2],
a.v[0] * b.v[1] - a.v[1] * b.v[0]};
return new Vector(entries);
}I’ve accessed the elements of the arrays with slightly different syntaxes in these operations, both to make the cross product more readable and to highlight a feature of Java that can be confusing the first time you come across it. Notice how in dotProduct, I called the instance method u1.get(i) in order to access elements of the array, while in crossProduct, we accessed the elements directly with a.v[0]. In Java, any code in the Vector class has access to the private members of any Vector object, but we can also call those members’ instance methods.
Direction and Magnitude
Often, we want to know the magnitude (length) of a vector, which is really a special case of the p-norm where p=2. Since L1 and L2 norms come up in machine learning contexts (for instance, Lasso and Ridge regression), let’s go ahead and make a generalized function for computing the p-norm, given a vector and a value of p:
// static method
public static double pnorm(Vector u, double p) {
if (p < 1) {
throw new IllegalArgumentException("p must be >= 1");
}
double sum = 0;
for (int i = 0; i < u.length(); i++) {
sum += Math.pow(Math.abs(u.get(i)), p);
}
return Math.pow(sum, 1/p);
}// instance method
public double pnorm(double p) {
return Vector.pnorm(this, p);
}// magnitude
public double magnitude() {
return Vector.pnorm(this, 2);
}
Having both a static method and an instance method for pnorm gives us a little bit of flexibility in how we compute the norm of a vector. We can call the method either from the Vector class itself (e.g. Vector.pnorm(u, 2) ), or we can call it directly on an existing Vector object (e.g. u.pnorm(2) ).
Once we’ve defined the pnorm method, we can wrap it up in themagnitude method, to give us the magnitude of a vector. And now that we can compute that, we can normalize a vector. Normalizing is accomplished by dividing the entries of the vector by the vector’s magnitude, which is why we need the isZero check before we proceed:
public static Vector normalize(Vector v) {
if (v.isZero()) {
throw new IllegalArgumentException();
} else {
return v.multiply(1.0/v.magnitude());
}
}
public Vector normalize() {
return Vector.normalize(this);
}Notice the difference in how we’ve handled a zero Vector and Vector objects of different lengths––any time we attempt an operation on Vectors with different lengths, we’re entering undefined territory, and we need to throw an IllegalArgumentException to abort, but there are some operations (such as pnorm) that are perfectly valid for all-zero entries, so we wouldn’t want to build the exception into isZero().
More Complex Operations: Enclosed Angles and Scalar Triple Products
Several operations rely on dot products, cross products, and magnitude calculations, and we can now perform using the methods we’ve built.
First, we want to be able to compute the angle enclosed by two Vectors, which requires the dotProduct method:
public static double angleRadians(Vector u1, Vector u2) {
Vector.checkLengths(u1, u2);
return Math.acos(Vector.dotProduct(u1, u2) /
(u1.magnitude() * u2.magnitude()));
}And next, we want to be able to compute the , scalar triple product (which requires both the dotProduct and the crossProduct methods:
public static double scalarTripleProduct(Vector a,
Vector b,
Vector c) {
return Vector.dotProduct(a, Vector.crossProduct(b, c));
}If you’ve ever had to do this by hand (for instance, on a linear algebra exam), you can probably understand how satisfying it was to watch these pieces fall into place with so little effort.
And there we have it! The purpose of this post was not really to break new ground, but to both explore how linear algebra topics build on each other, and highlight a few features of Java that I find interesting. Feel free to head over to the source code if you’re interested in playing around with this your own Vector objects.
