Programming Linear Algebra in Java: Vector Operations
A look at Java through the lens of a simple Vector class
It’s really easy for me to take linear algebra for granted when doing data science. Whether I’m overlooking the specifics of eigendecomposition in Principal Component Analysis, completely forgetting that every weighted sum is actually a dot product between a vector of inputs and a vector of coefficients, or sticking my fingers in my ears to avoid hearing the “tensor” in TensorFlow, I feel like I’ve been shortchanging my degree in math by not getting into the weeds and looking at the implementations.
Although there’s nothing in the definition of a vector space that requires vectors to be represented as lists of numbers, we’ll be focusing on vectors that do take this form. We’re also going to focus only on vector operations involving vectors and scalars––no matrices yet. They’re a can of worms that will not fit in this post.
Before we get going, just a few notes. First, to keep the code as accessible as possible (I’m assuming many people reading this are coming from Python), I’ve called static methods using the notation Vector.method()
. This is not strictly necessary, but if you are new to Java, it might help you distinguish between static methods (called from the .class
file) and instance methods (called from an object that exists in memory). Second, when talking about vectors as mathematical objects, I will represent them as a bold lowercase letter, e.g. b. Third, when talking about Vector
objects in our program, I’ll put the name of the vector in monospace font
(so it looks like code).
Finally, I’ll be removing documentation from code examples in this post, in order to keep things readable. If you would like to see the documentation, or you’re interested in skipping straight to the code and avoiding my dazzling commentary on how Java differs from Python, feel free to head over to my github to access the code directly.
Getting started: What data do we need to store?
As discussed above, the vectors we’ll be working with are simply arrays of real numbers, so a skeleton of our Vector class can be seen below:
public class Vector {
private double[] v; // methods not shown
}
Although they have some similarities in terms of syntax for accessing indexed elements, arrays in Java and lists in Python behave in very different ways. Arrays are fixed-length––to append to delete elements, you’d need to use something like an ArrayList
. This means our Vector
objects are locked into whatever length they’re initialized with, unless we replace the array with a new one of a different length.
Another way this code differs from Python is that keyword private
before the variable type. There are ways to hide data in Python, like using the @property
annotation on a method with the name of the variable we want to hide), but hiding your instance variables is a core feature that makes Java, Java. To avoid code unintentionally changing the array of numbers scored in the bowels of our Vector
object, we make it private
, and this means v
can only be accessed in ways that we deem to be acceptable.
Now that we have a skeleton to store data, we’ll need to have some way to create a Vector
object. The following constructor will do the trick:
public Vector(double ... v) {
this.v = new double[v.length];
for (int i = 0; i < v.length; i++) {
this.v[i] = v[i];
}
}
I’m taking advantage of Java’s Varargs feature, which allows us to pass either an array of doubles to the constructor, or simply list them in the call to the constructor:
double[] myArray = {1, 2, 3};
Vector u = new Vector(myArray);
Vector w = new Vector(1, 2, 3);
Varargs are a neat feature of Java that I wish I had known about earlier! They don’t work exactly like *args
in Python, but if you’re familiar with that notation, you’re close enough for our purposes.
In order to facilitate outputs statements with our Vector
objects, we’ll write atoString()
method, which will allow us to represent our Vector
as a String
in whatever format we want.
@Override
public String toString() {
String str = "[";
String sep = ",\n ";
for (int i = 0; i < this.v.length; i++) {
str += this.v[i];
if (i < (this.v.length - 1)) {
str += sep;
}
}
return str + "]";
}
Notice the @Override
annotation before the method signature. In Java, every class extends the Object class, meaning that when you create a new class, such as Vector
, you inherit a handful of useful methods from a base class. One of these is the toString()
method. By default, this method returns a String
representation of the object’s address in memory, but by overriding it, we can customize the way our objects are displayed––for example, when passed to an output statement. If you’re used to object-oriented programming in Python, this is similar giving your class a __str__
method.
I wanted to represent my vector as a column of numbers, so I’ve written this method to iterate over the list, adding the entry, a comma, a line break, and a space (if it is not the final object in the list) to a String
that initially contained only an opening square bracket. This results in a tidy representation like the one below:
[1.234,
-7.654,
2.222]
Maybe not the best representation for all situations, but it works well for what we’re doing here.
In order to get to the juicy bits, here’s a brief description of a few of the other methods that aren’t specifically linear algebra-related:
get(int position)
returns the element at positionposition
length()
returnsv.length
getV()
makes a copy ofv
and returns it (so the original cannot be directly modified)setV(double[] v)
replaces the contents ofthis.v
with the argumentv
set(int index, double value)
setsthis.v[index] = value
Now onto the fun stuff!
Two helper methods we need to discuss before we can do anything too linear algebra-y are isZero()
, which returns true
if all entries are 0 and false
if there are any non-zero entries in the vector. An isZero
method is important because some vector operations––like normalizing––will result in division by zero if we don’t check this first. Some operations are perfectly valid with the zero vector, so this check is only done under specific circumstances.
Second, we have checkLengths(Vector u, Vector v)
. All of the important vector operations––for instance, addition or dot products––are defined only for vectors u and v that have the same number of elements. checklengths
compares the two vectors. If they have the same length, nothing happens. If they have different lengths, an IllegalArgumentException
is thrown:
public static void checkLengths(Vector u1, Vector u2) {
if (u1.length() != u2.length()) {
throw new IllegalArgumentException(
"Vectors are different lengths");
}
}
Basic Operations
Vectors are famous having two primary operations: scalar multiplication (multiplying each entry by the same scalar) and vector addition (adding the corresponding entries of two vectors), so our Vector
class would be remiss not to support these. For each, I’ve created a static
method that can be called directly from the class, and then an instance method that can be called directly on the Vector
itself. Note that all of these operations create a new Vector
object, rather than modifying the existing one.
public Vector add(Vector u) {
return Vector.sum(this, u);
}public static Vector sum(Vector u1, Vector u2) {
Vector.checkLengths(u1, u2); // ** see comment
double[] sums = new double[u1.length()];
for (int i = 0; i < sums.length; i++) {
sums[i] = u1.get(i) + u2.get(i);
}
return new Vector(sums);
}
The static
method sum
does all of the work, and the instance method add
simply passes this
, which refers to the calling object, and u
, a passed vector, to sum
. Pythonistas will recognize this
as the Java cousin of self
. There are some important differences between the two, like how instance methods take self
as an argument on Python but simply require us to drop static
in Java, but if you understand one, it’s easy to wrap your brain around the other.
I draw attention to this
in order to show the contrast between calling a method on an object itself (this.method()
) and calling the Vector class’ static method with Vector.sum()
.
Also notice that in Vector.sum
, the very first thing we must do is check to see if the vectors are the same length by calling Vector.checkLengths(u1, u2)
. Because we’re adding the first element of u1
onto the first element of u2
, then their second elements, then their third elements, and so on, it’s important that they actually have the same number of elements.
There’s actually a bit of redundancy built into this code––because checkLengths
is inherently a method of the Vector
class, we could have simply written the line marked with // ** see comment
as
checkLengths(u1, u2);
To make it clear that we’re not calling any method that specifically requires an instance’s data, I’ve written it explicitly as Vector.checkLengths(u1, u2);
Because this method simply performs a check and throws an IllegalArgumentException
if the condition is not met, we don’t have any return value, and we can think of it almost as an assert
statement.
We can take a similar approach with scalar multiplication:
public Vector multiply(double scalar) {
return Vector.product(this, scalar);
}public static Vector product(Vector u, double scalar) {
double[] products = new double[u.length()];
for (int i = 0; i < products.length; i++) {
products[i] = scalar * u.get(i);
}
return new Vector(products);
}
As well as dot products:
public double dot(Vector u) {
return Vector.dotProduct(this, u);
}
public static double dotProduct(Vector u1, Vector u2) {
Vector.checkLengths(u1, u2);
double sum = 0;
for (int i = 0; i < u1.length(); i++) {
sum += (u1.get(i) * u2.get(i));
}
return sum;
}
We can apply this same logic to cross products, but we need to make sure the cross product is actually defined first––that means verifying that both vectors are actually of length 3:
public Vector cross(Vector u) {
return Vector.crossProduct(this, u);
}
public static Vector crossProduct(Vector a, Vector b) {
// check to make sure both vectors are the right length
if (a.length() != 3) {
throw new IllegalArgumentException("Invalid vector length (first vector)");
}
if (a.length() != 3) {
throw new IllegalArgumentException("Invalid vector length (second vector)");
}
Vector.checkLengths(a, b); // just in case
double[] entries = new double[] {
a.v[1] * b.v[2] - a.v[2] * b.v[1],
a.v[2] * b.v[0] - a.v[0] * b.v[2],
a.v[0] * b.v[1] - a.v[1] * b.v[0]};
return new Vector(entries);
}
I’ve accessed the elements of the arrays with slightly different syntaxes in these operations, both to make the cross product more readable and to highlight a feature of Java that can be confusing the first time you come across it. Notice how in dotProduct
, I called the instance method u1.get(i)
in order to access elements of the array, while in crossProduct
, we accessed the elements directly with a.v[0]
. In Java, any code in the Vector
class has access to the private members of any Vector
object, but we can also call those members’ instance methods.
Direction and Magnitude
Often, we want to know the magnitude (length) of a vector, which is really a special case of the p-norm where p=2. Since L1 and L2 norms come up in machine learning contexts (for instance, Lasso and Ridge regression), let’s go ahead and make a generalized function for computing the p-norm, given a vector and a value of p:
// static method
public static double pnorm(Vector u, double p) {
if (p < 1) {
throw new IllegalArgumentException("p must be >= 1");
}
double sum = 0;
for (int i = 0; i < u.length(); i++) {
sum += Math.pow(Math.abs(u.get(i)), p);
}
return Math.pow(sum, 1/p);
}// instance method
public double pnorm(double p) {
return Vector.pnorm(this, p);
}// magnitude
public double magnitude() {
return Vector.pnorm(this, 2);
}
Having both a static method and an instance method for pnorm
gives us a little bit of flexibility in how we compute the norm of a vector. We can call the method either from the Vector
class itself (e.g. Vector.pnorm(u, 2)
), or we can call it directly on an existing Vector
object (e.g. u.pnorm(2)
).
Once we’ve defined the pnorm
method, we can wrap it up in themagnitude
method, to give us the magnitude of a vector. And now that we can compute that, we can normalize a vector. Normalizing is accomplished by dividing the entries of the vector by the vector’s magnitude, which is why we need the isZero
check before we proceed:
public static Vector normalize(Vector v) {
if (v.isZero()) {
throw new IllegalArgumentException();
} else {
return v.multiply(1.0/v.magnitude());
}
}
public Vector normalize() {
return Vector.normalize(this);
}
Notice the difference in how we’ve handled a zero Vector
and Vector
objects of different lengths––any time we attempt an operation on Vector
s with different lengths, we’re entering undefined territory, and we need to throw an IllegalArgumentException
to abort, but there are some operations (such as pnorm
) that are perfectly valid for all-zero entries, so we wouldn’t want to build the exception into isZero()
.
More Complex Operations: Enclosed Angles and Scalar Triple Products
Several operations rely on dot products, cross products, and magnitude calculations, and we can now perform using the methods we’ve built.
First, we want to be able to compute the angle enclosed by two Vector
s, which requires the dotProduct
method:
public static double angleRadians(Vector u1, Vector u2) {
Vector.checkLengths(u1, u2);
return Math.acos(Vector.dotProduct(u1, u2) /
(u1.magnitude() * u2.magnitude()));
}
And next, we want to be able to compute the , scalar triple product (which requires both the dotProduct
and the crossProduct
methods:
public static double scalarTripleProduct(Vector a,
Vector b,
Vector c) {
return Vector.dotProduct(a, Vector.crossProduct(b, c));
}
If you’ve ever had to do this by hand (for instance, on a linear algebra exam), you can probably understand how satisfying it was to watch these pieces fall into place with so little effort.
And there we have it! The purpose of this post was not really to break new ground, but to both explore how linear algebra topics build on each other, and highlight a few features of Java that I find interesting. Feel free to head over to the source code if you’re interested in playing around with this your own Vector
objects.