Happily Coexisting: Building with exec-maven-plugin on both OS X and cygwin

Jimmy F
Jimmy F
Jul 27, 2017 · 4 min read
Working with Maven & POM files (Source: www.pexels.com)

At work at my job, we build almost everything with Maven. Prior to working here, I’d done a little work with the Java/Maven ecosystem of tools, but in general, I had no foundation of knowledge about it.

Here’s the general gist: All the code we write goes into a repo, that repo has a parent-pom. We have a big Jenkins server which takes your repo with its POM, puts it inside a larger project with an appropriate nickname for this big bundle of stuff, and builds the parent POM.

If you want something to be used anywhere within our company, you need to use this system. As an aside, it’s really nice because it enforces our policy of “config as code”, meaning that any oddities in how to build things must be encased in the POM file. There can be debates about this, but in a company with hundreds of developers: we need something, and this is it.

It was in this environment that I decided to fix a minor issue that had been bothering me one day: automated code which was stored in a Git repo. This is a sort of pet peeve — it’s like keeping an artifact that’s generated automatically in a Git repo. Without a really good reason (excuse?), you should never store things which can be generated with the repo inside that repo.

The way it worked was this: we have these Python files and they automatically generate a number of Scala files which we then use to make some super-type-safe operations even more type safe. When your employer processes 95 billion events a month, they generally don’t appreciate random exceptions that inspire your Hadoop job to blow up.

Regardless of the situation, I was annoyed two-fold:
1. We had to run the Python scripts “periodically” to regenerate new code
2. The new generated code gets checked in.

So from time to time, somebody would submit a huge 200–300 line change in a really obtuse Scala file. Our team would have to approve this and hope there weren’t any issues. I decided to change this — my strategy went like so:

  1. Add the Python stuff to the build process…
  2. Delete the generated code from the repo…
  3. Let the automatic builders generate the code…
  4. Profit.

And I almost did. Here is what (a skeleton) of the addition to the POM was:

The first solution

This seemed like a really great idea, and in fact, for me, on Mac OS X it was wonderful. Another teammate who was using Linux tried it too and he found it worked. Additionally our automatic builders all run Linux and they had no issues.

I had made a mistake: not all my team members use Linux or Mac OS X. In fact, some use something far more nefarious: cygwin. They were running Windows with cygwin inside it to emulate *nix. This isn’t really surprising given how much .NET and C# our company is doing, but having been a fan of developing on a Mac, I didn’t even consider using a Win machine when I started here (no slight against Windows devs, whatever your flavor is that works for you).

The first problem was that python3 on Windows ends in .exe and doesn’t include the number 3. No problem — we set up a profile for this and changed the executable name.

The next problem was more complex. I don’t know a lot about cygwin, Windows, or how paths resolves, but I realized after a little investigation that the maven-exec-plugin was searching not in cygwin, but externally in Windows for this executable. And it was for sure not finding it. Much worse, we were going to be limited to a really bad environmental issue if we started asking the other Windows devs to install Python3 with a certain name in a certain manner or not use cygwin or something like that. (Have you ever asked a question about VIM and gotten “why don’t you do it like XYZ in Emacs? I hate that sort of answer and didn’t want to pass the buck along.)

Anyway, since I have seen many questions about this on StackOverflow (execute a cygwin script in maven, maven-exec plugin executing a python script, this one from 4 years ago) and I figured I’d share my solution to getting maven-exec-plugin to work with cygwin and other platforms happily:

There are a few little quirks you should know about with this:

  • An undocumented (probably unsupported) operation on the family tag. It allows us to say !Windows for all non-Windows operating systems. This works, but I can’t find it anywhere in the documentation.
  • You cannot control what version of python the user has on their system if they are using Windows. Maybe you could add a check, but we know for a fact that the generate-sources step will fail if the code isn’t run with Python 3 so for us this is not an issue.

I hope if you’ve been struggling with this very niche problem you will understand it better and your build will work. If you have any questions, feel free to contact me on Twitter or wherever and I will try to help!

Jimmy F

Written by

Jimmy F

Doing data in Paris, France

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade