Happily Coexisting: Building with exec-maven-plugin on both OS X and cygwin

At work at my job, we build almost everything with Maven. Prior to working here, I’d done a little work with the Java/Maven ecosystem of tools, but in general, I had no foundation of knowledge about it.
Here’s the general gist: All the code we write goes into a repo, that repo has a parent-pom. We have a big Jenkins server which takes your repo with its POM, puts it inside a larger project with an appropriate nickname for this big bundle of stuff, and builds the parent POM.
If you want something to be used anywhere within our company, you need to use this system. As an aside, it’s really nice because it enforces our policy of “config as code”, meaning that any oddities in how to build things must be encased in the POM file. There can be debates about this, but in a company with hundreds of developers: we need something, and this is it.
It was in this environment that I decided to fix a minor issue that had been bothering me one day: automated code which was stored in a Git repo. This is a sort of pet peeve — it’s like keeping an artifact that’s generated automatically in a Git repo. Without a really good reason (excuse?), you should never store things which can be generated with the repo inside that repo.
The way it worked was this: we have these Python files and they automatically generate a number of Scala files which we then use to make some super-type-safe operations even more type safe. When your employer processes 95 billion events a month, they generally don’t appreciate random exceptions that inspire your Hadoop job to blow up.
Regardless of the situation, I was annoyed two-fold:
1. We had to run the Python scripts “periodically” to regenerate new code
2. The new generated code gets checked in.
So from time to time, somebody would submit a huge 200–300 line change in a really obtuse Scala file. Our team would have to approve this and hope there weren’t any issues. I decided to change this — my strategy went like so:
- Add the Python stuff to the build process…
- Delete the generated code from the repo…
- Let the automatic builders generate the code…
- Profit.
And I almost did. Here is what (a skeleton) of the addition to the POM was:
This seemed like a really great idea, and in fact, for me, on Mac OS X it was wonderful. Another teammate who was using Linux tried it too and he found it worked. Additionally our automatic builders all run Linux and they had no issues.
I had made a mistake: not all my team members use Linux or Mac OS X. In fact, some use something far more nefarious: cygwin. They were running Windows with cygwin inside it to emulate *nix. This isn’t really surprising given how much .NET and C# our company is doing, but having been a fan of developing on a Mac, I didn’t even consider using a Win machine when I started here (no slight against Windows devs, whatever your flavor is that works for you).
The first problem was that python3 on Windows ends in .exe and doesn’t include the number 3. No problem — we set up a profile for this and changed the executable name.
The next problem was more complex. I don’t know a lot about cygwin, Windows, or how paths resolves, but I realized after a little investigation that the maven-exec-plugin was searching not in cygwin, but externally in Windows for this executable. And it was for sure not finding it. Much worse, we were going to be limited to a really bad environmental issue if we started asking the other Windows devs to install Python3 with a certain name in a certain manner or not use cygwin or something like that. (Have you ever asked a question about VIM and gotten “why don’t you do it like XYZ in Emacs? I hate that sort of answer and didn’t want to pass the buck along.)
Anyway, since I have seen many questions about this on StackOverflow (execute a cygwin script in maven, maven-exec plugin executing a python script, this one from 4 years ago) and I figured I’d share my solution to getting maven-exec-plugin to work with cygwin and other platforms happily:
There are a few little quirks you should know about with this:
- An undocumented (probably unsupported) operation on the
familytag. It allows us to say!Windowsfor all non-Windows operating systems. This works, but I can’t find it anywhere in the documentation. - You cannot control what version of
pythonthe user has on their system if they are using Windows. Maybe you could add a check, but we know for a fact that thegenerate-sourcesstep will fail if the code isn’t run with Python 3 so for us this is not an issue.
I hope if you’ve been struggling with this very niche problem you will understand it better and your build will work. If you have any questions, feel free to contact me on Twitter or wherever and I will try to help!
