I can understand your reasoning to default to Java for code where Groovy is not really needed.
David Kron

During my days as a Groovy developer I only had problems with joint compilation couple of times and I always found out that it was due putting Java source files in a wrong directory. Java sources needs to live alongside Groovy files in src/main/groovy not src/main/java or similar.

On the other hand I can see where you are heading. I usually use just a one-way code dependencies — interfaces and enums and other API/core code written in Java and an implementations and other logic in Groovy classes.