Using Apache ANT to find DB Object usages

This January, I came up with Scan your SOA Suite JCA Files, where I explained how to use Apache ANT to find all your JCA files and fetch relevant info from them to create a CSV file from it.

This week, at my current customer I wanted to gather all the relevant usages of the tables and views from the database application. The application has an Oracle ADF Front-End and a backend based on Python/Jython scripts. So, I wanted to scan all the .py files from the back-end, and also the .sql files that are also in that folder structure. And I wanted to scan the ADF Business Components: the entity and view objects.

Apache ANT to the rescue (again)

The scan of the .sql and .py files is quite straightforward and resulted in AnalyseUtils — ScanScriptsForDBObjects.

Apache ANT Script in Oracle JDeveloper

Scanning for ADF BC Entity and View objects are a mere copy of AnalyseUtils — ScanJCAFiles, and resulted in AnalyseUtils — ScanADFBC.

They work quite the same. The ANT script scanADFBC searches for entity objects in the **/model/entities/*.xml path in one loop and for view objects in the **/model/views/**/*.xml path.

Since there may be association objects and maybe other files between the entity and view objects, they’re filtered using the construct:

    <fileset id="entityObject" file="${entityobject.file}">
      <contains text="DOCTYPE Entity SYSTEM" casesensitive="no"/>
    </fileset>
    <if>
      <resourcecount when="greater" count="0" refid="entityObject"/>
      <then>
       ...
    <if>    

And a similar construct in the view object loop. It checks for the Document Type Definition for Entity or ViewObjects referring to a DTD file.

I use the xmlproperty ANT-task. This requires that the XML is well-formed, but also namespace-less. To get to the information in the XML, the elements can be handled as properties using the dot notation. This means that you should be able to traverse through the elements by naming those elements. I couldn’t get to repeating elements using an index or by the value of a child element or attribute, like an XPath query. At least, I haven’t found any examples.

Therefore, I created an XSLT map for entity objects and one for view objects. Have a look at the loopAttributesForTables, loopAttributesForColumns, and loopViewAttributesForList templates in those files. They’re Recursive templates to build up a comma-separated list of repeating elements. I wrote about it, years ago in Recursion in XSLT and these templates are extracted from that article. These templates build up a comma-separated list of (deduplicated) table names, table columns, and view attributes. These lists are then added to the CSV file between quotes.

The before-mentioned DOCTYPE entry is an XML annotation (Document Type Definition), in this case to refer to a DTD file that the XSLT processor cannot find and stumbles upon. To prevent this, a copy is made into the temporary working folder using this snippet:

        <copy tofile="${entityobject.file.edit}" file="${entityobject.file}">
          <filterchain>
            <linecontains negate="true">
              <contains value="DOCTYPE"/>
            </linecontains>
          </filterchain>
        </copy>

This copies the file, skipping lines that contain the “DOCTYPE” annotation.

Why not Python or Groovy?

You may think it is weird nowadays to use Apache ANT. Since Apache ANT is originally created for compiling and building Java Applications, for which it is de-facto replaced by Maven long ago.

And nowadays most people would use Python or maybe Groovy to create scripts. And you could do the same indeed using those languages.

However, Apache ANT is a very power full (XML) extensible scripting language, with for me one very handy feature: file selection.

Using ANT I can create multiple FileSets that can be nested or combined using multiple paths and path selections like **/model/entities/*.xml.

This delivers a list of files throughout a large folder structure, where anywhere in that folder structure a folder model with subfolder entities is expected, with *.xml files in it. For each of those files, you can call a target (procedure). ANT also has smart property functions that create new properties from those paths getting the base folder of the file, and the parent base folder of that folder. Within that folder, you can search for other files, for instance, the .jpr or .jws file to get to the Java Project or Java Workspace.

I guess this can be done using Python as well, according to this StackOverflow article: Recursively iterate through all subdirectories using pathlib.

But, doing so I get a pretty compact script that finds all the files that I want to process. For some more complex string handling, you can use JavaScript snippets in later versions of ANT.

Conclusion

To me, this is another nice example of Apache ANT that makes me hope that it stays around for a long time.

But also the XSLT examples are interesting, I think. You might want to learn about recursion in XSLT for your Oracle Integration Cloud, Service Bus, or Apache Camel implementations