Code Series, Code Generator In Action

Writing code to generate code is a winning strategy.

Over the years I have written a lot of code. Typically, the code was purpose built to represent objects in memory, perform object value manipulation according to predefined rules then persist the output somewhere. Simple. Yet, I have often wondered if much of that code had to be written in the first place. As I have pondered such questions, my long term memory started to activate providing ample flashbacks of various projects. There were a good number of them where I have purposefully applied a code generator. Perhaps, I did have a moment of brilliance in the past and did not force myself to write everything from scratch. Perhaps the concept of a scaffolding code generator has been around for a long time … perhaps it is the secret behind an awe inducing productivity of small teams … perhaps it should be a major component of code writing code, perhaps …

I have been a long time subscriber of this concept. In fact, the code generation has been a key success factor on a number of projects I have been involved in. So, after a short deliberation, several late night hours and intravenous delivery of caffeine (delicious coffee) I have produced a simple python based code generator to illustrate the concept of a configuration driven code generator.

The process begins with the code initializing itself with instructions contained in the JSON configuration file. The main engine class then invokes those instructions in sequence, instantiating specified implementation class, passing in additional parameters and invoking the main process (take action) method, process diagram is featured below.

The following is a sample of a JSON based configuration file that drives the code generator engine. As specified, the “implementation” parameter identifies the class to be instantiated to perform a primary action, while “params” section contains all of the specific values controlling the processing invoked. This is a simple and effective implementation of a configuration driven (functional programming) paradigm.

{
    "name": "Code Generator",
    "description": "Your application description...",
    "version": "0.1.0",
    "task_list": [
        {
            "task_name": "your.task.name",
            "implementation": "folder.module_name.class_name",
            "params": {
                "source": {
                    "file_name": "./file_path"
                },
                "transformer": {
                    "file_name": "./file_path"
                },
                "output": {
                    "file_name": "./file_path"
                }
            }
        },
    ...
    ]
}

An interesting component of this generic code generator is that each of the tasks can be easily swapped out with an alternative implementation or simply expanded upon. The code generator engine is driven purely by the JSON configuration file and as such can be re-purposed for various applications. The provided example performs two simple transformations, first is an XSLT transformation of an XML file that produces a JSON file with its own unique format representing all of the data contained in the input file. The second transformation produces a series of working Python based classes as defined by the contents of the XML input file. This example can be easily extended with additional transformations to produce Java, .NET or other code. In fact, this code generator is so generic that it can be used to produce other JSON based configuration files and code generators.

You can find the code for this project at https://github.com/fszale/object-to-code-generator.

I have been using this model of code generation to successfully produce code and configuration files. The input to the generator consist of object definitions documented with modeling tools that export XML based entity definition files. However, this is just one possible use and you are encouraged to fork and modify the provided code base. Python was the language of choice as majority of my time is focused on big data and machine learning projects, however, given the simplicity of this project the code could easily be ported to language of your choice.

I look forward to your modifications, ideas and a unique input, transform and output combinations.