To Post or Not To Post
Can a computer program decide how to improve an image?
When my “artist’s assistant” app generates an image like the one above, I have to make a decision: should I leave it as it is or should I apply post-processing to enhance the image in some way?
The temptation to opt for post-processing is very strong because of the impressive array of tools available today. My favorite is GIMP with the G’MIC-Qt plug-in, which provides dozens of useful filters. The possible effects range from imitations of famous paintings and styles, to technical image-processing operations, to bizarre transformations that change an image beyond recognition.
Some examples
Given an image like the one above, I might consider using a filter such as G’MIC “Local Contrast Enhancement.” This filter performs a straightforward image processing operation, as its name implies. As can be seen here, the filter intensifies the colors without altering the composition of the image:
G’MIC has several similar filters that perform local operations to intensify some aspect of an image without significantly altering its composition.
Another G’MIC filter, “Dream Smoothing,” blurs an image, which softens the edges of the shapes in the image and to some extent homogenizes the colors within the shapes. As with Local Contrast Enhancement, the overall composition of the original image is still clearly visible.
The G’MIC filter “Brushify” attempts to give the image a painterly appearance:
The brush effect leaves the original composition and color scheme pretty much intact, but the brushy texture adds a new element to the image. It should be noted that the Brushify filter is a powerful tool in its own right. Brush size, shape, orientation, fuzziness, and many other features can be specified by the artist.
The next example uses the G’MIC filter “Angoisse (Anguish).” This filter casts a dark mood over the original image, but the essentials of the composition are still visible:
Some extreme examples
The final four examples are all dramatic transformations of the original image. With these transformations we go well beyond the type of image produced by my art app.
As we move away from filters that perform local operations on an image, we enter a realm where the original image starts to recede into the background while completely new effects come to the fore.
The contributions of the artists who created the G’MIC filters to be demonstrated are very clear. Perhaps these artists would have to be acknowledged as co-creators if I were to use these filters. For that reason I probably wouldn’t use any of these filters.
The first example uses the G’MIC filter “Barbouillage Paint Daub.” The overall outlines of the original composition are still visible, but the color scheme has been noticeably altered:
An even more drastic transformation is provided by the G’MIC filter “Color Abstraction Paint”:
This image is obviously very far from the original. The bare essentials of the original composition are still present but the color scheme has been completely changed. If I really wanted an effect like this I’d have to consider adding functionality to my app to generate this type of image directly rather than relying on post-processing by another program.
The next example, which uses the G’MIC “Painting” filter, takes a further step away from the original:
Here almost all the features of the original composition have disappeared. As with the preceding example, if I wanted this effect I’d feel compelled to modify my app to generate this type of image directly.
The final example superimposes a texture pattern with its own color scheme on the original image:
This image departs so far from the original that I wouldn’t modify my app to generate images like this or submit its typical type of image to this kind of treatment by another app.
The place of post-processing in my computer art
GIMP offers dozens more filters than the few shown here. There are, in fact, so many filters that making a choice of which to use can be overwhelming. It’s hard to know where to start.
My own take on post-processing is to let my app generate images that I generally like and then, at the end, apply a single filter to a selected image if I think it will improve the image. I use GIMP exclusively for this kind of post-processing.
I incorporated a small group of filters into my app just to handle situations that I know can arise because of the complexity of many of the image-processing operations performed by my app. I sometimes need to adjust contrast, so there’s a small group of filters for that, and I sometimes need to adjust the color scheme of an image, so there’s a small group of filters for that. So, for example, my app’s built-in filter “sharpen” was used to enhance the fine detail of the original image shown at the top of this article.
Most of the time the app’s built-in filters suffice to solve the most commonly encountered problems. As a result I usually have no need to resort to GIMP or other external app.
Can the decision to post-process be automated?
My app awards an “aesthetic score” to each image it generates. This score is a crude evaluation of the quality of an image. It’s sufficient to weed out the worst ones before they’re displayed or saved.
Examples of “bad” images are images that are just a solid block of a single color, highly repetitious patterns such as a checkerboard, random noise, and nearly empty images that have all their detail concentrated near the top or bottom or against one side. Automatic rejection of such images saves the time that would otherwise have to be spent culling good images from a run of images containing many bad ones.
The aesthetic score is not sensitive enough to identify aspects of a good image that could be improved by applying some form of post processing. This capability probably requires an AI or machine learning program many times larger and more sophisticated than my app.
