Create synthetic noisy document using Augraphy - 3

In this article, I will be sharing the methods to use Augraphy in creating a synthetic noisy document image by referencing another actual noisy document.

Original image, cleaned image and the synthetic image.

The reference image is extracted from a dataset called Tobacco3482 . In this document image, some distinct noise effects are bleeding ink, heavy noises, page border dark lines, binder clips mark, overlaying of page inside another document and etc.

Reference document extracted from Tobacco3482 dataset.

The noise effect in the document image above can be summarized below:

Each effect from the document image.

1. Binarization.
2. A little bit of fuzzy noise around the text.
3. Heavy noise at the right side of inner page.
4. Light noise at the left side of inner page.
5. Page border lines on bottom and right side of inner page.
6. Rotated inner page.
7. An overlay of inner page on top of main page.
8. Page border lines on bottom and left side of main page.
9. Two binder clips at the bottom of the main page.

In order to create a synthetic noisy document, two passes will be needed. In the first pass, augmentations will be applied on the inner page. In the second pass, the inner page will be overlaid on another main page and more augmentations will be applied.

First Pass

In the first pass, Augraphy’s augmentation pipeline will be used to create noise effect in the inner page. An augmentation pipeline consists of ink phase, paper phase and post phase. Ink phase and paper phase applies augmentations concurrently and post phase merge output from ink phase and paper phase first before to continue with the rest of augmentations.

First Pass — Ink Layer

The first augmentation in ink layer is Faxify and it is use to binarize the input image. monochrome is set to “1” to enable the binarization and the monochrome_method is set to “threshold_otsu” to enable Otsu thresholding method in the binarization process. More information on this augmentation can be found here.

Faxify(monochrome=1,
       monochrome_method="threshold_otsu",
       halftone=0)

Image before and after Faxify augmentation to binarize the document.

The second augmentation in ink layer is InkBleed and this augmention adds fuzzy noises around the text. intensity_range and severity adjusts the level of noise and kernel_size controls blurring effect of fuzziness. More information on this augmentation can be found here.

InkBleed(intensity_range=(0.3, 0.4),
         color_range=(0, 0),
         kernel_size=(3, 3),
         severity=(1.0, 1.0))

Image before and after Inkbleed augmentation to create fuzzy text effect.

The third augmentation in ink layer is BadPhotoCopy and this augmentation adds random heavy noises in the right side of the inner page. noise_type of “2” is use to create noises with fixed pattern. noise_iteration controls the number of applied noise layer while noise_size and noise_value change the size and value of noise in each noise layer. Higher noise_sparsity increases the area of noise while noise_concentration set the number of noise particles. edge_effect prevents the stacking of noises in the text by creating a layer of white edge surrounding the borders of text. More information on this augmentation can be found here.

BadPhotoCopy(noise_type=2,
             noise_side="right",
             noise_iteration=(30, 30),
             noise_size=(1, 4),
             noise_value=(0, 1),
             noise_sparsity=(0.7, 0.7),
             noise_concentration=(0.025, 0.025),
             blur_noise=0,
             wave_pattern=0,
             edge_effect=1)

Image before and after Badphotocopy augmentation to create noise at the right edge of image.

The last augmentation in ink layer is same with last augmentation and BadPhotoCopy is use to adds light noises in the left side of the inner page. The difference of current augmentation with previous same augmentation is the lower value of noise_sparsity and noise_concentration to create a lighter noises.

BadPhotoCopy(noise_type=2,
            noise_side="left",
            noise_iteration=(30, 30),
            noise_size=(1, 4),
            noise_value=(0, 1),
            noise_sparsity=(0.1, 0.1),
            noise_concentration=(0.01, 0.01),
            blur_noise=0,
            wave_pattern=0,
            edge_effect=1)

Image before and after Badphotocopy augmentation to create noise at the left edge of image.

First Pass — Paper Layer

In the paper layer, PageBorder augmention is applied twice to add dark border lines in the right edge and bottom edge of the inner page. side controls which side to apply the border line and width_range controls the width of the border line. noise_intensity_range adjusts the noise level in the border line. More information on this augmentation can be found here.

PageBorder(side="right", 
           noise_intensity_range=(0.5 , 0.8),
           width_range=(4,5))
           
PageBorder(side="bottom", 
           noise_intensity_range=(0.5 , 0.8),
           width_range=(4,5))

Image before and after PageBorders augmentations to create dark lines on the right and bottom edge of image.

First Pass — Post Layer

Post layer input is obtained by merging ink layer and paper layer output.

Ink layer output, Paper layer output and the merged Post layer input.

In post layer Geometric augmentation is applied to rotate the page slightly to the left. rotate_range can be changed to any range of positive or negative value in order to rotate the image in clockwise or counter-clockwise direction. More information on this augmentation can be found here.

Geometric(rotate_range=(-2,-2), randomize=0)

Image before and after Geometric augmentation to rotate the image slightly to the left.

Second Pass

In second pass, it takes another blank image as input and the augmented image from first pass will be overlaid on this blank image.

Second Pass — Ink Layer

In the ink layer, BindingsAndFasteners augmentation is use to overlay output image from first pass into the blank page. foreground is assigned with the output image from the first pass and edge_offset offsets the foreground and shift it up by the provided offset value. More information on this augmentation can be found here.

BindingsAndFasteners(foreground=img_output1, 
                     overlay_types="min", 
                     ntimes=(1, 1), 
                     nscales = (1,1), 
                     edge="bottom", 
                     edge_offset=(100, 100))

Image before and after overlaying output from the first pass into the blank page.

Second Pass — Paper Layer

In the paper layer, PageBorder augmentation is applied twice to add dark border lines in the left edge and bottom edge of the main page.

PageBorder(side="left", 
           width_range=(6,7),
           pages=5,
           noise_intensity_range=(0.0 , 0.2))
                   
PageBorder(side="bottom", 
           width_range=(8,12),
           pages=5,
           noise_intensity_range=(0.0 , 0.2))

Image before and after PageBorders augmentations to create dark lines on the left and bottom edge of image.

Second Pass — Post Layer

By merging ink layer and paper layer output, post layer input is computed.

Ink layer output, Paper layer output and the merged Post layer input.

In the post player, BindingsAndFasteners is the last augmentation and it is use to create two binder clips mark at the bottom of the main page. effect_type is set to “clips” and it is creating a synthetic clip mark effect in the page.

BindingsAndFasteners(overlay_types="min", 
                     effect_type="clips",
                     ntimes=(2, 2), 
                     nscales=(1.5,1.5), 
                     edge="bottom", 
                     edge_offset=(20, 20))

Image before and after BindingsAndFasteners augmentation to add binder clip marks at the bottom of page.

Overview of augmentation process

The link to the example code of the augmentations can be found here. I hope this article will be beneficial to anyone who read this and have fun in creating synthetic document !

An overview of the workflow in creating the synthetic noisy document.

Latest Update:

With the latest version of Augraphy, the same pipeline can be called with just 3 lines of code and here is the example notebook. Have fun!

from augraphy.default.pipeline import pipeline_archetype5

pipeline = pipeline_archetype5()
image_augmented = pipeline(clean)

Reference

1. The Augraphy Project. Augraphy: an augmentation pipeline for rendering synthetic paper printing, faxing, scanning and copy machine processes (Version 7.0.0) [Computer software]. https://github.com/sparkfish/augraphy.
2. Kumar, J., Ye, P., & Doermann, D. (01 2012). Learning document structure for retrieval and classification. 1558–1561.