NIFI Custom Processor Series Part 2: Add custom validation to custom processor

This discussion is a continuation of a NIFI Custom Processor Series Part 1. If you are want to know how to develop and test a custom processor in NIFI from scratch or want to know what we discussed in part 1, please check out this blog: https://medium.com/@mr.sinchan.banerjee/nifi-custom-processor-series-part-1-writing-your-first-custom-processor-dd7aa901c896 .

NOTE: If you are new to Apache NIFI, I would suggest you to check the official documentation of Apache NIFI. If you want a concise discussion covering the basics concepts of Apache NIFI, you can refer Chapter 8 of my book — Scalable Data Architecture with Java.

In our last part, we developed a custom processor called Salutation processor, which takes two properties — Salutation and BeforeOrAfter, and produces a flowfile with Salutation. In that implementation, property BeforeOrAfter can take the values A( for after) or B (for before). However, there was no way to check or validate if the String entered is A,B or some other value. If someone sets the value as C, it will fail during data processing. However, this kind of validation can be done, even before starting the processor.

All NIFI processor gives a capability to do compile time validation of the properties that are set in NIFI properties. This is done using Validator API provided by NIFI.NIFI provides few standard validators implemented using Validator API (we already have used one called StandardValidators.NON_EMPTY_EL_VALIDATOR in PART 1 of the series). Apart from that, you can create your own custom validators using Validator API.In this discussion, we will implement a custom validator to enforce that A/a and B/b are the only valid values for the property — BeforeOrAfter.

So, first let’s look at the PropertyDescriptor of BeforeOrAfter property defined in the Salutation Processor.

  public static final PropertyDescriptor BEFORE_OR_AFTER = new PropertyDescriptor.Builder()
            .name("BeforeOrAfter")
            .displayName("Insert Before Or After")
            .description("Whether you cant prepend or append the salutation. For prepend use - B and for Append - use A")
            .addValidator(StandardValidators.NON_EMPTY_EL_VALIDATOR)
            .expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
            .required(true)
            .build();

In the code snippet above, we have used StandardValidators.NON_EMPTY_EL_VALIDATOR for validating the property BeforeOrAfter. This only validates that this property cannot have an empty value.However, it doesn’t validate if the value set is only A or B. We intend to change this validator with our custom validator. So, let’s first build our custom validator.

Step 1: Building the Custom Validator

First we will create a package called org.example.validators under org.example package.Then we create a class called CustomValidators under this package. The code snippet of the CustomValidators is as follows

public class CustomValidators {

    public static final Validator BEFORE_OR_AFTER_VALIDATOR = new Validator() {
        public ValidationResult validate(String subject, String value, ValidationContext context) {
            String reason = null;
            if (!"B".equalsIgnoreCase(value) && !"A".equalsIgnoreCase(value)) {
                reason = String.format("Unsuported value present: '%s', supported values are:[A,B,a,b]", value);
            }
            return (new ValidationResult.Builder()).subject(subject).input(value).explanation(reason).valid(reason == null).build();
        }

    };
}

In the class (shown in the code above) , we define a constant field of type org.apache.nifi.components.Validator. We implement an anonymous class by implementing the interface Validator. The Validator interface contains an abstract method called validate() which returns an object of type org.apache.nifi.components.ValidationResult. In order to implement a validator, we must implement validate() method and return a ValidationResult object.

First we set a string variable (named reason) based on whether value is equal to A or B or not. Then we use the ValidationResult.Builder library to build the ValidationResult object. We set the value of reason variable to explanation using ValidationResult.Builder() method. We also calculate a boolean value based on whether reason variable is null or not (reason == null)and pass the result to the valid() method which accepts a boolean. A true passed to this valid() method denotes successful validation, otherwise, it is a failed validation.

Now that we have built a custom validator, let’s use it in our processor.

Step 2: Using Custom Validator to validate Processor properties

Let us now replace the validator used in the property descriptor to the custom validator we used just now. The code of this property will look like the following

  public static final PropertyDescriptor BEFORE_OR_AFTER = new PropertyDescriptor.Builder()
            .name("BeforeOrAfter")
            .displayName("Insert Before Or After")
            .description("Whether you cant prepend or append the salutation. For prepend use - B and for Append - use A")
            .addValidator(CustomValidators.BEFORE_OR_AFTER_VALIDATOR)
            .expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
            .required(true)
            .build();

For the sake of our discussion and to show how this change will create a different user experience, we copied the code of SalutationProcessor to a new class called SalutationProcessorV2 and changed the BEFORE_OR_AFTER property descriptor in that class.

Now, we will build the nar and redeploy it.

NOTE: you can learn how to build and deploy a nar in the previous post of this series

Step 3: Test the new feature of the processor

After deploying , let us set the value of the BeforeOrAfter property to be C in both SalutationProcessor and SalutationProcessorV2.

Once you have set the value to C, and then click the validate (tick) button present in top right hand corner of properties tab in the NIFI processor, you will notice different behavior.

The following screenshot shows that validation passed for a wrong value C, when we use SalutationProcessor ( implementation discussed in Part 1).

Figure — Validation Passed for wrong Property value in SalutationProcessor

However, with the custom validator, it displays validation error in SalutationProcessorV2 when BeforeOrAfter property is set to C. The following screenshot shows the validation error in NIFI UI as follows.

Figure — Validation Failed for wrong Property value in SalutationProcessor

As seen in this example, we were able to successfully enhance our processor built in Part1 using NIFI Validator API. You can find the full code base for this blog at: https://github.com/perfecxus/SalutationProcessor

Hope you found this read useful. Please feel free to discuss or post question in the comment section below.

I will see you again in the Part 3 of the blog series. Till then Happy Learning and Stay Safe !!