Externalized, Reloadable Validations
Dynamism in API & Reactive Event Validations.
Objective is to return the list of all validation errors at one shot, saving time and network bandwidth. Enables dynamic injection of custom validations with minimal effort for the objects and object graphs. Expresses the validations through a configurable, readable data serialization standard that can be loaded from external storage systems. Allows for customized error codes and error descriptions for errors.
When we need to meet the below two primary goals, we can adopt this. Otherwise regular and already existed spring/servlet annotation based validations are enough.
- Configurable and Relodable during runtime from external systems. Like Spring Cloud Configurations.
- Mappable Custom Error codes and Error Descriptions to produce.
Background: The software and technology world is moving rapidly to cloud-based, cluster-oriented micro-services solutions. Many software services are built to collect input from end customers’ activity on the web or system generated messages. Single input payloads to these services can consist of hundreds of thousands of data elements to process. With this magnitude of data, it becomes critical to have appropriate validations to ensure the customer-supplied or system-supplied data elements are correct, also known as input validation before the system processes the data. These validation services are responsible for responding to customers or system inputs with accurate failure representations, including details on why the provided list of input payloads were invalid. When the service cannot process the given payload, this feedback enables the input user to correct the errors, before attempting to send the request again.
Problem Statement: Applications are required to validate customer or system-supplied inputs, while accounting for various custom enterprise standards for API or system calls. Responding to customers or systems explaining what input is invalid among many nested data structures is very challenging. There is an additional need to avoid solutions that validate each input element with custom error codes and descriptions because it is time intensive to create and often needs multiple updates as requirements change, wasting development resources. Also, there is an opportunity to gain computing efficiency when input elements are requiring multiple validations to fail, by exiting the process after the first failure.
Solution Summary
Configurable: All the data configurations and validations, including the error codes and corresponding error descriptions, are externalized from the source code. The system reads from a yml file structure to simplify the externalization and to avoid text duplication.
Reusable: The same validation with corresponding custom error code and error description, can be used for multiple data inputs. The existing validation can also be extended to build a brand new custom validation and it will be injected automatically.
Dynamic: Users create the validations and the auto injection mechanism inserts them to the framework. The system will choose the validation dynamically during runtime and will run the validator based on what is passed from the injection mechanism.
Extendable: Existing validations can be extended to build new custom validations, such as performing several validations on a single input, without duplicating the checks.
In-Memory: Validations will be loaded during the system start-up time and kept ready in-memory, allowing quick execution and not requiring reloading. With this approach the validations can be run quickly on input payloads, because at runtime there is no waiting on load time for multiple properties from multiple type elements.
Reloadable: As and when needed we now have the opportunity to reload the configurations at the runtime.
The above high level diagram gives an overview of the following functional flow:
1. The configurations are supplied from external locations.
2. The validation framework’s interaction with the configuration files to produce an error list.
3. The error list produced by the Framework has corresponding error codes and error messages.
The above diagram illustrates the internal details of the Framework:
Configurations: This module loads the external validation configurations, also known as YML files, from an external storage system and parses them to be understood by the framework for further executions.
Domains: The actual objects supplied to validate. Irrespective of the type of the system object provided, it reads the object and loaded validations and initiates the validation process.
Executor: The heart of the framework that executes validators to validate the properties from a supplied object. Reads every property from a supplied input object, finds the validation to run, and then runs the validator on the property. After the basic validations are finished, the executor kicks off two different processes, “RunWhenConditions” and “CustomUnDefinedConditions”, for further filtering of the produced errors. These processes refine the error list by incorporating User-defined logic, such as ignoring a set of errors if a certain validation failed. All errors are then wrapped up into a Custom JSON object and are sent back to clients in one call. This technique saves a significant amount of time and network bandwidth for customers versus a process that sends requests for each validation failure.
The above diagram illustrates further details at the executor level.
For every given property, the Executor collects the property metadata: Type, Name, Parent Type and the Property Value. Using the metadata, the executor constructs a key to search for the validations in the supplied configurations.
Example Configuration:
"[employer.employee.fullName.firstName]": NULL_EMPTY_CHECK: 206 | Employee first name is Empty or Invalid
The key here is:
employer.employee.fullName.firstNameProperty name to validate becomes:
firstNameNested parent name becomes:
employer.employee.fullNameThe key constructed by the framework using its meta-data
employer.employee.fullName.firstNamewill now be searched and the set of defined validators retrieved.
The map of validations will now be retrieved from the configurations and becomes:
NULL_EMPTY_CHECK: 206 | Employee first name is Empty or InvalidValidator is
NULL_EMPTY_CHECKError Code to produce is:
206Error Message to Produce is:
Employee first name is Empty or InvalidThis information loaded from the externally stored, configurable YML will be extracted while triggering the validator
NULL_EMPTY_CHECK.When the validator finds the error message, the Validator Framework will create a wrapper of the ErrorCodeDescription object, which will be added to the global list of error codes. The validator will then proceed to the next property to validate.
As part of this process, the system first reads all the properties, aka elements of a given object supplied, irrespective of its type, and collects each property’s metadata.
Using this metadata system, the executor will now determine the type of property and intelligently find the corresponding validation to run.
The system constructs a searchable key using the property metadata and retrieves the map of validations to run against using the same key from the external configurations loader.
Then the system verifies the existence of each of the validations retrieved from the validations map. If the system cannot find the validation, it logs an error message as a missing injection for that particular validation and proceeds to validate the next validation defined in the map.
If this is custom type requiring further validations for the nested fields, the system will start the process in a recursive manner and proceed forward.
Consider an example to apply for a job to an employer. Employer will receiving the application with multiple fields inside and lets validate them from the configuration like below:
👉 How to run this?
Cite me or post a comment to to view the source code. :-)
Find the publication here: https://medium.com/thinkspecial
