Input validation in GoLang

Having used GoLang for 2+ years now, I am constantly being asked how to do input validation in Go so I decided to walk through the existing practices and libraries

DISCLAIMER: This is not “the best way”, “should do”, “must do” article, instead I will just try to share my experience and what I find to work out well.

Pre-kickoff

Assuming that you are familiar with GoLang and knowledge is at the level

if err != nil {
   return err
}
syntax annoys me... 🤦‍♂️

Prologue

As long as data is mostly used, manipulated or transferred through structs in Go it all boils down to validating a pre-definedstruct which matches the expected input data format

Whether the input consumer is an HTTP API or Lambda/Cloud function the very first thing is to unmarshal the input stream into a struct

So all the examples below assume that you have already done the unmarshalling part

A Junior Gopher approach

If you are a junior Gopher like me a couple of years ago and you wonder how to validate user input you would probably come up with the following solution

Assuming that you have a domain entity User which has IsValid() method which can be used to validate it wherever needed

Fast, easy and cheesy if you have a small service with 1–2 endpoints or 1–2 Lambda/Cloud functions. This solution could be a good way to go, it is a hassle-free time-saver and it does the job though the general rule is avoid doing it!

bottleneck:

• does not scale well
• extremely non-reusable
• not flexible
• time-consuming to maintain when codebase gets bigger and bigger
• unit testing (don’t be one of those bros who is too good to unit test 💩)

Once your API/Service scales to more endpoints, more domain entities and more business logic this will bring you pain on the arse. Actually, you will probably spend more time maintaining and testing validation of your domain entities rather than developing the business logic itself.

The idiomatic way to Go!

After getting to the point where the first approach does not work anymore it is time to dig deeper to find a more generic way to validate. As an ordinary modern cyberman, I google’ed validation in Golang and the very first link revealed the idiomatic way to validate structs in GoLang

Go offers struct tags which are discoverable via reflection. These enjoy a wide range of use in the standard library in the JSON/XML and other encoding packages

Which also means tags also can be used to define validation rules for each field inside struct. For the User example it would look something like this

There are few libraries out there which take in a struct with validation rules defined in it and do the validation for you

Go Playground Validator

The first one we’re going to look at is the most famous and has the most stars on GitHub among the existing libraries

go-playground/validator

How to use

Here is the basic usage of the library

If you run the code above the output would be

Key: 'User.Email' Error:Field validation for 'Email' failed on the 'email' tag
Key: 'User.Name' Error:Field validation for 'Name' failed on the 'required' tag

Add custom validation rule

https://godoc.org/gopkg.in/go-playground/validator.v9 provides almost every validation rule you may need, but sometimes you still need to have a custom validation rule like password strength formatted string or even duplication in the database

The output for the code above would be

Key: ‘User.Password’ Error:Field validation for ‘Password’ failed on the ‘passwd’ tag`

Customise error messages

As you might have noticed the error messages look a bit dodgy and not quite representative. If you are running a Lambda/Cloud function or an internal API probably you are good to go with these messages, but if you have a client-facing API probably you may need to customise them

So the output is

Email must be a valid email
Name is a required field
Password is not strong enough

The very last bit is when you receive a JSON payload with lower-case keys

{
    "email": "some"
    "name": "My",
    "password": "123"
}

you also want them to be lower-case in the error messages as well. Just register a custom name mapper

validate.RegisterTagNameFunc(func(fld reflect.StructField) string {
    name := strings.SplitN(fld.Tag.Get("json"), ",", 2)[0]
    if name == "-" {
        return ""
    }
    return name
})

and the output is

email must be a valid email
name is a required field
password is not strong enough

This package will cover all real-life (ish) scenarios you will have.

There are a couple more libraries with the exact same approach and do almost exactly the same and they are customisable as well. I will just leave it here if anyone whats to discover (I never used them though)

asaskevich/govalidator

go-validator/validator

Yet another approach 🎉

Having choice makes us feel stronger! Like many of you, I also like having different choices to compare. So I picked up another existing library with a different approach

go-ozzo/ozzo-validation

The approach here is to code validation rules, which, IMHO will get annoying at some point

The only advantage of this approach is if you, for some reason, need to validate a value like string int this fits well. It also allows you to add custom validation rules

can validate custom data types as long as they implement the Validatable interface.

as well as custom validation error messages

customisable and well-formatted validation errors

Do not make a mess 💩

I have got a number of suggestions on how to organise validation and how to structure your application

Use validator as a dependency

This one is pretty simple and quite obvious. According to The Twelve-Factor App

Explicitly declare and isolate dependencies

Create it in one place and pass through as a dependency to the destination.

Keep it in the internal package

This one is more about how to structure your application and where should the validation live in the filesystem

Given this is, vaguely, your project structure

--project
---cmd
----server
------main.go <- create an instance here, pass all the way down
----lambda
-----my_func
-------main.go <-- do all validations here
----internal
------user
--------handler.go <-- Do all validation here and keep service clean
--------service.go   <-- Service will just do the business logic
--------repository.go
------validator
--------validate.go <-- Register all custom rules, messages, trans here

Create a validator instance at the very beginning of the app and pass it all the way down to handler (or wherever the validation takes place), do all the validation in the handler and pass the valid input to service to do the business logic. I find this extremely testable, robust and easy-to-make-changes. In the case of Lambda/Cloud function do everything just before triggering Service or processing the business logic

Do not define validation rules on domain entities 🙅‍♂️

A good general rule would be — keep you domain entities as clean as possible. According to Clean/Hexagonal/Onion/DDD architecture

I strongly recommend to have a read article

Entities encapsulate Enterprise wide business rules. An entity can be an object with methods, or it can be a set of data structures and functions. It doesn’t matter so long as the entities could be used by many different applications in the enterprise.

Long story short — in one part of the application an entity can have some validation rules in another part entirely different.

A good solution here that works out pretty well is to always define an input struct with validation rules per endpoint/lambda even if you get to copy the entire entity

Few things here…

• a create-user request comes in
• gets mapped into createUserRequest entity
• the entity is being validated
• if it is not valid we are done here, no rubbish data gets pushed further of this point
• if the data is valid it is passed to the service to do the business logic
• service gets to work only with a valid, trusted core domain entity

🎉🎉 Easy peasy 🎉🎉

Hope this will help you to validate inputs efficiently and structure your project better