Building a platform for the future, not just solving for specific client requirements

We at OrderGroove enable a large number of e-commerce clients to operate a frictionless commerce program within their business using our platform. Every so often a client will have a unique business requirement that is not handled by our platform out of the box. While most integrations are a smooth sail, sometimes we have to solve a problem outside of our platform’s core offering. On the platform team we analyze these requirements to extract a more abstract problem so that we can move the platform forward while also allowing our client to address their current needs.

As an example, Incentives appear to be something that is endlessly customized.

What is an incentive? It’s a benefit, you as a customer, can obtain on one, or many of your orders based on certain criteria or conditions. To name a few:

For a couple of years we solved each connecting line in the graph as they arrived as client requirements. We already had a good handle on applying a set of incentives to a given order. Still, the conditional part remained an ad-hoc thing.

I believe this to be a symptom of a lot of tech companies out there, where a bunch of “similar” problems are presented, but only appear between long periods of time. We, as software engineers, tend to focus on the problem at hand and figure out the best and cheapest way to transform a requirement into testable and releasable code, not knowing that our codebase is slowly decaying in front of us, and worse, by our own hands.

We’ve all been there. We’ve all seen it in some shape or form. But as sad as that might sound, given enough time and thought, there’s usually a clean and maybe even clever solution to the problem. Fear not — solutions are coming!

So how did we go about solving all of these problems by solving just one? We focused on the problem statement. After doing this over and over again, we noticed that they all followed the same pattern: When Condition X is met apply Incentive Y.

We went to the drawing board to rethink our approach. We created Flexible Incentives to deliver these requirements quickly and with little to no effort. Flexible Incentives became a tool used by employees, not just engineers, within our company.

The tool proved to be so useful that we’ve extracted its core and started coding different functional execution based on application/business domain level conditions.

Here’s how it all started.

Internal requirements

• Configurable — with minimal input we would get a Condition up and running
• Extensible — new Conditions can be easily implemented and hooked into the platform
• Reusable — an existing Condition can be used multiple times with different configurations
• Composable — a Condition can be comprised from the combinations of other Conditions using logical operators (and, not, or)

Since a Condition can be composed with logical operator, we created the concept of a Premise. At its core, a Condition is built from one or more Premises. For example:

1: condition = And(Premise1, Or(Premise2, Not(Premise3)))

where Premise1, Premise2 and Premise3 are Premise configuration identifiers. Something we can use to get that Premise configuration instance. A slug, id, etc

To configure Premises, we stored it as a string “And(Premise1, Or(Premise2, Not(Premise3)))” as a database field within the Condition model described below. Where premises is the field holding the string representing the condition. You can choose a location that best suits your needs, a file or database.

Going from literal string to something useful

If we take a closer look to our condition: And(Premise1, Or(Premise2, Not(Premise3))) we can see that it’s no more than an unbalanced tree.

What we need is a way to parse the keywords from our condition (And, Or and Not) where each keyword is a non-terminal node in the tree and the premise identifiers are the terminal nodes.

Lucky for us, there’s a Python package that makes these type of problem solving pretty straight forward: pyparsing

Pyparsing provides all the building blocks to define your grammar. It lets you define how each term in the grammar should be treated (for example: call a function every time you see a specific term, cast it to an integer, etc). In our case, we care about our operators (internal nodes), our terminal nodes and how every expression needs to be constructed.

The short of the long is the following:

• Operators: And, Or, Not, Condition
• Terminal nodes: Slug identifying a premise (Premise1, Premise2, etc)
• Parenthesis: Left and right parenthesis. We don’t care about them but they are part of the construction
• Arguments: One or more of either a terminal node or a sub-tree
• Expression: operator + lparen + args + rparen

Moving forward we’ll refer to the Condition operator as C. C is how we define a Condition with a single Premise (example: C(Premise1)). Whenever we see a single premise, we wrap it around a C constructor.

Given how we defined our terms, parsing our example condition will result in the following list of terms:

['and', <Premise1 Instance>, ',', 'or', <Premise2 Instance>, ',', 'not', <Premise3 Instance>]

You can define your terms to work however you see fit because Pyparsing is extremely flexible.

Let’s move ahead and continue with the logical operators and a way to reduce the Condition terms into a boolean expression:

So far we've built a way to configure Conditions, a parsing tool and a way to represent our configuration in actual python code. Let's put these pieces together. Based on the parsed terms from pyparsing we need to construct the same representation using the ConditionInterface. Here's how we do it:

Feeding the parsed result into the Grammar Interpreter would give us what we need. It looks something like this:

ConditionInterface.And(<Premise1 Instance>, ConditionInterface.Or(<Premise2 Instance>, ConditionInterface.Not(<Premise3 Instance>)))

Since our ConditionInterface in its core is reducing Premise Types evaluation into booleans, let’s continue with another core building block: The Premise Type base class.

The Premise Type base class has the following:

• The premise (first argument) is the premise configuration instance (such as customer type or location) which is everything we need to fulfill the “When Criteria X is met” part of the problem statement.
• The order (second argument) is when the order conditions are being checked against to determine if incentives can be applied. From the order we have access to the customer, the order items, and anything abstractly representing the “thing” that premises check against for whatever they may need.
• _get_values_from_premise is fetching that information and storing it locally for further usage.
• _is_valid is where we put everything together and from the configuration values we are able to determine if the Criteria is met or not.

A concrete example of this might look like this:

Going deeper into Premises, a Premise configuration is a Django model that looks like this:

So far so good. Yet, there are still missing pieces to this puzzle. Starting with: How do we go from a Premise Configuration identifier to an actual Premise Instance?

If you recall, in our condition definition: And(Premise1, Or(Premise2, Not(Premise3))), Premise[1|2|3] were identifiers. We need a way to map those identifiers to an actual Premise instance. Above we said that we can tell pyparsing to execute a function whenever we see a specific type of term. Each Premise has a unique identifier (slug) that we can use. Let’s create a factory to use with pyparsing so that whenever it sees a slug term can give us back an instance of a Premise. Here's how we do it.

Some pieces are hard-coded for better readability here , but we keep a mapping from a Premise.premise_type to an actual class that we can instantiate properly.. From the Premise configuration record, we have access to its type, and from the type mapping we have the actual class we want to instantiate.

And that's basically it. We managed to satisfy all our internal requirements: It’s configurable, extensible, reusable and composable.

Now that we have all the necessary pieces available to create a service layer to orchestrate the grammar parsing -> premise instantiation -> premise evaluation -> condition evaluation policy, let’s build a simple Service as an example.

Here's a base class:

And a “concrete” implementation looks like this:

We feed the service with an Order to be processed and the corresponding Item. From there, we get all the conditions, loop them over, then parse their premises constructing our logical blocks from the different premises. Be feeding the order into each Premise Type we get necessary information to say if the Premise is True or False for the given order. After evaluating all premises, we reduce the Condition with its Condition Interface into a single boolean. If True, we add those Condition Incentives to the returning collection.

In the beginning, when we created the Condition model, we defined a ManyToMany field for Incentives. And since we already knew how to apply a set of incentives to an order and/or item, the rest is history.

Conclusion

Sometimes the right solution is not clear. Sometimes the right solution will not pop up until you iterate over the same type of problem multiple times. That’s exactly our case.

Keep striving for it. Keep a close look at the problem statement. Think about the terms of the problem statement. See if there’s a repeating pattern in those terms. And fear not — solutions are coming!