# Tabu Search — gentle introduction

Recently, I attended a course on discrete optimization on Corsera. One of the problems I was trying to solve is the Travelling Salesman Problem, the famous NP-Hard optimization problem. The course explained how to solve several real problems with several algorithms, one of them is Tabu Search.

Today, in this post I will explain this algorithm and implement it using Python to solve the Travelling Salesman Problem TSP.

# Hmm.. so what is Tabu search and where can I use it?

Tabu Search is a meta heuristic procedure for solving optimization problem designed to guide other methods to escape the trap of local minima. Tabu Search is used to find optimal and nearly optimal solutions for a wide range of classical and practical problems. From scheduling, to telecommunications, character recognition to neural networks. To avoid falling in a local minima, it uses a memory so it can remember moves and solutions that are already exploited. Also, it uses memory functions to allow searching strategies like intensify and diversify (will explain them soon).

Tabu Search can be used to guide other processes that uses a set of moves for transforming one solution into other and provides a guidance for measuring the attractiveness of theses moves. Example of moves are swapping between two tasks, changing value of a variable (increase, decrease).

A partial list of Tabu Search applications:

- Employee scheduling
- Maximum Satisfiability problems
- Character recognition
- Space planning and architectural design
- Telecommunication path assignment
- Probabilistic logic problem
- Neural network pattern recognition
- Machine scheduling
- Traveling salesman problem
- Graph coloring

# Tabu Search Principles

Tabu Search based on three main concepts:

- The use of flexible attribute based memory structures, that allows evaluation criteria and historical search information to be exploited move thoroughly than by rigid memory structures or by memory-less systems.
- A mechanism that evaluate the movement based on a specific criteria in tabu search
- The use of memory functions of different time spans from short term to long term, to implement strategies for intensify — to focus on a specific region, to diversifications — that drive the search in new regions.

# Short term memory and aggressive search:

Short term memory in tabu search can be represented as an aggressive exploitation that seeks to make the base move possible. By making a list of candidate moves that can lead to a new solution from the current solution.

Tabu Search provides constrains that prevents repeating of certain moves. By considering those moves as forbidden (Tabu)… (Ahhh that’s why it called Tabu Search!). Tabu Search is intended to prevent cycling back into a local minima, and broadly to introduce the search to follow a new trajectory.

But.. how can Tabu determine the best candidate? Simple!, using the objective function!..

Things are still not clear?! don’t worry I did not understand it also before I saw the following boxed graph:

# Example time!!!

ِA simple illustration for Tabu Search is the minimum cost spanning tree problem that includes constraints to prevent certain edge from appearing . The problem can be represented by five nodes so the spanning tree consists of four edges and every edge has a cost as illustrated in the image below:

In this problem we need to minimize the cost of connecting the nodes with each other. Every edge is represented here by xi where i=[0..7] and xi can take 0 or 1 (0 the edge does not exists). Every edge has a cost. For example the cost of the initial solution here is 6+2+8+0 = 16 (pretty good huh).

But wait.. don’t forget the other part of the problem, some edges are prohibited to appear.. so lets put some constraints here:

x1 + x2 + x6 ≤ 1

x1 ≤ x3

When we violate those constraints we penalize the cost of 50 for each unit of violation . So in the figure above we have 2 violations so the penalty is 100. and the total cost of that spanning tree is **116** :)

We need to find a solution that gives a low cost without violating the constrains above. To apply Tabu Search we apply swap transform by dropping one edge and add another to transform to another solution.

When we choose to add an edge we must drop another edge in a way that don’t create a cycle. Admissible move is the move that have the lowest cost taking into consideration the violation penalty cost.

Tabu restriction is defined here as the added edge we define as a tabu status. This makes edges that are selected as tabu to not been dropped out of the tree as long as they are tabu. In the example we allow only two edges to be tabu. Which means any added edge remains tabu for two iteration.

The aspiration criteria here is to override a tabu status if the resulting tree is better than the best tree produced so far. We will run the algorithm now for and discuss each iteration. Iterations are represented in figure 1 below:

**Iteration1**: from iteration 1 the best move that we can do in a way not to violate the constrains is by adding x3 and dropping x1. This will reduce the penalty from 100 -> 0. while increasing the cost from 16 -> 28. X3 now is considered as tabu.

**Iteration 2**: now by the tabu status rule we make x3 as tabu. so we can’t drop x3. From the remaining the best choice is to add x7 and drop x6. Then make x7 as tabu. The chosen move produce worst cost than the previous.

Now we have x3, x7 are considered as Tabu. and if we want to make any move we will make the cost worse. so we are in a state that has a bad cost and we can’t make any other move which is called local minimum.

**Iteration 3**: Now since we reach a local minimum we can override tabu status by adding x2 and drop x3. This move satisfy the aspiration criterion by producing a tree that has a better cost so we make this move.

**Iteration 4**: X2, X7 are now tabu. The algorithm will continue the moves by adding x3 and dropping X5. X3 and X2 now are tabu

**Implementation:**

I forked an implementation of tabu search in Python and improved it to solve the problem of Traveling Salesman, please feel free to use and modify the code:

**Conclusion:**

Tabu search is a meta heuristic search algorithm that utilize the idea of having short term memory to avoid sticking in a local minima. It has been used in many applications one of them is Traveling Salesman Problem. Speaking about TSP it worth to mention that the best reported algorithm to solve it is guided local search algorithm.

**References:**

I used the following reference as the main source of information written in this post (really this is the best resource for tabu search: