Crux Learnings.
Published in

Crux Learnings.

Are we alone in the universe? The AI is our daily companion

Since the beginning of my career I have always been passionate about complex issues that help people on a daily basis to perform their daily tasks. When I joined Crux Consultores I was fortunate to learn about Nebulosa, a very interesting software that involved me almost immediately once I learned that it uses an Artificial Intelligence to analyze suspicious L.A.F.T. ( Money Laundering and Financial Terrorism) cases, but then I asked myself What is behind an Artificial Intelligence?

Since we managed to look up to the sky, this question comes to our mind, are we really alone in the universe? To answer this question, let’s stop for a moment and look our cell phones, from the suit of applications that are installed, for example, Google Maps, this application like many uses artificial intelligence to perform multiple tasks, by storing information of the places we visited, this can be used to analyze and generate suggestions according to the places where we have frequented, like a restaurant to satisfy our appetite. As well as Google Maps, there are many other technological tools that use this branch of computer science, from video games, NASA, or companies in charge of data analysis. And this is where we realize, we have a smart friend accompanying us daily in the everyday of our lives.
But going back to the second question, behind an artificial intelligence there are mathematical, metaheuristic and computational algorithms that make the subject even more interesting.

The first topic focused on Artificial Intelligence (AI) is learning, reinforcement learning takes care of this task, next to this we are going to explore a family of RL algorithms called Q- Learning. It is an area of automatic learning inspired by behavioral psychology, which determines what actions a software agent should choose in a given environment in order to maximize an action while expecting a reward for this fact. To understand it more simply, the idea focuses on the objective of training a dog, when you want the animal to perform an action, for example, sit down, you tell him and once he does it you give him a cookie, but if he performs an action that should not scold him and you give him a negative reward, this way he learns what he should or should´nt do.
To understand how the concept applies to an artificial intelligence we can see the following image:

The labyrinth at the top of the image represents the environment in which we need and want to make a decision, basically the problem we have to face.
The agent, which is in the lower part of the image as a robot is the mind that will navigate in the environment that will have to learn about if an action taken is correct or not, and this is where the positive or negative rewards come in.
Based on the action taken a new state is given and this gives a reinforcement to the agent so that he knows if what he did was right or wrong, in other words it is our reward. This is where our agent will try to get positive rewards to know what results are favorable for what we need to accomplish.

You don’t like math? Or do you think they’re not used in computing these days? Well, thanks to Bellman’s equation we obtain a conceptualized and developed artificial intelligence, but like any mathematical equation, it is created to fulfill a function and solve complex problems in a simpler and more understandable way.

The explanation of this pile of letters and scribbles is simpler and more complex than it appears to be but it explains mathematically the reinforcement learning, giving us a non-deterministic vision that is the purpose of this one, then let’s see what this means based on the previous table. The quality of an action in the current state of having taken an action is equal to the previous quality, plus the learning factor (which is the weight we give to the new information according to the previously evaluated one), multiplied by the reward obtained in the current state when making a decision, plus the discount (defines a punishment based on the time the agent is alive to provide a motivation for the longer the time it takes to solve the problem, speeding up the process), multiplied by the maximum quality obtained in the next state according to the action to be taken minus the quality in the state prior to taking an action.

Interesting and complicated isn’t it, well I invite you to read a little more about it in the links I will leave at the end of the article.

The last missing piece for all this to become a reality, are the artificial neural networks are a family of metaheuristic algorithms used for machine learning which is the main function of artificial intelligence.

This topic can be summarized as a set of parts intertwined to work on
mutually to solve much more complex problems, in other words, to divide the task
complex in multiple simpler tasks. In case of a neural network to each of these
separate components is known as the neuron.

The neuron is the basic unit of minimal processing in a neural network, which is basically a mathematical function, because if math, an internal algorithm that performs a basic function that will give us a result to present a decision that will determine a specific action.

However, a set of neurons is known as a network, neurons are intertwined between layers, that each of these layers are a set of neurons, which will give us an output value as we introduce input values. Taking output values, and introducing them into new neurons that are in new layers, give us a greater view of a network, which as I mentioned at the beginning, is a set of small mathematical algorithms that are intertwined to give us an even more complex solution.

If you are interested in the subject, in another publication I will develop in depth the topic of a neural network and how it works learning this, with Bellman´s equation we saw a brushstroke of this, on a large scale becomes in character recognition, language translation, genetic analysis, voice recognition or fraud prevention as Nebulosa.

Interesting links to extend knowledge
Reinforcement Learning
https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-0-q-
learning-with-tables-and-neural-networks-d195264329d0
Bellman Equation
https://www.rand.org/content/dam/rand/pubs/papers/2008/P550.pdf
Neural Networks
http://proceedings.mlr.press/v15/glorot11a/glorot11a.pdf

josquesada@cruxconsultores.com

Created by: Josué Quesada.

Learn more about Crux: Website · LinkedIn · Facebook· Twitter

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Crux Consultores S.A.

Specialists in development and integration of solutions for finance field. | Especialistas en desarrollo e integración de soluciones para el sector financiero.