SUPERALGOS DATA MINING

Indicator Building In Superalgos: The Ultimate Guide to Tame the UI

Understanding the UI, JavaScript coding, and mathematics.

Photo by Jamar Penny on Unsplash

Prior conditions to understand this article are of course to have completed all the basic tutorials included in Superalgos and to be familiar with the different concepts of the user interface (UI).

Building an indicator in Superalgos is seen by many as a very complicated process. The three main psychological barriers are: understanding the UI, JavaScript coding, mathematics. Well, with a bit of practice, a good guide and elbow grease everybody can build an indicator.

Let us demystify the process! First of all, the UI is built in a way everything is logically chained and understanding what is the role of a node and the relationships and references between nodes, are the only requirements here.

JavaScript is an interpreted programing language, i.e. except for some specific situations, the code is executed in the exact order it has been written. As long as you speak English and have basic knowledge on logical processing you can use JavaScript to code an indicator. In terms of math expertise, well, there are a whole bunch of basic indicators that does not require more than what a 7th grader could achieve.

So, do you feel more confident now? Then brace yourself and take a trip with me down the rabbit hole!

Data Mine Creation

A New Data Mine

The Data Mine node is where indicators and plotters are defined.

Superalgos is provided with a set of existing data mines developed by seasoned developers. The most efficient way to start a new data mine is to reuse an existing one. Since it would have been reviewed by the Data Mining Team and built and maintained by an experienced developer following good practices, it is a good starting point.

Find the Data Mines plugin folder at…

//Superalgos/Projects/Datamining/Plugins/Data Mines

… and drag-n-drop a data mine to your workspace:

Mark Your Territory

Hover your mouse on your data mine and left click on the title to rename it :

You can now proceed to some clean-up and remove the unwanted nodes, plotters and indicator bots.

Change the Codename of the data mine at the Configure menu:

It will be the name of the folder where the mined data will be stored. Choose something wise and avoid characters that would not be handled by your operating system.

Hit Ctrl + Shift + S in order to save your workspace. Since the data mine have been manually added to the workspace, it will be saved inside.

A Bit of Cleaning and Tiding

After the first steps, you should have ended with a data mine with only the remaining nodes of interest :

Rename both your plotter and indicator bot. To keep things clean, use the same name for both.

You can now proceed to the Codename changes

As per the data mine codename, it will be the name of the folder at the data storage, avoid spaces and choose simple names without special characters or others funny things (spoiler: or it will not work!)

When reusing existing nodes, it might be some references are tied to the original data mine, cleaning a little bit will save you some long and cold sleepless winter nights. Break all the irrelevant references

Setting up the Process Definition

The process definition node is where the indicator’s calculation process is organized. It will tell the bot when to start/end, what are the outputs to store and where to find required data like the candles.

The Output dataset is referenced to the Dataset Definition of the multi-time frame XXXX at the product definition. At least one Status dependency is self-referred to the Status report of the process definition. A reference between a status dependency and the status report of the candles is required. A reference between a data dependency and the Dataset definition of the Candles is required.

Each status dependency needs to be configured as per required by its usage:

The process definition for Market and Daily time frames does not necessitate the same references.

The Daily require both market starting and ending point whereas the Market will need only the Market ending point :

Setting up the Record Definition

The record definition literally contains the Objects that will create stored variables.

  • Clean existing Record Property except for Begin and End that will be the stamp for Begin of Candle and End of Candle (time reference for the variables)
  • Establish the list of variables you want to record
  • Create a set of Record Property + Record formula for each variable you need to store
  • For each record property, set the « codename » parameter with a name in order to remind the name of your variable (choose something simple, space and special chars are prohibited, it will be the name you will use at the Plotter)

Tip: this is where you define the data to be stored on your SA machine drive; keep it simple and only with what it is strictly necessary to minimize disk space and CPU usage

Edit the record formula to declare the recording of each variables : « variable.your_variable» your_variable will be the name of the variables at the Procedure Loop code definition, it needs to respect JavaScript format.

Tip : now a bit of work has been done, it should be a good time to have another level of backup ; use the clone node function and save your datamine file at a safe place. In case of a disaster, you will just have to drag’n drop it to revive your data mine

Write Down the Indicator’s Code: the Procedure Loop Node

The data building procedure node is situated under the Product definition node of the indicator bot. the procedure loop contains the main code to be executed at each iteration. The Procedure initialization contains the variable initialization: you will declare the array typed variables at this place.

As written in the title, the code used in the procedures is JavasScript. Plenty of education material can be found on the web; so, this tutorial will not cover JavaScript education.

The rabbit hole may sometime be dark and damp so here are 3 tools you may find useful to avoid traps.

Always code your indicators outside Superalgos : the Procedure JS Code node is handful but difficult to read. A good tool to type your code is Notepad++: it is free, lightweighted and can recognize a large variety of coding languages; if you work on a linux/debian based plateform, Geany is also a super great tool.

Sometime, a simple synthax checking is a good start before trying to execute your indicator: Esprima offer a free online version of a synthax checker: https://esprima.org/demo/validate.html. When it comes to debug, VS-Code is nearly the best tool you can find.

Finally, testing the math of your indicator outside Superalgos in a first time will save you a lot of debugging time. There is a lot of very good datamining tools available but if you can spend time to discover a new software, Knime is the best data mining and Machine Learning tool you can find at 0 cost: https://www.knime.com/knime-analytics-platform. Plus : it provides free online courses on data mining.

Write Down the Indicator’s Code: the Procedure Loop Node

0- THE INITIALIZATION RULE: the mother of all rules! The charting space is really annoyed by non-numerical values. Be sure to avoid NaN, undefined, infinity in the data.json files. In the best case, the chart will not work, but it can go up to a crash of the browser. Not to mention also the devastating effect of such a value in a live trading session. Please, manage exceptions with hard coded procedures at initialization and during execution.

1- Comment your code: there is nothing more annoying than a huge block of code without any comments to indicate what it is done. It will help you and other developers to keep track of the meaning of the code, will make it easier to maintain and to review

2- Avoid redundant or useless lines: Javascript is an interpreted language, each line is executed; save a CPU, degrease the code! SA is meant to run on feather light hardware with low cost of ownership, if there is too much to execute, your indicator will stress the system and may not be calculated in time or compromise other processes

3- Keep it clean: when developing it happens we take different directions and want to keep track of what we have done. Most of us simply comment the lines. When you release your work, please, clean before PR

4- Do not store useless variables: you may have found interesting to follow the evolution of sub variables at the record definition but it is costly in terms of disk space. If you intend to deliver your indicator to the community, clean your record definition

5- Do not be afraid of data mining specialists: they don’t bite, at least if they are fed, and it will save you a lot of time and a lot of frustrating nights. Also, specialists from the Community Mod team are committed to help you so when we ask for a clone of your node it’s for your own good and it will be easier for us to find a bug with something running on a known configuration with a good overview!

6- Accept your own learning curve and that you will not have a perfect result at the first try. Also, since people with little knowledge on programming will have no choice but to trust the content of the indicators, it may happen your contribution is postponed due to further needs of investigation on the reliability of the code.

7- Submit to the community first and demonstrate: snapshots of the results of your work with some statistics on the stability are very appreciated by the community and will grant you trust and exposure.

Plug ‘n Pray: Pull the Trigger, Find Errors, Debug, Repeat to Exhaustion…

Settle and indicator Bot Instance for your brand-new indicator Run the data task and then the Candles tasks This first step should show you all errors that are not at the code.

If your indicator now processes funny values (NaN, all values to 0, undefined…) then you may have some bad stuffs at your code. Use the debugging procedure from SA doc using your preferred IDE.

Most of the non-coding errors are generally due to a bad or missing reference. Never try to correct the code directly in the JavaScript node, prefer your code editor, then copy/paste at the node. While debugging, keep in mind some of your variables may needs several iteration to be defined (if you run a SMA 200, you will need 200 periods… ). If you feel the non-coding error does not come from a reference, then you are probably wrong, re-check the references, unless you have broken something else due to wild experimentation. Same for a coding error… JavaScript code are only executed according to what you have written, there is no black magic.

Actually, you should not have NaN or undefined values… unless you have not respected the Mother of all Rules. Having 0s everywhere or NaN or undefined… means there is a problem.

Once you get things working, time has come to design your plotter bot.

Plotter Design

The Plotter is the node where the graphic representation of your dataset is defined and produced. Since the Plotter of your indicator has been reclaimed from another data mine, the 1st step is to rename it and set the codename in the parameters.

Unwrap the Plotter node and proceed to the renaming of The Plotter Module and the Plotter Panel and eventually the Shapes node; don’t forget to change the codename of each node at the Configuration menu.

Unwrap the Plotter Module and check there’s a reference between the Product Definition and the Plotter Module. Each plotter module at the plotter will define and produce a chart for each product definition referred.

Each plotter module contains a Plotter Panel (display of the indicator values in a panel), and a Shapes node (plot geometry and colors)

Plotter panel gathers Panel Data. A panel data defines the indicators value to display. A Panel Data contains a Data Formula to indicate the source of the data and a Text Style node where you can define the format of the displayed value.

Set the number of decimals do display in the Configuration menu of the Panel data.

Type the formula to point at your recorded value: record.<record_property_codename>

Shapes node contains the definition of the Chart Points (x and y coordinates) and the Polygos (geometry and colors).

Chart Points node contains Point nodes defined by Point Formula nodes. To plot a simple curve, 2 points are needed to define the segment between the begin and the end of a candle.

The begin point formula refers to record.begin and must point at the record.previous.<your_record_property>

The begin point formula refers to record.end and must point at the record.<your_record_property>

The Polygon node contains the definition of the geometry of the plotted indicator. It uses Vertices to define the points and Polygon Border and Body for the border and the filling of the plotted indicator.

The chart is plotted according to the clockwise rotation around the Polygon node following the vertices.

The polygon vertices have to be referred to the Plot Nodes at the Chart Points node according to the clockwise direction around the Polygon. Begin and End will have to be addressed. Add one Polygon vertex per each and build the reference to each Point.

To build a simple curve add the Polygon Border node and define the curve style at the Style Node.

The modification in the Plotter node will not appear at the chart until you refresh the UI. Save your workspace and hit the refresh key of your browser.

You have reached the last step of the indicator building procedure. Save your workspace and clone your data mine and store the file in a safe place. You can now add the layer for your indicator at the Time Machine or use the Add product function.

If you enjoyed this article and want to participate in the most promising open-source social trading platform… come and join us !

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store