Creating an FX price feed
Full stack part 1: Node, Express, Socket.io and Angular
Intro
I have been meaning to do some meaningful spikes using the full stack of Node, express, mongoDB, socket.io and angularJS, which will enable me to test performance, investigate best practise and create a few re usable chunks along the way be it widgets, components, modules etc
I just needed a context and given the performance requirement I decided on finance and specifically the Forex Market.
Part 1:
Will be focused on creating an FX streaming price feed over web sockets using node, express and socket.io and a little angularJS to display the feed.
Getting my data
After a few searches I stumbled across truefx.com a company who provide a price feed of live tick data and for free ( V important requirement).
The price feed API is exposed via a RESTful end point with a variety of responses — I decided to deal with the HTML response.
Requirements so far:
- HTTP server
- Socket server
- Module to poll requests to the RESTful API
- Module to parse HTML response
Checking my data
I’m a firm believer in checking your data before writing any code. Especially if I don’t own the data and more often than not you’ll find requesting the data is not as simple a process as you first thought. Even if this is not the case at the very least you will get more familiar with the api.
This could be achieved by an integration test in certain cases although in this case I’ll use my favourite REST client POSTMAN, which comes as a nifty chrome plug in. Chrome Postman plug in
Note : As a pre requisite to access the data you need to set yourself up with a truefx account.
After going through their api documentation I fire up postman and get going.
Things i discovered …
Point 1 : Need to make an initial request to set up a token that represents your session — this is not a problem as it only needs to happen once and the session persists.
Now I’m ready to fire off my configured request to return the currency pairs I require.
Outcome a nice HTML table returned with my currency pairs.
Point 2: After rapidly firing requests at trueFX I discover the response is an update of the pairs that have changed since the last request.
Setting up an express server
Node and express - I will start by configuring my simple HTTP server.
// modules =================================================var express = require('express');
var app = express();
//---------------------var bodyParser = require('body-parser');
var methodOverride = require('method-override');
var http = require('http');
var io = require('socket.io');// configuration ===========================================var port = process.env.PORT || 8020;
app.use(express.static(__dirname + './../client'));
app.use(require('connect-logger')());
app.use(bodyParser());
app.use(methodOverride('X-HTTP-Method-Override'));//==============================================================// start app
================================================================//:::::::: Create HTTP servervar server = http.createServer(app);
server.listen(port, function(){ console.log('Express server listening on port ' + port);});
FX price socket server
To implement the socket server I will be using the socket.io module.
To parse the HTML I will use the cheerio module defined as.
Tiny, fast, and elegant implementation of core jQuery designed specifically for the server
The request module will be used for interacting with the RESTful api
Simplified HTTP request client.
Additional requirements:
- Update collection of price data with partial updates
- Encapsulate and loosely couple FX price socket server
Node fxprice-srv module creation
In order to create and encapsulate the FX price socket server I will create a node module and enable configuration via the module constructor.
module.exports = function (io,trueFXConfig) { 'use strict'; // ======== Dependencies
var request = require("request");
var cheerio = require('cheerio'); // ======== TRUE FX configuration
var curPairs = trueFXConfig.curPairs;
var trueFXusername = trueFXConfig.userName;
var trueFXpassword = trueFXConfig.password;
var trueFXID = trueFXConfig.trueFXID; // ======== TRUE FX static
var fxGroup = 'fxrates';
var format = 'html';
var priceFeedHost = 'http://webrates.truefx.com/rates/connect.html?'; // ========
var fxServiceYetToBeInitiated = true;
var userNotAuthenticated = true;
var currencyPairs = []; //================================================
// SOCKET SETUP
//================================================ io.on('connection', function (socket) { if(fxServiceYetToBeInitiated){ setInterval(function () {
requestFXprices(socket);
}, 1000)
} socket.on('disconnect', function () { fxServiceYetToBeInitiated = true;
socket.emit('disconnected');
}); });
In the above code I have defined my dependencies, configuration and set up the socket.
The socket is very basic, we have on “connection” handler, which will set up our polling and we have a “disconnect” handler, which will reset our fxServiceYetToBeInitiated flag.
Request FX prices
//================================================
// PRICING API //================================================ function requestFXprices(socket){ if(userNotAuthenticated)authenticateSession(); request(trueFXconfigPath(), function(error, response, body) { if (!error && response.statusCode == 200) { var fxPriceData = updatePrices(body); fxServiceYetToBeInitiated = false; socket.broadcast.emit('fxPriceUpdate', {
payload: fxPriceData
});
}
else if(error){ fxServiceYetToBeInitiated = true
console.error ("There has been en ERROR when requesting prices from : " + trueFXconfigPath());
console.error ("ERROR status code : " + error);
}
})
}
The key element here is that on a successful response we pass the response on to updatePrices() and by using the socket we then broadcast.emit the fxPriceUpdate event with the fxPriceData as a pay load.
broadcast.emit will emit the event to all connected clients.
Updating the price object
function updatePrices(body){ var $;
$ = cheerio.load(body); var priceUpdates = $('table').find('tr').length; if(priceUpdates === 0 )return currencyPairs; //SET - UP : Initiate currency pairs
if(currencyPairs.length === 0){ currencyPairs = $('table tr').map(function() { var $row = $(this);
var symbolVO = createSymbolVO($row); return symbolVO;
}).get(); return currencyPairs; }
............... continued for updating pairs
From the code snippet we can see how cheerio has loaded the HTML response
$ = cheerio.load(body);From here we have used the map method to enable us to get hold of an instance of the individual tr elements in the table.
The $.map() method applies a function to each item in an array or object and maps the results into a new array.
Then converted these individual elements into a symbolVO (value object), which will return us an array of objects representing the currency pairs.
The helper method is outlined below.
//================================================ // HELPER METHODS //================================================ function createSymbolVO(row){ return { symbol: row.find(':nth-child(1)').text(),
timeStamp: row.find(':nth-child(2)').text(),
bidBig: row.find(':nth-child(3)').text(),
bidPoint: row.find(':nth-child(4)').text(),
offerBig: row.find(':nth-child(5)').text(),
offerPoint: row.find(':nth-child(6)').text(),
high: row.find(':nth-child(7)').text(),
low: row.find(':nth-child(8)').text(),
mid: row.find(':nth-child(9)').text() }; }
Instantiating and configuring the socket server
By creating a node module we have decoupled our fxprice-srv from the http server implementation enabling re use later on down the road and by implementing a constructor on the node module we can initialise through the configuration object, which enables the encapsulation of the fx price server logic.
//:::::::: Create fx price servervar trueFXConfig = { userName:'username',
password:'Password',
curPairs : 'EUR/CAD,GBP/USD,EUR/JPY,GBP/JPY,GBP/CAD,EUR/AUD',};io = io.listen(server);require('./sockets/fxprice-srv')(io,trueFXConfig);exports = module.exports = app;
Summary
We now have ……
FX service being polled every 1000 milliseconds
Results being parsed from HTML into an array of objects
The resulting collection then being emited via the socket.io module
….. with the following dependencies
package.json
{ "name": "starter-node-angular",
"main": "server.js", "dependencies": { "express": "~4.0.0",
"body-parser": "~1.0.1",
"method-override": "^2.0.2",
"socket.io": "~0.9.16",
"cheerio": "^0.17.0",
"request": "^2.36.0",
"connect-logger": "0.0.1" }}
The client
The purpose of the client is to display the data in the most simplest of forms and to use angular. Smartening this up will be the part of the next post.
angular.module("fxPriceFeed",['btford.socket-io']);(function() { 'use strict'; function AppCtrl ($scope,fxPriceService){
$scope.$on('socket:fxPriceUpdate', function(event, data){
$scope.rates = data.payload;
});
} function fxPriceService(socketFactory){ var socket = socketFactory();
socket.forward(['fxPriceUpdate','disconnected']);
return socket;
} angular
.module('fxPriceFeed')
.controller('AppCtrl', AppCtrl)
.service('fxPriceService', fxPriceService);
})()
The key point here is that when using the client side socket.io library, once you have got an instance of the socket factory you have to define the events you want to listen to via the forward method.
socket.forward(['fxPriceUpdate','disconnected']);The subsequent events will then be added to the angular life cycle and you can simply use the angular event handling methods.
$scope.$on('socket:fxPriceUpdate', function(event, data) { $scope.rates = data.payload; });
The resultant UI looks like … as i said not pretty
The example is available to download from GitHub and you will also find instruction for how to set up and run.
UPDATE : 16/8/2014
The implementation has changed a little and the service is now open source and available for download on NPM gch-truefx-pricefeed.
