Create an online IDE with Angular 6 + NodeJS ~ Part #3

Part #3 — Buildup your API’s routes

What you will do on this part

On this part of the project, you will build your API’s routes, configure a connection to external API services, you’ll do this while implementing some conceptual features of clean coding, through refactoring process.

Contents Table — Part #3

• Say Hello to Jdoodle
  ◢ Introduction to Jdoodle.
  ◢ Subscribing to Jdoodle services.
  ◢ Investigating Jdoodle API .
• Visualize your API functionality.
• Talk No More — Implement !
  ◢ Implementing the API routes.
  ◢ Start Refactoring.

If you haven’t checked out the previous posts in the series, here are the links:
Part #1 — Building your IDE client-side with Angular and Bootstrap
Part #2 — Creating your server-side with Node.JS, Express and Typescript

Say Hello to Jdoodle 🖐

Introduction to Jdoodle

Now we can continue to the rest of the project, the next thing will be to connect and configure Jdoodle Compiler services in to your server.

Jdoodle Compiler is an online API service to compile and execute Programs online via APIs, it supports Java, C/C++, PHP, Perl, Python, Ruby and many more languages.

Jdoodle Compiler API was designed to be used with “server to server” calls, after subscribing to Jdoodle’s services, you will receive your private API credentials, with those you will be able to send requests, from your server, to compile or execute a code received from your client.

Client to Jdoodle-endpoint request process

Subscribing to Jdoodle services

Jdoodle API services are pretty easy to use, and more importantly, they got a free plan with 200 API calls available to you per day, that’s enough for our needs.
Enter Jdoodle home page, register, then enter to the API section and subscribe to their Free Plan.

Jdoodle signup & subscribing

You will immediately receive your subscription details, Client Id & Client Secret, those are the API credentials you will assign to every request you will send to Jdoodle API services.
for now, paste your credentials in some text file (optional — you can easily fetch them from your account page), and let’s see how it works, what we need to send and what will be returned.

Investigating Jdoodle API 🔍

A great way to investigate API behavior is with Postman (if you don’t have it installed on your machine, now it’s the time).
Postman interface isn’t so complicated to work with, for our simple usage, this tutorial should cut it;

quick tutorial to launch a request

First of all , Jdoodle docs got all the info you’ll need

Huge part of working with APIs or libraries is to know how to extract the needed information from their docs, therefore I suggest you read it and find what you need.

Basically what you need is the following :

1. Jdoodle endpoint -
   https://api.jdoodle.com/execute, the URL you’ll send your requests to.
2. Parameters for an API call -
   clientId : Client ID for your subscription.
   clientSecret : Client Secret for your subscription.
   script : program to compile and execute.
   stdin : StdIn.
   language : language of the script.
   versionIndex version of the language to be used.
3. Supported languages and their versions are presented in the docs (in a big table), take a look.

Let’s send a simple request for testing.
Enter to Postman, set the method to “Post” and enter the request body parameters : (if you scroll a bit 👇 you could see a live example).

{
   "script" : "console.log('hello')",
   "language": "nodejs",
   "versionIndex": "1",
   "clientId": MY_CLIENT_ID,
   "clientSecret": MY_CLIENT_SECRET
}

and sent it, the response should be something similar to :

{
   "output": "hello\n",
   "statusCode": 200,
   "memory": "28332",
   "cpuTime": "0.07"
}

the output property value is the “stdout” — what you’ll see on the program console in your regular IDE, the statusCode is the response HTTP status code (we discussed earlier).

Try to execute a bad request, with a missing parameter, or an invalid code, and see what is returned, examine the response body structure.

example of postman POST request to Jdoodle API

Visualize your plan before Implementing !

After you’ve understood the behavior of Jdoodle endpoint, (requests and responses structure), it’s time to implement it into your server.

Before start the coding process, you must see the big picture, ask your self, from your client-side point-of-view, what are the expectations from your server? to what requests it will need to respond to?
OK, so your client might request for :

• the supported code languages your server will work with, (this can be stored as static data on the client, but this isn’t the recommended approach — with that you’ll create two sources to synchronize and updated for any changes).
• executing a code piece the end-user have entered.

This was a relatively quick planning 😌.
In most projects, this part will take you much more time (as it should — the more you plan the less you’ll refactor).

Talk No More — Implement !

Your server API will look like :

GET : /langs   /* send supported languages */
POST : /run    /* run received code and send result */

As you can see, the server API isn’t a complex one and can be handled in a single routes class, so you’ll start with this :

AppRoutes class with empty routes

Implementing the API routes

Let’s start with the easy one, for GET : /langs route you will need to store your supported languages, at the first look, you might think an array would suffice, if languages names was all you need to store, that was the case, but Jdoodle system works with — language id, language version, and version index, so you need some data structure for storing it in an efficient yet understandable way, my suggestion is a language table/map;

Language Table :
   key : languageId        // id of the language
   value : {               // array of that language versions
      languageId, 
      languageName, 
      languageVersion, 
      versionIndex 
   }[]

I chose several languages that I prefer,
(I’ve place the file on server/util/languages/) so ;

Stored languages

As you continue developing you might decide to add more languages to that table, and accidentally miss a property or use a different type, to avoid this, type-define your wild objects.

In this route you’ll simply import languagesTable object, and send it ;

.get('/langs', (req, res) => {
   console.log('GET: \'/langs\'');
   return res.status(200).send({langs: languagesTable});
})

Keep in mind —

• The default response status is 200, for later testing you state the status.
• What you pass to send() will become the response body, as a rule, always wrap any result you send in an object — so the result won’t become the response body, but a property of the body, this will give you the freedom, in the future, to add more data to the body with less heavy changing
• Both the lines, return res.status(200).send({langs}); and res.status(200).send({langs}); are acting the same, the return at the start of the line will stop all code lines after, from being executed.
  — hope I’m not insulting your coding skills —
  Later you’ll see that there will be multiple cases on a route block, where you will call res.send(), the thing is, res.send() can be executed once (any call after the first will throw an error), adding return before will save you a lot of headaches.

The next route will be POST : /run,
First install the dependencies :

npm i lodash @types/lodash request @types/request --save

lodash is a modern js/ts utility library — the undisputed library in it’s field.
request is an HTTP client library, I found the simplest to use (especially with typescript), you’ll use it to send requests from your server to Jdoodle endpoint.

Add the subscription details (Client Id & Client Secret) to the config.json ;

{
   "dev" : {
      "PORT": 3000,
      "JDOODLE_CLIENT_ID" : "...",
      "JDOODLE_CLIENT_SECRET": "..."
   },
   "test" : {
      "PORT": 3000,
      "JDOODLE_CLIENT_ID" : "...",
      "JDOODLE_CLIENT_SECRET": "..."
   }
}

I’ve created two types to help define better wild objects ;

type ClientRunRequestBody = {   // your client request scheme
   lang: string,
   version: string,
   program: string,
}
type JdoodleResponseBody = {   // Jdoodle response body scheme
   output: string,
   statusCode: number,
   memory: string,
   cpuTime: string
}

On this route you will simply send a request to Jdoodle endpoint, listen to any responses and pass them to your client, we will break it down and later do some refactoring ;

line 12 ~ _.pick() is a nice helper method (see the doc), with that you are fetching the fields “lang”, “version” and “program” from the request body and assigning them to body variable.

lines 17 –19 ~ from languagesTable object you’re fetching an array of languages associated with lang (language id) on the request body, finding the language with the right version on that array, and then the language version index.

lines 21 –27 ~ constructing the request body to send to Jdoodle endpoint.

lines 29 –45 ~ sending a post request, using request object (reference to Request library), to Jdoodle endpoint, then with on() listening to events (just like when you using EventEmitter).
On error event ⇒ sending 400 status with the error object.
On data event ⇒ the data parameter is of type Buffer of raw data, so you parse the data to a regular response object, then check if in the response there's an “error” property, if so, an error occurred in Jdoodle system while processing your request, then you’re sending a 400 status with the error object, else the request was successful, so you’re sending a 200 status with the response.

Start Refactoring

Side-Note — your API route listeners are the core of your server code, the thing is, they can easily become complicated, clumsy and unreadable.
Your goal while coding them (besides making things work correctly) is to code in the most readable way.
Developing an API is a work in process, you (your team) need to understand every bit of it, in a week from the moment its been written.
Adding more code, methods and classes to your existing “working” code to improve readability — WORTH IT☝️.

The first change will be to add validation to the request body parameters;

place the validatePostRun method calling under _.pick() (where the TODO comment) ;

if(!this.validatePostRun(body)) {
   console.log('invalid body parameters');
   return res.status(400).send('invalid body parameters');
}

You can notice that there’s some repetition of code handling the languages table (finding in the languages table an entry, then loop over the versions array in the entry — the purpose is not so clear in first sight), refactor it to a class dedicated to handling the languages table can be nice.

With LanguagesManager class we can continue refactoring ;
on .post(‘/run’, ... ), the lines 17-19 can be replaced with one line :

const index = LanguagesManager.getLanguageVersionIndex(body.lang, body.version) 

on the validtionPostRun method you can replace the lines 9 -10 with :

&& LanguagesManager.isLangSupported(reqBody.lang, reqBody.version);

Bare with me, you are on the last change, and this one can be a useful trick to know in the future.

In the data event listener (lines 37–45) you’re actually listening to two events, when Jdoodle process has succeeded and when it has failed.
On top of that you are receiving an unparsed data as a parameter,

I’ve figure all that by trial and error, in a week I’ll probably won’t remember any of this, that is a great example way readability is so important.

It’s fair to say that the event name “data” not implying its context or content,
let’s fix all that!

The goal is to separate the cases (Jdoodle process succeeded/failed) to two individual events, and on each one receiving a parsed parameter,
you will do so by wrapping “data” event in a custom event.

Create a new class named “RequestHandler” (I’ve placed on server/utils) ;

request wrapping in it, an EventEmitter object, and with it, piping some EventEmitter functionality with similar methods (E.g. on, off, emit).
In the EventEmitter docs, you can see that by providing (to on method) a listener function as a normal (ES5) function (not an ES6 arrow function), the keyword this will be bound to the event-emitter object (the one that’s emitting the current event).
That little detail gives you the ability to emit events from inside an emitted event listener (piping events), and that is exactly what you’re doing on the data event listener.

In the refactored data event listener, first, you’re parsing the data buffer to an object, then if it’s an error, you’re emitting a new custom event called “jdoodle-error” with the parsed error, else, if the parsed object is a success response, you’re emitting another custom event called “jdoodle-success” with that parsed response.

Now, on .post(‘/run’, .. ) where you called request.post() and listen to the events, you are calling :

RequestHandler.postRunRequest(body.lang, index, body.program)
.on('error', (error) => {
   console.log(postRunRequest on error', error);
   return res.status(400).send(error);
})
.on('jdoodle-error', (error) => {
   console.log('postRunRequest on jdoodle-error', error);
   return res.status(400).send(error);
})
.on('jdoodle-success', (result: JdoodleResponseBody) => {
   return res.status(200).send({runResult: result});
})

After refactoring POST : /run route, app.routes.ts will look like :

Let’s review

1. You met Jdoodle services and subscribed to their Free Plan.
2. You investigated Jdoodle API, went over their docs, and send request with Postman.
3. Understand what are the requirements from your API.
4. You implemented your API routes, GET: /langs then POST: /run.
5. Finally, you Refactored POST: /run route to prevent code repetitions, and improve readability.

You’re done building your API’s routes for now,
On the next part — Integrate logger service into your app, build your testing environment and test suites for your API routes.