There has been a lot of discussion about the fairness of pseudo-random number generator (PRNG) algorithms in gambling and about fairness in general. While we are confident in the fairness of our algorithms, some providers have gone a step further by providing a way for players to verify their fairness using simple cryptographic calculations. How does this process work?

Provably Fair uses a simple process where the server announces a hashed seed for the upcoming round, which is publicly available to all players. The server also assigns a default player seed that the player can choose to change at any time before the round starts. The player creates a bet, and before the round is generated, the server combines the server seed and the player seed to create a game hash that contains the result. After the round is resolved, the server reveals the unhashed server seed, which is used to obtain the result from the game hash.

The same process is used for round-based games in which multiple players are playing the same round. In this case, the player seeds are typically taken from the first three players who make a bet. If there are less than three players in the round, their seeds are used instead.

Here is the example from one crash cash game:

• Server announces hashed server seed for the next round:
  64a436bfafb564dc9831530c975c73ca5d43cb29a34fc3bc93e0b1dd4d69274e
• Player sets seed to some value:
  x8QBX5DPsyIHf066TRht
• Player creates a bet and starts round
• Algorithm uses combined server and player seed to generate result and announces game hash at the end of a round:
  50aa38ce0e390e4f7ece4138878e46132e99fbe3887fca2a6c3f6c43ca2dc5611a15e5dc5de7b9ea9db79c9bfc48763e8d6df138ff24c377f4704c23d41bf911
• Player can use online tool to check game hash and view result (in this case result is decimal number):
  11.97

Generating server seed

To generate server seed we need to use SHA256 algorithm.

const crypto = require('crypto');

function generateServerSeed() {
  const seed = crypto.randomBytes(32).toString('hex');
  const seedHashRaw = crypto.createHash('sha256').update(seed);
  const seedHash = seedHashRaw.digest('hex');

  return {
    seed,
    seedHash,
  };
}

Server seed hash needs to be publicly available, usually in game rules or special provably fair section of the game settings. It can also be stored on blockchain. Do not show server seed, only server seed hash. New server seed is generated before each round and announced to players.

Generating result

To generate result we need to use server seed combined with player seed that player provided during bet placement. For example, if we wish to generate result for Roulette game we can use this example:

function generateResult(serverSeed, playerSeed) {
  const gameSeed = `${serverSeed}${playerSeed}`;
  const gameHash = crypto.createHash('sha512').update(gameSeed).digest('hex');
  const gameHashHmac = crypto.createHmac('sha256', gameHash).update(gameSeed).digest('hex');

  const resultNumber = parseInt(gameHashHmac.substring(0, 13), 16);
  const result = resultNumber % 37;

  return {
    result,
    serverSeed,
    playerSeed,
    gameHash,
    gameHashHmac,
  };
}

The generation of the result in Roulette is determined by a game logic. To generate a random number between 0 and 36, we use the modulo operator. The result number, which is used to determine the outcome of the game, is derived from a game hash created using a combination of the server seed and the player seed, and is then converted to an integer.

For other games, like crash games, we would use different formula to get result. Here is the example:

function generateResult(serverSeed, playerSeed) {
  const gameHashSeed = `${serverSeed}${playerSeed}`;
  const gameHash = crypto.createHash('sha512').update(gameHashSeed).digest('hex');
  const gameHashHmac = crypto.createHmac('sha256', gameHash).update(gameHashSeed).digest('hex');
  
  const resultMaximum = 2 ** 52;
  const resultNumber = parseInt(gameHashHmac.substring(0, 13), 16);
  const resultRandom = resultNumber % 33 === 0 ? 100 : ((100 * resultMaximum) - resultNumber) / (resultMaximum - resultNumber);
  const result = Math.floor(resultRandom) / 100;

  return {
    result,
    serverSeed,
    playerSeed,
    gameHash,
    gameHashHmac,
  };
}

In this example, the result is a decimal number, unlike in Roulette where the result is an integer.

Validating

Upon the completion of a round, the player receives the game hash. This hash can be used to verify the validity of the seed that was made public before the start of the round. To confirm the game hash, a validation function can be employed:

function isValidHash(serverSeed, playerSeed, gameHash) {
  const gameHashSeed = `${serverSeed}${playerSeed}`;
  const gameHashCheck = crypto.createHash('sha512').update(gameHashSeed).digest('hex');

  return gameHash === gameHashCheck;
}

A player can also utilise the same process to determine the result from the game hash, thereby validating the entire process. It is uncertain whether game providers make public the logic for determining the result from the game hash. From my research, I have not been able to locate this information on official websites. Perhaps this could be something that could be added to each game, allowing players to independently verify this part of the process also.

Conclusion

The implementation of a provably fair algorithm is certainly noteworthy from a technical and certification standpoint, however, I am uncertain about its actual utilisation by players. This approach requires a certain level of technical understanding which the typical player may lack. Furthermore, for games like slots, the algorithm can be even more complicated as the outcome of a spin is not a singular number but rather a matrix of numbers determining the positioning of symbols on the reels. Even if a player is able to use the formula to confirm the result from the game hash, they would still need to check its validity against the paytable of the slot game, which can be quite complex.

Provably Fair — How do I apply it? was originally published in NSoft on Medium, where people are continuing the conversation by highlighting and responding to this story.

The IT industry has been hit hard by the looming recession, with many companies being forced to freeze hiring or lay off employees in order to stay afloat. This is a problem that has been building for some time, as the industry has been plagued by over-hiring in recent years.

Prior to the pandemic and especially after the first wave of the alpha variant, the IT industry was booming. Companies were scrambling to hire as many workers as possible in order to keep up with the rapidly-evolving technology landscape. This led to a glut of IT workers, with many companies offering high salaries and generous benefits in order to attract the best and brightest, and also all the others.

However, the recession has brought the IT industry to a grinding halt. With businesses forced to shut down or operate at reduced capacity, the demand for IT services has plummeted. This has left many companies, especially the Big Tech, with a surplus of employees, forcing them to make difficult decisions about their staffing levels.

One of the biggest problems facing the IT industry is the abundance of inexperienced workers. Many companies were so eager to hire during the boom that they were willing to take on workers with little or no experience in the field. This has left these companies with a workforce that is ill-equipped to handle the challenges of the current economic climate.

In order to weather the storm, many IT companies are being forced to freeze hiring or lay off employees. This is a painful decision, but one that many companies feel is necessary in order to stay afloat. It remains to be seen how the industry will recover, but it is clear that difficult times lie ahead.

One shining bright light is the final breakthrough of AI services. The field of artificial intelligence has made great strides in recent years, and two new developments in particular are worth noting. The first is ChatGPT, a state-of-the-art language model that is capable of generating human-like responses to text-based queries. The second is Midjourney, a tool that uses machine learning to generate amazing images. Overall, the breakthroughs of ChatGPT and Midjourney are exciting developments in the field of AI, and have the potential to greatly improve a wide range of industries. It will be interesting to see how these tools are used in the future, and what other innovations the field has in store.

Fortunately, NSoft has been able to avoid bloat in its workforce, mostly because our middle management wished to maintain lean and efficient development teams. Our Games department has started the year with 39 employees and ended with 44. We kept our focus on improving stability of our virtual games, as well as adding additional features to our lottery and casino products. We are currently maintaining and developing over 30 products, from round based draw games like Lucky Six to casino slots like Temple of Horus.

The coming year will be a challenging one for us, as we will need to not only add new features to our existing products, but also research the potential use of AI in our products. This will require a lot of hard work and dedication, but we are confident that we are up to the task. We are excited about the opportunities that AI presents, and are looking forward to exploring its potential in our products.

In addition to AI, there are many other new features being developed for casino games. These include advanced graphics and animations, new game mechanics, and innovative bonus features. All of these innovations are designed to enhance the gameplay experience and make casino games even more enjoyable for players.

Overall, the world of casino game development is constantly evolving, with new and exciting features being developed all the time. This is great news for players, who can look forward to an ever-improving gaming experience as the industry continues to innovate.

Note: This blog post has been generated with help of the ChatGPT.

The Short Recap was originally published in NSoft on Medium, where people are continuing the conversation by highlighting and responding to this story.

We see more and more companies using JS as their tech stack for both backend and frontend. It makes sense because the development cycle reduces in time, cost, and increases in efficiency, and gives a unique stack for different parts of the system.

Today I’ll show you how you to build a small web app using Node easily. JS enhanced with TypeScript, PostgreSQL as our database, and how to dockerize it.

This article starts with the basic setup of our Node.JS app, Express server, and PostgreSQL configuration. The Docker part is described at the end of the story, so if you want to see the Docker setup for this app only, scroll to the bottom.

Creating a Node.JS application

I assume you have installed Node.JS before. Creating every Node.JS application starts with a simple command npm init If you add -y , it will fill all input for you and create a package.json file necessary for managing our application dependencies required for it to run.

Since we said we’re going to use TypeScript, we have to set up our project to work with TypeScript right away.

Next, we’ll install TypeScript and tslint in our project by typing the following:

npm install typescript tslint --save-dev

This command will install TypeScript in our dev dependencies. After we have installed TypeScript, we’ll edit our package.json file and add tsc command for accessing TypeScript commands. We’ll be using this command for starting and bundling our application.

After we’ve installed these packages, we’ll run the following command to initialize our tsconfig.json file where compiler options for our project are stored.

tsc --init

Since we’ll be using Express, it’s important to install a package that helps TypeScript identify express types by typing:

npm i @types/express @types/node --save-dev

This way, TypeScript will be able to recognize Express classes and global Node types. For example, after you install the types package, you’ll be able to import Request and Response types from Express directly.

If we initialize tsconfig.json with the first command, we’ll get a file like this:

This is our tsconfig.json file with plenty of compiler options you can customize.

We’ll replace it with the following:

{
 "compilerOptions": {
 "module": "commonjs",
 "esModuleInterop": true,
 "target": "es6",
 "moduleResolution": "node",
 "sourceMap": true,
 "outDir": "dist/src"
},

 "lib": ["es2015"]
}

We are mostly interested in our outDir property where we specify the output directory for our transpiled TS code into JS.

Creating and starting our server using TypeScript

It is always a good idea to separate your scripts for defining your server and start your application using that server configuration.

For now, our project structure looks like this:

I’ve created dbconfig directory with database configuration that we’ll use when initializing thePostgres connection.

In the server.ts file lies our server configuration looking like this:

We’ll explain the configuration as we go. Firstly, we initialize our Express application, move to server specific configurations such as configuring body-parser to handle our incoming data. In the end, there is a public method we’ll use to start our server in our app.ts file. We also set up our routing to use separated router configuration for getting all todos.

Here we call our “start” method and log in case of success or failure.

Postgres configuration

Pooling is a process of creating connections and caching them to reuse them. There is no resource waste as it would be if you create a new connection every time.

Below is the explanation of the mentioned configuration properties :

I am using pgAdmin4 for accessing my database cluster and managing my databases.

TodosController

We’ll create a TodosController and route for fetching all todos from the database.

A controller would look like this:

• Importing pool from our database configuration class
• Initializing new connection and creating an instance of pg client
• Sending our raw query to our client’s query method — async
• Releasing our connection —THIS IS IMPORTANT!
• Returning fetched data

Always release your connection when using the pool. That way, the client will return to the pool of available connections.

Todos router

We’ll set up our router in a separate file stored in src/routers and give it a route to the GET method defined upper in our TodosController.

Now, all we have to do is register this router in our server.ts where we set up our server.

We import our todos router and we say to our Express server that anytime someone hits “/todos” in the URL, pass the instance todosRouter to handle all the requests with that route.

Project structure

So far, we have the following project’s structure:

Building our application

Now that we have all set up, we can test our endpoint for fetching all todos from the database.

First, we’ll run the following command from our terminal

npm run build

Using this command, we will look up our tsconfig.json and see the output dir we specified so that it knows the output directory for our transpiled files.

Make sure your main in the package.json points to the same directory as the output dir you specified for the TypeScript since it will be our main point of entry for our application.

"main": "dist/src/app.js",

Our scripts section in package.json should be like this:

"scripts": {

"build": "tsc",

"start": "tsc && node dist/app.js",

"test": "echo \"Error: no test specified\" && exit 1"

},

Now, if you check your directories tree, you’ll see a new directory dist just as we specified in the tsconfig.json.

Testing our application

To start your application type and check your terminal:

npm start

We have it up and running, and since we previously set up our Express server, database configuration, and router, let’s test our endpoint for todos.

Now, if we go to the browser and visit:

http://localhost:4000/todos

we should get a result from our database

Such a piece of good advice!

Dockerizing our application

Since containers are so popular and almost a must when you’re building, testing, and deploying your applications, we’re gonna make our application run inside a Docker container.

To achieve that, we need to build an image from our application. Images are basically our packed applications containing everything for the application to run.

Containers are running instances of our images.

You can read more about containers here.

Docker compose

We’ll create a file called docker-compose.yml where we can setup configuration for all of the services our application will be using.

We need a service for our web application running in Node.JS and our database service, which is PostgreSQL.

We use docker-compose to run multiple containers for our application.

Our docker-compose file would look like this:

If you take a look at our Postgres container in our docker-compose configuration, you can see that we are using a 10.4 Postgres image to build our container, expose the “5432” port on our local machine and map it to the container’s 5432 port. Meaning, if I want to access my Postgres instance running inside my container, I would use localhost:5432 port along with the defined username and password.

We also run our pgAdmin on port 80, which is our database management service where we can access our Postgres databases, create clusters, and do all kinds of operations.

Initializing our database schema on the container startup

It would be great to initialize the database with schema, so we don’t have to manually create it.

- ./src/migrations/dbinit.sql:/docker-entrypoint-initdb.d/dbinit.sql

This piece of code in our docker-compose file will trigger our dbinit.sql file from our project and use it to execute whatever SQL we wrote inside of it.

Dockerfile

Our Dockerfile would look like this:

What this will do is create an image of our application with our virtual directories structure to hold everything that we already have on our machine.

We list which version of Node do we want for our image created our working directory, then copied everything from our src folder to our virtual one.

Here we used -alpine image. Alpine comes from Linux and represents a minified version of Linux that has enough resources to run your applications. It is beneficial when creating Docker images because the output size will be a lot smaller.

The container might be ready but our database might not!!!

What is that part with that URL? It looks like we’re also downloading and running something when building our image, some kind of a waiting process.

Yes, your container may be ready for communication, but the process you’re running inside of it may not. In this case, it’s our Postgres database. It takes a bit more time for it to start than our application. Hence, we are going to download a shell script that will delay our database connection call until it is ready to accept connections.

Our final project structure

Running our application services with Docker

Now when we have our Docker configurations all ready, we can set them up and running in a matter of seconds.
We can achieve this by typing the following command in the root of our project where our Docker configuration files are:

docker-compose up --build -d

The -d says to run the container in the background, and the — build builds the image before starting containers. It is important because you often want to have access to the command prompt while the container is running.

Now, we should have our app image built and containers up and running. To check your running containers, execute the following:

docker ps

We see that our containers are up and running, and we can see on which port each and every one of them has been exposed and mapped.

It would be nice to see what is going on inside our containers, some kind of logs or something.

To see the output of your containers during their execution time, just execute:

docker logs {container}

This is the output of our Node app — the same output as if we’d execute npm start. It’s the output of our application starting, but now running in the container.

PgAdmin

We also see that our db management is running on port 80. If we go to “localhost:8080”, we’ll get our database management login page where we can log into our system.

We’ll use our credentials from the pgadmin container’s environment variables, and now we have access to our database server.

You see that we already have our todo-db, and the todos table created.

Testing our /todos route

Okay, we have our database services and application up and running inside docker containers, so now we can test our todos endpoint.

I’ll hit a simple CURL request towards our TodosController with the JSON formatting pipe.

curl 'http://localhost:4000/todos' | json_pp

And there we have our data.

Conclusion

The point of using Docker for your development is that you can put together a bunch of cool services, set up a testing environment, work with different versions of tools and see how your app behaves and a lot of other stuff. Just edit your docker-compose file, put some new services, and play around.

Thank you for reading. I hope you learned something useful!

GitHub repo: https://github.com/CyberZujo/todo-app

Building and running a Node.JS, TypeScript, PostgreSQL app with Docker was originally published in NSoft on Medium, where people are continuing the conversation by highlighting and responding to this story.

If you’re a web developer, especially backend developer, who develops APIs, among the gazillion things you need to consider is validating user input and preventing security leaks from a possible malicious user. Validating requests that hit our API is simply a MUST if you want to have a secure web application.

Manually checking payload properties and validating them can be of good use when you’re dealing with a type of pre-defined data with few properties that won’t subdue any major changes. However, dealing with bigger objects and their properties can be painful and slow if you’re using standard manual methods of validation instead of something like AJV.

AJV stands for Another JSON Schema Validator and represents the fastest validator for JSON schemas around. You can easily find it on npm and in the official github repo.

Let’s go through a simple example where we’ll see how to apply AJV within our NodeJS app.

Suppose you have an entity with the following structure (Typescript):

As you can see, there are plenty of things to be validated here. I’ll ignore the fact that there are database-level constraints I’ve previously defined. Validating the user’s input before it reaches any service that communicates with the database is something you always want to do.

Installing AJV

This is an easy step since AJV is available as an npm package. We will just do the following:
npm i ajv

Defining our JSON Schema

I won’t go into the JSON Schema details. You can find more info here. I’ll explain defined properties as we go.

Here we have defined our JSON schema with our properties and set some rules for each one of them.

There are plenty of other attributes you can find in the official documentation. We have set our rules for required properties, and each property has its own constraints defined, and that is what’s important.

Be cautious when defining rules here if your model will be persisted into the database and make sure rules here don’t break the rules defined on the database level.

Using ajv to validate against the defined schema

import tutorSchema from './schema/tutor-schema.json';

import ajv, { ErrorObject } from 'ajv';

We’ll import our ajv library and the ErrorObject for accessing validation error properties if you want to do some parsing, as well as our defined schema.

1. We’re initializing our ajv with the following options of allErrors and async for asynchronous usage.
   allErrors option is very important. If not provided, the program will end after the first validation first occurs.
2. The next step is to compile our schema using the compile method that will create a validate function for our schema. This is a faster way of validating by compiling our schema.
3. We use our validate method to validate against our model that represents our user input.
4. We check if there are some errors and can do some parsing

Here I’ve created a small parser method for parsing our errors (this is just for testing purposes and my predefined properties, you should use some of the npm libraries for humanizing ajv errors).

If I send some invalid payload towards my API, I’ll get a response like this:

Ajv does not format our error messages. We can directly access ErrorObject and do some parsing instead, or use another library such as better-ajv-errors to humanize our error messages. In the end, it all should be readable to end users without any generic data.

Using AJV for schema validation with NodeJS was originally published in NSoft on Medium, where people are continuing the conversation by highlighting and responding to this story.

In Part One, we have explained what serverless is, how to use Netlify Lambda, and created a simple example of functions. In this article, using lambda functions, we will create an API needed to run a simple workboard.

To have a working API for client application to consume, we need to have these functionalities:

• View and search boards
• View, create, update and delete board
• View, create, update and delete columns
• View, create, update and delete tasks

Extract Database Connection Handling

Since connecting to the database will be a common task, we will extract this functionality to utility function and use it in lambda functions.

Create Boards Function

This function will provide a client application with a list of boards and the ability to search them by name. We will create a new function boards.js for this purpose.

Now, we have a GET /boards route that returns response data consumable by a client application. We can use this data to build a workboard list.

We can paginate data using query parameters: limit and offset, and filter data using a query parameter name.

Modify Board Function

To get a single board detail, we will modify board function. We need to find the board by an internal identifier which we will pass using a query parameter. Since we will find the document by internal _id , we need to use ObjectId.createFromHexString method which we will add to db utility file.

Since we need to support adding, editing and deleting actions, we will update the board function to implement POST request with different actions.

Implement Board Columns Manipulation

Since each board can have board columns, we will support manipulation of columns inside board documents. One board can have zero or more columns.

Creating function for tasks

Task is created inside a column of the board meaning that it is necessary to send the column and board inside the payload of task creation and retrieve action. Since each task has id, for delete and update actions, it is enough to pass id. We will also create a new task collection inside the Mongo database. In this example, we will add just a few properties for task document and expand them once UI is built.

Since all tasks need to be shown inside the board column, paging is not necessary, but we will keep it to limit tasks in the column to some reasonable value.

Since functions need to be kept small, I did not use any utility functions like lodash for now. It is visible that we could use thedefaultTo method of library to create query parameters fallback. We will check for possible optimisations with tree-shaking in further articles.

Conclusion

Lambda functions can be used to create API in single-page applications. Every function is a single path in the API URL. We differentiate actions using HTTP verbs GET and POST and using custom action property inside the payload as we did in this case.

State of the application in this stage can be viewed on apiLambda branch of the repository: https://github.com/manico/workboard-lambda/tree/apiLambda

In Part Three, we will create a visual representation of the workboard using functions as an API and upgrade some parts of the API.

Creating Serverless Workboard with Netlify Lambda — Part Two was originally published in NSoft on Medium, where people are continuing the conversation by highlighting and responding to this story.

Infrastructure running our applications has gone through many changes in recent years — from using dedicated machines to run even simple applications, virtual machines reducing the cost of infrastructure to containers that simplified deployment and maintenance. To provide more development time and simplify application development, large companies like Amazon, Google, and Microsoft now offer an entire ecosystem for all applications allowing you not to think in terms of servers, but instead, you think of infrastructure as part of the workflow of your application.

Term serverless has been described differently during the past few years. To make it clear, I will use term serverless to describe an application that runs in third-party stateless event-driven compute containers. In this architecture, a unit of application logic is a single function executed in these containers — which is where we get term FaaS (Function as a Service). The benefit of this approach is that the application developer is only focused on application, not on infrastructure. Infrastructure is in total control of third-party, scales automatically and charges per function execution (idle time is free of charge).

The downside is that we relinquish control over infrastructure to third-party making the debugging and monitoring process more difficult as we depend on how detailed are these services and how good is local development tooling provided by the third-party. Luckily, this is getting more robust every day.

The other downside is usually vendor lock-in, although functions should be easily deployable to different providers. From the business perspective, additional risk is an unpredictable expense that can increase significantly depending on the usage of functions.

Currently, most popular providers are:

• AWS Lambda
• Google Cloud Functions
• Azure Functions

Netlify Lambda

In this example, we will use Netlify Lambda to create functions needed to power client application that manages simple workboard application.

In the traditional approach, to run this application, we would require:

• Client application that contains client logic
• Service application that contains backend logic
• Server that runs the client application
• Scalable infrastructure for service application
• CDN service for security and speed optimisation
• Database cluster
• Preferably caching layer for service application

Using the serverless Netlify approach, we will try to reduce this to:

• Client application that contains client logic
• Functions that execute application logic and communicate with the database
• Netlify infrastructure
• Managed database service

Netlify infrastructure provides automatic atomic deployment of client application and functions from a git repository, offers powerful CDN out of the box as well as other features like branch deploys, split testing, client application pre-rendering, automatic deployment and packaging of functions to AWS, DNS management, proxy and redirect rules, unlimited snapshots and rollbacks, cache invalidation, etc.

Netlify uses AWS lambda behind the scenes, but enhances developer experience by versioning, building and deploying functions along with the client application. It also implements a custom API gateway that handles service discovery and the ability to rollback deploy. This process is totally seamless and the developer only needs to worry about the application logic inside functions.

Project Setup

We will start this project by creating a client application in Vue (popular JavaScript framework) which will be the base of the project. The entire project will be contained in a single repository.

To create a new Vue application, we will use vue-cli command:

vue create workboard-lambda

From the CLI wizard, choose features: Babel, PWA, Router (with history mode), Vuex, CSS pre-processors (SASS with Dart), Linter (ESLint with Airbnb config), Unit Testing (Jest). Choose to save the configuration to dedicated config files.

To keep the initial setup clean, we will remove About.vue from files and router and rename HelloWorld.vue component to WorkboardCard.vue and remove entire generated content. The initial setup can be checked on branch initialSetup inside git repository.

We will immediately connect the git repository with Netlify to have this initial project built. Go to Netlify Application and click New Site from Git button. Follow instructions to authorise Netlify with GitHub and choose git repository to connect to. Once connected, enter build command and deploy the site.

After the build, the site is available at a given domain. If you wish, you can set up a custom domain in Domain Settings section.

Although it is possible to develop functions without additional tools, we will use Netlify Dev (part of Netlify CLI) for local development.

Netlify Dev brings the power of Netlify’s Edge Logic layer, serverless functions and add-on ecosystem to your local machine. It runs Netlify’s production routing engine in a local dev server to make all redirects, proxy rules, function routes or add-on routes available locally and injects the correct environment variables from your site environment, installed add-ons or your netlify.toml file into your build and function environment.

To use Netlify Dev, first install Netlify CLI as a global package:

npm install -g netlify-cli

Then log in to Netlify:

netlify login

You can now connect to your remote Netlify site meaning that Netlify Dev will pull all settings locally, so your local environment is in sync with remote. Consequently, local development is much easier and without changes when it comes to pushing to production. Use link command to link to your remote site:

netlify link

Choose from a list of sites to connect. Once linked, Netlify Dev creates .netlify folder inside your project with state.json file that contains site identification. From now we can run all cli commands inside the project directory.

To start the project locally, use netlify dev command. It will detect your project and run internal development environment command (in our case this is vue-cli-service serve command). It also starts Netlify local server that serves as a proxy endpoint for the client application, functions, add-ons and redirects. By default, Netlify chooses a random free port to run the dev server. You can override this with port parameter:

netlify dev -p 8888

Local server is now ready on http://localhost:8888.

Creating Simple Function

To create a function and serve it locally, it is required to create netlify.toml configuration file in the root directory of the project.

[build]
  command = "npm run build"
  functions = "lambda"
  publish = "dist"

Using build block inside configuration we are defining build command, functions directory and publish directory. Now we can create functions.

Netlify Dev offers commands for creating a function out of templates for easy learning. To create function this way use command:

netlify functions:create

Choose hello-world template and name your function echo. Command will create sample function that you can run using command:

netlify functions:invoke echo

Alternatively, you can visit http://localhost:8888/.netlify/functions/echo in your browser to get JSON result with hello message.

Since we wish to use the same JavaScript style (like using module syntax in Vue), we will not create functions directly in functions directory, but inside src/lambda directory where Netlify dev will perform a compilation of source code and output it to root lambda directory. For this, we can use any build tool we wish, but as a convenience, we will use netlify-lambda. First, install it as a project dependency:

npm install netlify-lambda --save

Add build script to package scripts:

"build-lambda": "netlify-lambda build ./src/lambda"

Now, when we execute npm run build-lambda, the tool will compile source functions to production-ready functions. Netlify Dev detects this script and recompiles function when the source changes.

Once we push changes to git, Netlify will start build and deploy process. When the process finishes, site and functions are available on the deployed site with the ability to revert to an older version in Deploys section of Netlify Application.

Project in this stage can be checked at setupLambda branch of the git repository. To execute remote function request URL ${baseSiteUrl}/.netlify/functions/echo.

Anatomy of a JavaScript Serverless Function

Serverless JavaScript function needs to export thehandler method. Name of the JavaScript file determines the name of the function. As demonstrated, every function can be invoked by requesting URL ${baseSiteUrl}/.netlify/functions/${functionName}. Function can also be triggered by internal Netlify events. We can use async to omit the last callback parameter recommended to return success or error:

Netlify provides two parameters in thehandler function. Event is an object that contains information about the request. It contains properties: path, headers, httpMethod, queryStringParameters, body which is JSON string representation of request payload and isBase64Encoded flag that indicates the encoding of payload. Inside context, Netlify provides information about the context in which function is invoked like clientContext in which we have identity and user information stored. Identity usage with functions will be shown in future parts of this article series.

To inspect theevent, we will add it to the echo function.

Once executed, this is the response:

${baseSiteUrl}/.netlify/functions/echo?testQuery=true:

In the response, data from headers is removed due to security reasons and for brevity. It contains standard headers along with the custom ones.

Connecting to Database Inside Function

To test getting data from the database we will connect to MongoDB named workboard and pull board collection documents to show in the output. You can create a free MongoDB database on Atlas. Create a database workboard and collection board that several documents with single property (along with _id) called name. Example of document:

{
  "_id": {
    "$oid": "5e0b43311c9d44000063556e"
  },
  "name": "Netlify Dev Release"
}

Create new board.js function inside src/lambda directory:

It is visible from the code that we are using environment variables. Where do they come from? Environment variables come from netlify.toml configuration file or environment variables set on Netlify Application project Deploy Settings. Since this information is secret, we are setting it inside the Netlify Application project settings. Once netlify dev is started locally, remote environment variables are pulled locally so we can run functions without changing anything.

If we inspect bundled (compiled) board function we can see that it is pretty large, nearly 0.5MB. This is because it contains bundled dependencies, in this case, MongoDB node driver.

Netlify Application also offers functions view where we can check the execution log of each function. This is the example log where we can see that once started from a cold state, the function does not have initialisation duration added to execution time. It also shows memory usage per function execution.

2:34:24: Duration: 698.72ms Memory: 80MB Init Duration: 186.89ms
2:34:26: Duration: 631.54ms Memory: 81MB
2:35:03: Duration: 645.15ms Memory: 83MB
2:35:17: Duration: 634.43ms Memory: 83MB
2:35:18: Duration: 658.80ms Memory: 83MB

In this stage, project can be checked at setupDatabase branch of the git repository.

Optimising Function Deploy and Execution

The first thing before going further into the development of functions is to check for possible optimisations. It is best to follow AWS instructions on best practices to work with lambda functions.

In the current code, we are instancing MongoClient on each execution. We need to move this outside of the function handler so that the connection is cached and reused during the function lifecycle.

When we check the function execution log now, we get quite different results:

3:31:13: Duration: 704.53ms Memory: 80MB Init Duration: 193.64ms
3:31:17: Duration: 90.32ms Memory: 80MB
3:31:19: Duration: 89.91ms Memory: 80MB
3:31:20: Duration: 90.34ms Memory: 81MB
3:31:22: Duration: 95.28ms Memory: 82MB

Duration time is decreased by almost 85% after the initialisation!

Another small optimisation we advise in this scenario is to set callbackWaitsForEmptyEventLoop property of context object to false. I have not seen any significant duration changes when this is set. The explanation is in AWS documentation and MongoDB best practices:

By default, the callback waits until the runtime event loop is empty before freezing the process and returning the results to the caller. Setting this property to false requests that AWS Lambda freeze the process soon after the callback is invoked, even if there are events in the event loop. AWS Lambda will freeze the process, any state data, and the events in the event loop. Any remaining events in the event loop are processed when the Lambda function is next invoked if AWS Lambda chooses to use the frozen process.

Other best practices can be found here.

In the Part Two we will create entire API for our application using Netlify functions.

Creating Serverless Workboard with Netlify Lambda— Part One was originally published in NSoft on Medium, where people are continuing the conversation by highlighting and responding to this story.

In NSoft, we produce and integrate games inside our online solution as well as third-party websites. The integration process on a third-party website involves several steps where third-party developers are involved to make things work, too. This process can be complex because it often requires custom layout development to suit customer look and feel, message exchange arrangement between a third-party website and game iframe, etc.

To speed up the entire process and offer customers unique experience we decided to create a new system for running our virtual and other games. We named it Grame. Simply put, Grame is a system for integrating and running multiple products inside Seven (our betting platform) or a third-party website.

Solve integration problems

Using this system, a third-party developer will just have to integrate a snippet of code and update a single placeholder on their side. The entire integration process and message exchange are handled inside Grame system integration script. Moreover, there is no custom design and development since Grame offers a unique and standard look and feel. Customisation can be done using our Chameleon Builder where customer can choose which modules to use inside Grame and set some basic color scheme.

Multiple games at the same time

With Grame, a user can run multiple games at the same time in the same view. This offers faster betting because it removes the need to switch between the games. Without the game switching and with the ability to run up to four games at the same time, a punter can play multiple tickets in a short period of time. More tickets mean more revenue.

This system is flexible enough to accept all our games, but also third-party ones like Casino games. The user can play LuckySix and Slot game at the same time!

Features and Customisation

Main features of the Grame system are:

• Fast and easy integration
• Pay in one ticket or play multi tickets
• Focus on the game
• Play multiple games
• Build using Chameleon
• Run in full screen
• Games lobby inside Grame

Grame system will be available on web, mobile and terminal channels with the same standard UI and UX.

Grame is configured and built using Chameleon Builder. Configurable options per customer are:

• Full or integrated version
• Show or hide header
• Change color palette, show or hide customer logo, show or hide balance
• Use multiple games or just single game
• Support favorites, game help or game settings

Showcase

You can check out our showcase videos for web and mobile:

We would like to thank our Design Team for the idea and for pushing it into development.

Grame — a New Way of Online UX was originally published in NSoft on Medium, where people are continuing the conversation by highlighting and responding to this story.

A big part of the new cloud narrative is the observability which encompasses monitoring, logging and tracing with a single purpose of gaining visibility into the performance of your apps and infrastructure.

How to handle logging in Kubernetes has long been a topic of discussion here and there isn’t a simple answer. I personally have searched for “best practices” and failed. It’s all up to you. Act according to your needs and circumstances.

At first, we used our Graylog centralized setup which was already serving our existing infrastructure and just deployed Fluentd into our cluster. The issue here was that we had a bunch of logs streaming to our Graylog in high volume. This included Ambassador API Gateway, Istio, Prometheus and all other infrastructure apps. You can only imagine the volume of such streams.

In order to rationalize the process, our decision was to deploy a local stack into the cluster which will collect all the logs from the cluster (including Kubernetes API events) and store them there. All other application logs can still be forwarded to Graylog if they are critical for storage.

While searching for a complete solution I ran into many guides on how to deploy an ELK stack for logging but something was always missing. This provides a simple and robust way to deploy such logging stack by using a single helm chart (which is just a meta chart that combines other charts).

Components Used

The first part of the stack is Elasticsearch which is a distributed search and analytics engine used worldwide mostly with Elastic Stack (Elasticsearch, Logstash and Kibana — ELK) for open source logging purposes. You can read more at Elasticsearch logging.

Here I am using a helm chart provided by Elastic for deploying Elasticsearch. There is also a chart in stable/charts but it is scheduled for depreciation, info here. Similar to Elasticsearch, there is a new chart from Elastic for Kibana which is going to be used for this.

There is an elastic operator that is being developed by Elastic which may be used in the future but right now it is still in alpha. It is capable of deploying Elasticsearch and Kibana using Kubernetes CRDs and promises to provide a way to dynamically scale storage which is not possible with the chart. I will update when that matures a little. More info on the operator here.

Next component used is elasticsearch-curator, designed do rotate older indices in order to save space (logs can take a lot of space and retention policy varies from company to company. Our Graylog setup is used for long term storage and this logging stack deploy is more of a short term storage). It also has an option of archiving older logs to S3 but that is not the scope of this article.

In order to read logs from the container stdout and ship them to Elasticsearch nodes, fluent-bit is used. Additionally, to read Kubernetes API logs we use metricbeat.

Metachart

It’s called a metachart because it doesn’t have any templating implemented and it is being used just to enable deploying the whole stack with a single chart that has others in its requirements. The chart is located here.

The idea is that by using requirements one can still use the chart values for each component of the stack enabling deployment customization according to needs using requirement aliases as shown:

dependencies:
 — name: elasticsearch
 version: 7.1.1
 repository: https://helm.elastic.co
 alias: elasticsearch
 — name: elasticsearch-curator
 version: 1.5.0
 repository: https://kubernetes-charts.storage.googleapis.com
 alias: curator
 — name: kibana
 version: 7.1.1
 repository: https://helm.elastic.co
 alias: kibana
 — name: fluent-bit
 version: 2.0.5 
 repository: https://kubernetes-charts.storage.googleapis.com
 alias: fluentbit
 — name: metricbeat
 version: 1.6.4
 repository: https://kubernetes-charts.storage.googleapis.com
 alias: metricbeat

Where versions are defined, the chart can be updated using:

helm update deps

to fetch the latest versions (if explicitly set in requirements.yaml file) from upstream charts. This saves time from maintaining forks.

One can also use values from upstream charts in valuefile by nesting them under the defined alias for each requirement:

elasticsearch:
 clusterName: “logger-elasticsearch”
 nodeGroup: “master"

which enables you to use all the options that the chart maintainers created.

Deploying the stack

The logging stack is deployed by cloning the chart repo:

git clone git@github.com:volatilemolotov/k8s-logging.git

and entering the chart directory:

cd k8s-logging/logger

The next step is to edit the valuefile in order to set up the stack. The provided default values are more than enough to run the stack. If you choose to edit you can read about options on the respected chart repos:

Elasticsearch

Kibana

Elasticsearch-curator

fluent-bit

Metricbeat

One thing to note is that you have to change the valuefile for the elasticsearch-curator, because the image used for curator is no longer maintained and does not work with Elasticsearch 7.1.1.
You can use the following image. I have just forked the elasticsearch-curator git and set up an automated build on Quay.io. Also, remember to change the command used.

curator:
 cronjob:
 schedule: “0 1 * * *”
 serviceAccount:
 create: true
 image:
 repository: quay.io/volatilemolotov/curator
 tag: v5.7.6
 pullPolicy: IfNotPresent
 hooks:
 install: false
 upgrade: false
 dryrun: false
 command: [“/curator/curator”]
 configMaps:
 ---

Next step is to install the chart:

helm upgrade — install logger . — namespace logging -f values.yaml

and wait until all the components are running.

After that port-forward into the Kibana pod and go to localhost:5601.

When starting Kibana for the first time, index patterns are required to be set up. This can be done by going to the Discover panel where you are presented with a list of indices. Create two : kubernetes_cluster* and kubernetes_events*. Select @timestamp for Time Filter field name for both.

Storage Caveat

Given that the Elasticsearch chart uses StatefulSet for deployment, it is not possible to change the size of the Persistent Volume Claim. In order to do this you need to manually edit the claim (all three of them if deploying from standard values) and wait until the resize is finished. Use:

kubectl get pvc logger-elasticsearch-master-logger-elasticsearch-master-0 -o yaml

to check the status of the PVC. When the resize action is done, you will get the following message:

message: Waiting for user to (re-)start a pod to finish file system resize of 
 volume on node.

After this message, you need to reset the pod which is using that disk. Make sure you reset the pods one by one.

If you wish to contribute, you can join us in our open source projects, or you can apply for a job at NSoft.

Kubernetes Logging Stack was originally published in NSoft on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introducing NSoft’s BAS and NDS

The formation of BAS (Business Analytics Specialists) team started 2 years ago with the arrival of a small BI team to NSoft in order to face new challenges in a completely new industry.

Considering our business background and experience in Data warehouse (DWH) development and maintenance, reporting and data provisioning, as well as the previous, almost decade long, cooperation and teamwork, we were always well aware of the importance of data and its impact on business performance. But what we have encountered at NSoft has solidified our thinking that the sole data provisioning is the thing of the past — the present data provisioning is being performed by automated systems without human interaction. That was the starting point of something that consequently evolved into a completely internally developed full-fledged Data Analytics Solution backed by BAS team and data engineers in NDS (NSoft Data Service) team. Implemented, with the best practices in mind, we are currently operating Data warehouse (DWH) capable of nearly real-time processing and ingesting millions of data points per second which then feed into our analytics layer. It resulted with a clear new goal to strive for — Data Analytics.

What is a Data Analyst?

Data Analyst by definition is someone who is able to successfully translate large quantities and a variety of data into plain language understandable to others. But the role of the data analyst isn’t strictly tied to data “translation” and understanding. It’s more of a role that incorporates generating clear and advisory actions that have an immediate business impact. Within the context of NSoft, upon initial observation of the entire business process, we have soon realized the requirements for a data analyst and how one could contribute. This included familiarization with vast quantities of data, from numerous and various sources, combining, preparing and finally analyzing and extracting the value. What seems like a list of cursory terms is actually enormous effort that involves preparation, analysis, development, and usage of a system that enables BAS to govern valuable data and generate value for NSoft and it’s partners alike.

Importance of Turnkey

Striving towards common growth is the simple definition of what the Turnkey is. Coming to NSoft and adopting the Turnkey concept has made us completely change our mindset regarding the usage of product and user generated data. For NSoft the turnkey product solutions represent symbiosis with our partners with the common goal of mutual growth. Taking it all into consideration, through time, we have realized our mission statement has shifted: “We will not be providing data, we will provide the value!”

Data Analytics as a product aid

One of the ways through which data analytics provides value is by improving the existing product portfolio wherever possible. By providing services of statistical analysis, testing product logic, identifying user behavior as well as positive user feedback, we aim to implement our findings and insights into bettering each of our products.

Based on our analyses and observations, the numerous improvements, tweaks, upgrades have been made to our products which have led to greater recognizability of our products and far better player retention.

All products from NSoft’s rich portfolio receive the same treatment from our Analytics team in order to provide the best possible product to our partners and the best gaming experience to our players.

All of that combined sums to a sentence that best describes our aspirations:

• Through data analysis, we aim to improve existing products and help to create new ones, to provide our partners with the better business results, give the players the best possible playing experience and maintain the highest standards our company has set up and insists on.

Data Analytics as a partner aid

Everything previously mentioned has resulted in the development of workflows capable of consuming and digesting this enormous amount of information which enables us to provide our partners with momentary information regarding anything of interest.

Additionally, by identifying the reasons for user churn we can provide our partners with actionable insights. Actionable insight is more than an information with context, it’s a prescription for a clear course of action. The final goal of those actions is an increase in partner’s revenue, as well as an increase in confidence in our information provisioning.

Our team has streamlined dashboard generation into an assortment of standardized report packages offered to all of our partners and in-house teams and individuals alike.

Today NSoft’s BAS team aims to improve existing products and help create new ones, to provide our partners with the better business results and give the users the best possible playing experience.

The technology that made it all possible

The technology stack involved was designed to be robust, easily scalable, adaptive and highly available and it required an enormous effort.

In our DWH implementation, we’ve chosen a streaming solution instead of ETL processes. As a starting point we should mention in-house operational data stores used for streaming sources:

Currently, from NoSQL databases, we have MongoDB as a source, and from SQL databases we’re currently using MySQL and PostgreSQL. Additionally, due to our partners’ specifics and requirements we are supporting various CSV, Google Sheets, SQLite, MS Excel data enrichments which can be blended at source or anywhere in the streaming pipeline seamlessly.

For data streaming Apache Kafka is used as a backbone and it’s accompanied by ZooKeeper, Kafka Connect, Kafka Streams, Kafka Schema Registry (as we’re using AVRO for primary streaming format) and Confluent kSQL.

Through streaming, pipeline data is enriched and transformed to ensure resulting data form compliance and quality and minimize data cleansing overhead.

Due to high-level data processing in the pipeline we’ve also decided to implement a caching layer using Redis.

For resulting DWH solution we’ve opted for Vertica, a leading Enterprise SQL Column Store Database solution, as our long-term store and analytics back-end, as well as opting for Elasticsearch for our near real-time data insights.

The Tableau is our primary information presentation driver, coupled with Superset, Metabase and Google Data Studio. Tableau was versatile enough to meet all the requirements we have had from an information distribution platform. Grafana and Kibana were our primary choices for visualization of near real-time data.

Programming languages in use are Java as a primary, Python as a secondary but as it usually happens, you need to expand your palette to deliver a good product so Bash, NodeJS, and PHP happen to be regular visitors of our team.

Monitoring needs are being effectively covered with Sensu for the systems part and Prometheus for service monitoring — ranging from availability checks to performance metrics.

Most of our stack is powered by Docker and due to stack size it was impossible to keep things smooth without orchestration solution, that’s where we decided to use Kubernetes with Rancher on-top for easier management.

Current CI/CD pipeline is custom implemented using Helm based Jenkins and Kubernetes publishing. It’s also worth mentioning that our K8S setup is expanded with Istio service mesh, MetalLB, fluentd, Rook and many other improvements compared to the standard issue of Kubernetes setup.

The entire setup is currently configured on bare-metal dedicated server machines and is horizontally and vertically scalable, as well as highly-available. The configuration of such stack led to high proficiency in DevOps department and faced us with some interesting challenges.

If you wish to be a part of our company and learn more about data science, find out more about us and apply for a job.

Data Analytics at NSoft was originally published in NSoft on Medium, where people are continuing the conversation by highlighting and responding to this story.

Here at NSoft, we have an in house design team with a total of 9 designers. We tend to hire ‘T’ shaped designers meaning that most of us have general knowledge across many different areas of design, but everyone is really, really good in one thing. UI and UX designers, Interaction Designers, Graphic Designers, and Illustrators are working together to bring the best possible product to our customers.

When people think about design, they mostly refer to products’ appearance. Currently, we do have one of the best looking products on the market. However, now we work more on evolving our Design System (standardized set of rules and elements that unify look and feel of all our products), product updates and market/user research, so we can bring best possible products to our customers as well as a great experience for their users.

We don’t just design products, but also we suggest what might be appealing to the market.

We want to deliver something beautiful with both functional design and the best possible user experience.

I’ve been working in NSoft for 7 years now, so I could say I grew up with the organization. I’ve participated in shaping our Product Design team into a unique group of people.

It’s important to say that we were lucky enough to have more than just a talented team, but leaders who believe in the power of design. We could say we’ve deserved it by bringing good ideas to the table and making great products. The Designers from other companies struggle to bring their vision and ideas to top management, regardless of their work.

Process

Everything might look easy when you read and learn about the different processes of product development, especially when some of them go through 5 phases and others go through 12 phases. Some companies first do the research, others start designing their product and make changes on the go while some define problems they want to solve and then proceed to another phase.

What works for one company, most likely, does not work for another.

To make a good product you have to understand your clients’ needs, who makes the requests and what are the capabilities of your team.

We use different approaches depending on the type of request:

If a product is completely new, we like to start with a few concept designs which can help everybody in the team to understand what we are actually making.

If we need to update the existing features, we use iterative design — we do research and based on the analyzed results, we make changes.

We realized that Design sprints are a good way of making new innovative products. We usually use these sprints in the early phase of planning a new product. The idea is to describe and let everyone understand what is the problem we want to solve. Afterward, everyone in the team works independently on their solution for one day before coming together again to share their work. We go through all solutions, discuss them and decide which is the best one.

Collaboration

We truly believe in the strength of collaboration, not only within the Design team but within the company. When making or updating our products, we have brainstorming meeting(s) that involve members from different teams — Development, Product Management, Business, etc.

What I’ve learned through my work is that there are Backend and Frontend Developers with a great sense of product and UX. Despite the thought that Designers have the best and most creative ideas, if you give chance to other people to participate in the process of product design you could be surprised with results. Anyone can have and suggest ideas and every idea is valued equally.

Leave your silo, explore, talk and develop great products with the help of people around you.

Robert Pressman in his book “Software Engineering: A Practitioner’s Approach” said: “For every dollar spent to resolve a problem during product design, $10 would be spent on the same problem during development, and multiply to $100 or more if the problem had to be solved after the product’s release.” Solving the problem at the start is the most cost-effective and you can do it only if you include people from different areas.

Our company and team have grown rapidly in a few years, and it was hard to motivate people to work through collaboration. The truth is nobody should believe that their ideas are the best ones.

Biggest challenge

If you know any passionate designers, you can agree — designers are, let's say, unique. They all have cutting edge ideas, their own style, and unique mindset.

Now, imagine 9 people with different ideas and try to make them work with the same design elements under the same vision. You have to make them feel that the product they are building is actually theirs because if they do not feel that, it will not be made with passion.

How we help our clients?

Today, if you have a bad product or service, users go to other providers. There are numerous companies in the same business, so you have to continuously evolve to keep up with the trends and customers’ needs.

The primary goal of any business is to increase its sales, make sure the business grows and build brand awareness.

Product Design plays an essential role in achieving this goal by improving user experience and customer satisfaction. The result is an increased number of product users.

Each customer has a Customer Lifetime value and this value depends upon the customer’s willingness to keep using your product. If you have a great UX design that does not frustrate the customers, they will more likely dismiss the small mistakes.

Do you know how many users don’t use your product because they had trouble registering? How many of them don’t know how to use your product at first sight? How long does it take to do an action and can you make users do it more easily? Conversion rates depend highly on the user’s ability to use the application easily.

We get to know our users by doing research on what they expect from the product. We try to iterate regularly on existing products and improve the experience for partners and their users.

If you have the best product today, it doesn’t mean you should stop working on it.

It is necessary to research and always stay on alert. That’s the reason why we are always on top of new technologies, and among the first ones in our industry offering concepts for our products that include virtual and augmented reality, artificial intelligence, face recognition etc. These products don’t necessarily bring us money, but they prove that we don’t work in a bubble and that we are ready for the future.

Wrap Up

Design became an integral part of our vision and additional value driving company strategy. NSoft will keep creating a culture, environment, and process that attracts the best people for our team and company.

If you wish to be a part of our Design or any other team, apply for a job and join us at NSoft in the creation of innovative products.

Cheers.

Product Design team at NSoft was originally published in NSoft on Medium, where people are continuing the conversation by highlighting and responding to this story.