The main key point for all our projects is trying to automate all things, it helps to reduce errors (e.g. human mistake), fast and easy to deploy / rollback, and improves customer satisfaction.

We do have some approaches to automate the deployment process that we have been applying to our projects such as using Amazon Elastic BeanStalk or Amazon Elastic Container Service. However, in some cases, our customers don’t want to use those approaches because they don’t use AWS, they’ve already had the servers to run the app, and they want to keep their app on their On-Prime Data Center and have their IT guys manage it. It led us to have to find out another approach to automate our delivery process. And we have chosen Ansible as the tool to implement our Continuous Delivery pipeline.

So I would like to write this article to share with you and outline our workflow and provide a brief introduction along with sample code to build this flow.

The Tools

• Jenkins: Jenkins is a self-contained, open-source automation server that can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software.
• Ansible: Ansible is an open-source automation platform. Ansible can help you with configuration management, application deployment, and task automation. It can also do IT orchestration, where you have to run tasks in sequence and create a chain of events that must happen on several different servers or devices.
• Slack: a cloud-based set of proprietary team collaboration tools and services.

Why Ansible?

As you know, there are a lot of tools for infrastructure automation (infrastructure as code) such as Terraform, Chef, Ansible, Juju, and more. But there are a few reasons that make me select Ansible as below:

• It is very, very simple to set up and yet powerful.
• Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications.
• Automate in a language that approaches plain English, using SSH, with no agents (like Puppet or Chef) to install on remote systems.
• Easy to integrate with our CI/CD pipeline using the Jenkins server.

The Steps

What we are going to build is the following deployment flow:

1. Launch EC2 instance(s) (optional).
2. Update OS and install needed dependencies such as install Node.js.
3. Deploy the Node.js app to EC2 instance(s).
4. Configure DNS (optional).
5. Smock test.

Let’s code…

Project Structure

When doing any coding stuff, the project structure is one of the most important things I pay attention to. If you have experience working with Ansible, you should know how to organize the Ansible project. I followed the alternative approach mentioned in this article, feel free to select your own approach.

My project contains the following root directories/files:

• scripts: it contains a shell setup script to install Ansible, Ansible Galaxy, and generate a self-signed certificate to run our Node.js app in secure mode (HTTPS).
• cert: directory to keep a self-signed certificate.
• ansible: put all Ansible code into this directory including playbooks, roles, var files, etc.
• Jenkinsfile: the Jenkins pipeline code for CI/CD pipeline.

Role

In Ansible, the role is the primary mechanism for breaking a playbook into multiple files. This simplifies writing complex playbooks, and it makes them easier to reuse. The breaking of playbook allows you to logically break the playbook into reusable components.

Here are the following roles we will use on this project:

• apache-server: we don’t use this role on this project but I wanted to keep it here. This role will configure the Apache using our own httpd.conf template file. To install Apache, we can use geerlingguy.apache role.
• common: put all your comment stuff into this role such as updating your OS to ensure your server(s) are up to date.
• launch-ec2: this role to launch EC2 instance(s) to deploy our app.
• node-server: contains all needed steps to deploy and run the Node.js app
• nodejs: install the Node.js runtime.

Conclusion

Using Ansible, you can easy to provision and install software to run your application automatically, you can manage your infrastructure as code (IaC). However, running your software is not simple as this demo, you will need to do more things such as setup the network, configure the firewall, and auto-scaling. To do those things, you might need to combine them with other tools. For example, you can use the AWS CDK framework to provide your AWS infrastructure (network, database, EC2 instances, and auto-scaling), then use Ansible to provision your EC2 instances and configure your applications.

I hope this post will bring some ideas to you to design and deploy your application. If you have any issues or need help, just let me know by adding comments to this post.

Thank you for reading!

This post is originally published on our blog.

Continuous Delivery — Deploying a Node.js app to AWS EC2 using Ansible was originally published in INNOMIZE on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this article, I will let you know what is MVP and benefits that MVP development can offer to your business.

What is a Minimum Viable Product (MVP)?

“The minimum viable product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.” by Eric Ries

MVP is the basic version of the new product with a certain set of core features to resolve a specific issue of the targeted customers. Companies such as Facebook, Twitter, Uber, Dropbox, Zappos started out with an MVP and become the largest applications and influential today.

So, why a business should start with an MVP?

1. Minimum time to market

Building a full-blown application that would require overcoming extreme technical hurdles and months of development. Instead, MVP development takes only a couple of months, which allows you to roll out your product fast and thus have better chances to prosper on the market.

2. Early Feedback From Users

The original idea of Instagram was to allow people to check-in where they are on their mobile web app. But after it was launched, the leaders decided to change the concept to meet the expectations of users.

Don’t wait until your application has full-blown features because your idea might not meet the expectations of users. Starting with an MVP and early get feedback and then improve your application.

3. Reduce rework and minimize development costs

Turning an idea into a full-blown application and meet the user’s expectation will take a humungous amount of effort. Rework can never be totally eliminated. However, with the MVP approach, you can make it as less as possible. You know features that customers don’t want and improve from an early stage. So rather than revamping the entire product from scratch, you just need to make the basic fundamental changes to the product. By doing so, you ensure the smooth execution of your product then save much cost.

4. Attract Investors

With an idea in your head and you are looking for investors. Believe me, it may sound not very promising for investors. One of the benefits of an MVP development is that you will have a rather serious perspective than just your thoughts to show to investors interested in funding you. This will definitely escalate your chances of being noticed and heard.

Takeaways

MVP development is always a safe and wise way to launch a new product. it helps you test your ideas in your chosen market before you create the full product. It also allows you to speed up time-to-market, maximize your budget, get early feedback, provide you valuable improvement ideas and analytical data.

Ready to develop an MVP for your idea?

At Innomize, we offer full consultations for a minimum viable project. Are you interested in developing your MVP with us? Don’t hesitate to get in touch.

Why Do You Need an MVP for Your Business? was originally published in INNOMIZE on Medium, where people are continuing the conversation by highlighting and responding to this story.

Problems

One of the features of automated application testing is uploading files. You can easily create a file in the workspace and upload it to the web applications by protractor when running locally. But when you try to run your test on docker, a problem occurred. There is a message: “Failed: invalid argument: File not found: /home/headless-auto/data/abc.txt”. It means the webdriver server is not able to reach the folder contains the file abc.txt.

This is because when we run protractor in docker, the Selenium server and Protractor are running in 2 individual containers, so the Selenium server container cannot get the file was created and saved in protractor container, to put to the web application.

• Is there any way to get a file in a container from another container?

Yes! It is not actually getting a file in a container from another container, but it still helps to solve the problem above. The docker supports us to share the folder between containers by using volume. All files in the shared folder will be synchronized with the volume. So the selenium server can get all files in the shared folder in the protractor container, then able to execute the uploading files.

Jump to coding

Let get the idea by the example below:

Protractor needs two files to run, a spec file and a configuration file.

Let prepare an uploading test script:

The Describe and Its syntax are from the Jasmine framework. Browser is a global created by Protractor, use path.resolve helper function to resolve the relative path to an absolute path.

• Configuration file:

This configuration file means:

• The default browser is Chrome (The protractor also supports Firefox, IE, Safari..)
• The address of running the selenium server is http://localhost:4444/wd/hub. The Protractor can test directly against Chrome and Firefox without using a Selenium server. To use this, in your config file set directConnect: true.
• The framework is Jasmine. Protractor supports two behavior-driven development (BDD) test frameworks out of the box: Jasmine and Mocha.
• “specs” option tells to protractor where is test files.

Start the Protractor test:

We need to run tsc command to compile into javascript that node.js can understand.

Run “webdriver-manager update” to download the necessary binaries. Then start up the selenium server: “webdriver-manager start”.

Now we can execute “protractor output/protractor.config.js” to run the protractor test. For now, we are able to open the web application and then uploading the file.

To run on docker, add the Dockerfile and docker-compose.yml files like below to the workspace.

Docker-compose use “selenium/standalone-chrome:3.141.59-dubnium” as selenium server so we need to change the seleniumAddress in protractor.config.ts file to http://selenium-chrome-standalone:4444/wd/hub

We will share all files and folder in /home/headless-auto directory of protractor and selenium server containers into “data-volume” volume.

Now we can run the test in docker by executing docker-compose up command. Then you will able to upload the file to the website without any issue.

Conclusion

Running the end-to-end test on Docker is an easy way to implement the automation tests. With Docker run can your tests on cloud-based such as AWS Fargate or even AWS Lambda. I hope this post will help you and save your time!

How to Perform the File Upload Operation with Protractor on Docker? was originally published in INNOMIZE on Medium, where people are continuing the conversation by highlighting and responding to this story.

JavaScript is a popular language among web developers which gave rise to several frameworks that simplify your code. According to the largest online developer communities, StackOverflow in their 2019 survey of the most sought-after programming languages shed some light on what to expect in the coming year:

As you can see, JavaScript is the most commonly used programming language. So how to learn JavaScript, how to up-level your programming language skills, and what is the path for learning JavaScript from the beginning? If you don’t know how to get started, then you should read this post.

Step 1 — Learn basic and fundamentals

Firstly, you need to understand the basic concepts and fundamentals of JavaScript, you can learn from

• W3 Schools
• Javascript.info

You-Dont-Know-JS is a series of books diving deep into the core mechanisms of the JavaScript language. You should read this to get more overview and advanced concepts of JavaScript such as Scope & Closures, Objects & Classes, Sync & Async, ES.Next & Beyond.

Step 2 — Learn modern JavaScript

Learning basic isn’t enough, you need to learn more about advanced and modern JavaScript concepts/syntax, it helps you write code efficiently, quickly, and high performance

• ES6 for Everyone
• Eloquent JavaScript

Step 3 — Coding Style and Convention

Coding standards (also sometimes known as ‘Coding Conventions’ or ‘Coding Rules’) are a set of guidelines that a group of developers stick to, to ensure that they all essentially follow the same style.

• Airbnb JavaScript Style Guide
• Google JavaScript Style Guide
• jQuery Style Guide

Read about prettier and see how to configure it in order to format your code and save time.

Step 4 — Review your knowledge

Now is the time to review your understanding of JavaScript by answering interview questions

• Top 50 JavaScript Interview Questions You Must Prepare in 2020
• Top 85 JavaScript Interview Questions & Answers
• 37 Essential JavaScript Interview Questions

Step 5 — Practices

• Learning JavaScript libraries (e.g. Lodash, ReactJS, Moment, jQuery, Ramda, and a lot more)
• Learn JavaScript Frameworks such as Angular, VueJS, EmberJS)
• Task runner (e.g. Gulp, Grunt) and module bunder (e.g. Webpack, Browserify)

Step 6 — Be an expert

• Continue learning core concepts in JavaScript such as Closure, Hosting, Scope, Coercion, Prototypes
• Learn OOP, Functional Programming
• Read this book Learning JavaScript Design Patterns or design-patterns

If you want to become a web, backend or full-stack developer, JavaScript isn’t enough, you need to learn more. Check out this roadmap and pick your path Web Developer Roadmap 2019 and keep yourself up to date.

A brief summary of JavaScript Learning Path was originally published in INNOMIZE on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this port, I’ll be guiding you on how to build an API with AWS Lambda and API Gateway to paginate data using the Serverless framework.

Technologies stack

• AWS Lambda for running code without provisioning or managing servers. You can utilize free-tier to run your service with zero cost.
• AWS API Gateway to create, publish, and secure Rest APIs.
• Serverless framework to build and deploy your backend services on the cloud, you can build and deploy a serverless function in minutes.

Prerequisites

• An AWS account: https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/
• Have some experiences on AWS Cloud computing and the Serverless framework.

Cursor-based Pagination

Have you ever heard this concept, it just is simple a pagination technique but it does not work the same with the classic pagination. Yes, the classic pagination that I think you will be familiar with, it works based on the offset-limit concept. An offset is a number that specifies the number of rows to skip before starting to return rows from the query. For simplicity, you want to see the data on page 3 with page size is 10, just skip 20 rows and return the next 10 rows as the result.

Offset-limit is one of the most commonly used techniques for data paging. However, there are several drawbacks of using this approach:

• With a database that has large datasets, the offset clause makes slower querying as the number of records increases because the database still read up all records before skipping with offset records and return it.
• In datasets that change frequently, the paged data becomes unreliable and possibility returns duplicate results.

Cursor-based pagination (aka keyset pagination) works based on a cursor that is a unique identifier for a specific record in datasets. We will deal with the cursor as the pointer of the next record that we will receive on the next page. And on subsequent requests, the server returns the cursor until there is no record on the next page.

There are a couple of approaches to implement cursor-based pagination, below are some interesting articles that I think you should take a look to get more details:

• Evolving API Pagination at Slack
• “Cursor Pagination” Profile
• Evolving API Pagination at Slack

Advantages:

• Prevent many of the drawbacks of classic pagination that I have mentioned above.
• Scale well with the large datasets.

Disadvantages:

• The end-user cannot see the total number of items in the list.
• The end-user cannot jump to a specific page or back to the previous page.

With the above benefits, I chose cursor-based pagination for this implementation.

In this demo, I will show you how to implement cursor-based pagination using a unique and sequential column of the table (i.e. auto-increment column). Yeah, it is pretty simple and easy to understand.

Serverless

The Serverless Framework helps you develop and deploy your AWS Lambda functions, along with the AWS infrastructure resources they require. It’s a CLI that offers structure, automation and best practices out-of-the-box, allowing you to focus on building sophisticated, event-driven, serverless architectures, comprised of Functions and Events.

There are ways to create and deploy an AWS Lambda function by using AWS Cloudformation or AWS Console. But I chose serverless because of its convenience, automation and the ability to increase development time.

One most of my favorite Serverless feature is the plugin, some of the plugins I recommend to use in this article. In this guide, I will use serverless-plugin-typescript and serverless-offline for building Serverless applications.

• serverless-plugin-typescript: this plugin helps us to compile typescript during development and deployment. I like this plugin in usage with zero-configuration but there is an issue that I can’t solve during using this plugin is path resolver. You have to use relative import and make your code becomes lacking in aesthetics. To use non-relatitve import you can use another option such as https://github.com/serverless-heaven/serverless-webpack.
• serverless-offline: emulates AWS Lambda and API Gateway on your local machine to speed up your development cycles. If you are using serverless this plugin be indispensable during testing on local.

With the serverless application, we must define a template file for the definition of your AWS resources, configurations, list of plugins, etc.

In the functions section, we will link to the file that contains code for reading data with pagination. Before implementing code for that function, we have to create a database and datasets for testing purposes (should have more 50 items in the datasets). I will use the MySQL database and you can choose others such as MSSQL or PostgreSQL, etc for creating the database and preparing datasets.

Continue, to connect to the database we can install this package, it is an awesome Node.js module for communicating with the MySQL database. However, the AWS Lambda function scales by creating separate instances for each concurrent request (you can see here for more details). And this is a serious problem for the RDBS database like MySQL when we can get the maximum connection error anytime with high concurrent requests.

Luckily, Jeremy Daly has built a package that is wrapped for MySQL to solve the above problem. We implement code for database helper as below:

We create a file to build a query for reading data support cursor-based pagination:

As you see above code, I use the squel to build the query string, you also manual to build the raw query. But I recommend you to use the squel by its benefits like:

• Supports parameterized queries for safe value escaping.
• Supports Non-standard SQL.
• Uses method chaining for ease of use.

Finally, implement code for the function as below:

The API we develop accepts two params:

• limit: a number value to limit records that return from the server.
• cursor: a base64 string that is returned from the previous request. It indicates the starting point to retrieve data for the next page.

For testing, I will run the following command to start API on local:

sls offline

Let’s explore our work. Using the Postman, we can easier invoke API

Deploy

Deployment is much easier with the Serverless framework and can be done with one single command. So let’s deploy and see results by running below command:

sls deploy --stage [your-stage] --region [your-region]

Feel free to check out your deployment and try to invoke API Gateway on the AWS and compare results. I hope it works for you. If not, please let me know.

The source code is available on our Github

Conclusion

I hope this guide will be helpful to you and bring to you some ideas to help you build your services with cost-optimized, secured, flexible, high availability.

In this guide, I only write a simple code to show how to build and deploy. You are free to extend this project with some additional features as below:

• Supports more param options such as fields, sort by …
• Implementing classic pagination (offset-limit pagination).
• Implementing the error handler for handling such as InvalidCursorError or UnexpectedError.
• Uses other databases such as Amazon DynamoDB is a great option.

How to build serverless cursor-based pagination service on the AWS Cloud was originally published in INNOMIZE on Medium, where people are continuing the conversation by highlighting and responding to this story.

Security is one of 5 pillars of the Well-Architected framework, it can archive by applying best practices and principals in IAM, Encryption, Complician, and Governance. Of course, best practices aren’t enough, you need to learn more. In this post, I only share our best practices and tip when working with AWS SSM Parameter Store. By sharing our best practices, my hope is to encourage you to build and deploy secure and reliable applications and also giving us your feedback.

As you know, AWS Lambda supports native environment variables, you can easy to define and add any environment variables you want during deployment or change on the AWS Console Management. But using native environment variables contains some disadvantages:

• It stores plaint-text variables which easy to see its value. You had an option to encrypt variables in the console using KMS, but it still fetching per innovation causes increase your bill.
• Hard to share across projects and teams that add complexity to your applications and services. More complexity requires more time to operate and increase the cost, therefore, you won’t meed the conditional for the Operational Excellence pillar of the Well-Architected framework.
• As per Yan Cui, it hards for implementing fine-grained access to sensitive data.

What is AWS Systems Manager Parameter Store (aka SSM Parameter Store)?

AWS Systems Manager Parameter Store provides secure, hierarchical storage for configuration data management and secrets management. You can store data such as passwords, database strings, and license codes as parameter values. You can store values as plain text or encrypted data. You can then reference values by using the unique name that you specified when you created the parameter. Highly scalable, available, and durable, Parameter Store is backed by the AWS Cloud. ~AWS

What are the benefits?

There are a lot of benefits when using AWS SSM Parameter Store, I just copied those from the AWS documentation:

• Use a secure, scalable, hosted secrets management service with no servers to manage.
• Improve your security posture by separating your data from your code.
• Store configuration data and secure strings in hierarchies and track versions.
• Control and audit access at granular levels.
• Configure change notifications and trigger automated actions for both parameters and parameter policies.
• Tag parameters individually, and then secure access from different levels, including operational, parameter, Amazon EC2 tag, and path levels.
• Reference AWS Secrets Manager secrets by using Parameter Store parameters.
• Use Parameter Store parameters with other Systems Manager capabilities and AWS services to retrieve secrets and configuration data from a central store.
• Configure integration with the AWS services for encryption, notification, monitoring, and auditing.

So now you understand what is the SSM parameter store and its challenges, let talk about how we use it by reviewing the following our best practices and tips:

#1 — Organizing parameters into hierarchies

AWS provides detailed instructions on how to organize your SSM Parameter Store to define and manage parameters easily. Following its best practices can help you and make your life easier. Below are a couple of formats/conventions that our team normally using:

• /environment/service-name/type/application-name/parameter_name i.e. /prod/billing/databases/invoicing-portal/db_connection_string
• You also can add your department name as well i.e. /prod/human-resource/employee/user_list

#2 — Consistent naming convention

Using a well-defined hierarchy helps you to manage and retrieve parameters more efficiently, but you also need to use a consistent naming convention across your AWS account, your departments, and your teams.

By archiving this best practice, it reduces your reviewing efforts by focusing on critical business logic rather than syntax and naming standards and then increase your productivity and quality which can increase your customer satisfaction.

#3 — Restrict IAM permission

AWS SSM Parameter Store normally keeps your sensitive information, so restrict permissions are required to improve your security of the application. Each Parameter Store has a unique Resource ARN per account and region, so you can easier to define role and policy base on the hierarchy of the parameter store.

Below is a sample code from the AWS official document shows how to define a policy to restrict access to the Parameter Store

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ssm:*"
            ],
            "Resource": "arn:aws:ssm:us-east-2::parameter/*"
        },
        {
            "Effect": "Deny",
            "Action": [
                "ssm:GetParametersByPath"
            ],
            "Condition": {
                "StringEquals": {
                    "ssm:Recursive": [
                        "true"
                    ]
                }
            },
            "Resource": "arn:aws:ssm:us-east-2:123456789012:parameter/Dev/ERP/Oracle/*"
        },
        {
            "Effect": "Deny",
            "Action": [
                "ssm:PutParameter"
            ],
            "Condition": {
                "StringEquals": {
                    "ssm:Overwrite": [
                        "false"
                    ]
                }
            },
            "Resource": "arn:aws:ssm:us-east-2:123456789012:parameter/*"
        }
    ]
}

#4 — Combine into a single parameter likes database connection string and keep it all together (co-location)

By using consistency hierarchies and naming convention you can archive this idea. Keep all related parameters all together makes easy to find and retrieve. Using fewer parameters can reduce your bills.

Instead of using 4 separated parameters for the database connection string as below:

/{env}/{service}/databases/master/host = db.domain.com
                               /user = username
                               /password = password
                               /port = 3306

We combine into a single parameter using a standard connection string format:

/{env}/{service}/databases/master/db_connection = jdbc://username:password@db_host:port/database_name

Using community library such as connection-string-parser, you can easy to parse the parameter values and use to open connection, see below code snippet:

import { createConnection as createConnectionPromise, Connection } from 'promise-mysql';
import { ConnectionStringParser } from 'connection-string-parser';

const parseConnectionString = (dialect: string, connectionUri: string) => {
  const connectionParser = new ConnectionStringParser({
    scheme: dialect || 'mysql',
    hosts: []
  });
  const connectionStrParams = connectionParser.parse(connectionUri);

  return {
    host: connectionStrParams.hosts[0].host,
    port: connectionStrParams.hosts[0].port || 3306,
    database: connectionStrParams.endpoint,
    user: connectionStrParams.username,
    password: connectionStrParams.password
  };
};

export const createConnection = (connectionUri: string): Promise<Connection> => {
  return createConnectionPromise(parseConnectionString('mysql', connectionUri));
};

#5 — Use tool/library to fetch, cache, and export to environment variables at runtime

You are charged for API integration to SSM parameters, every time you retrieve your parameter from the store, you will increase your bill. What could you do to reduce your billing?

By default, max throughput (transactions per second) to retrieve parameter via API is 1000, how do you manage and avoid exceed throughput error?

@Yan Cui wrote an article describes reasons why you should use AWS SSM Parameter Store over Lambda environment variables, he also mentioned approaches for caching and cache expiration using his custom client library.

Our team is using middy middleware to deal with some cross-cutting concerns outside business logic, like input parsing and validation, output serialization, error handling. Application configuration also an aspect that every developer needs to work out and manage to run business logic. Out of the box, middy provides ssm middleware support fetch and cache parameters from the AWS SSM Parameter Store, it also supports assign parameter values to environment variables.

Here is a sample code on how to use middy to fetch and cache parameter store

const middy = require('middy');
const { ssm } = require('middy/middlewares');

export const handler = middy((event, context, cb) => {
  // You can access the parameter value inside function handler
  console.log(process.env.HARVESTAR_PCMSS_DB_CONNECTION);

  // Your business logic here
}).use(
  ssm({
    cache: true,
    names: {
      // Should have a prefix that include this micro service i.e. pcmss
      HARVESTAR_PCMSS_DB_CONNECTION: '/dev/harvestar/pcmss/db_connection'
    }
  })
);

There are some alternative open-sourced libraries out there:

• aws-parameter-cache
• ssm-cache-python

Do you really trust the community package?

I head from some people, basically, they don’t want to assign variables into environment variables (i.e. variables you can access through the process.env global object in Node.js runtime). If you do so, I have some advice as below:

• Instead of assign variables to environment variables, you also have another option to assign the context object of the AWS Lambda when using middy/ssm middleware.
• To avoid sending your sensitive information such as data credentials, accessing to the /tmp directory, or running a child process when executing your serverless functions. You can use @puresec/function-shield library. We are also using it in our production environment.

TIPS — Avoid fetching parameters at build/deploy time, fetch it at runtime instead. If you do so, you have to redeploy each time the parameter changed.

#6 — Using hardcoded environment variables for your local development

Do you need to run your function locally that fetches AWS SSM Parameter Store directly? The answer is it is optional, for your local environment, you might not need to use AWS SSM Parameter Store, you can use a .env file to keep your local variables. Below are some approaches you can use to archive that idea, note you still need to test your function with your desired approach on AWS environment:

• Use the env-cmd library to load, extract and assign to process.env global object. By running env-cmd serverless offline command, you can access all variables defined in your .env file.
• Using the serverless-secrets-plugin to define the environment variable in a secured manner, you can easier to share across the team and commit the encrypted file.

Using the same code as below with a modification, you can skip fetching parameter store from AWS and reduce your bill:

const middy = require('middy');
const { ssm } = require('middy/middlewares');

const isLocalEnv = process.env.IS_OFFLINE || process.env.IS_LOCAL;

export const handler = middy((event, context, cb) => {
  // You can access the parameter value inside function handler
  console.log(process.env.HARVESTAR_PCMSS_DB_CONNECTION);

  // Your business logic here
}).use(
  ssm({
    cache: true,
    // By setting the paramsLoaded, you tell the middleware to
    // not fetch it from AWS SSM
    paramsLoaded: isLocalEnv,
    names: {
      // Should have a prefix that include this micro service i.e. pcmss
      HARVESTAR_PCMSS_DB_CONNECTION: '/dev/harvestar/pcmss/db_connection'
    }
  })
);

#7 — Pay attention to services limits

Likes other AWS services, AWS SSM Parameter Store also has some limits, such as the maximum number of params per account and region, max param value size, max history. Understanding its limits help us design and build applications with high reliability. For example, avoid storing large items into the SSM parameter because of size limits (4KB for standard and 8KB for the advanced parameter). Refer to AWS service limits documentation for more others.

Resources

• You should use SSM Parameter Store over Lambda env variables
• Sharing Secrets with AWS Lambda Using AWS Systems Manager Parameter Store
• Managing Secrets Using AWS Systems Manager Parameter Store and IAM Roles

Conclusion

By applying best practices, you can implement your applications with more reliable, secure, efficient, and cost-effective software on the cloud.

I hope this post brings some ideas to you and saves time. There are more interesting and useful articles, so find and read them to get more information. Feel free to let me know your recommendations or suggestions by adding comments below.

Thank you for reading!

This post appears first on our blog.

07 best practices when using AWS SSM Parameter Store was originally published in INNOMIZE on Medium, where people are continuing the conversation by highlighting and responding to this story.

I have been working on various cloud computing projects from start to finish; involved in the full project lifecycle, including design, planning, development, deployment, testing, maintenance, and support. Some clients prefer to use their on-premise data centers, but most of my projects used the cloud — Amazon Web Services, Microsoft Azure, and Heroku. So, I wanted to write this post to share my cloud computing journey. My hope is to encourage you to build secure, scalable, highly available, and cost-effective cloud applications. Learn to share and share to learn will help us all grow, don’t you agree?

If you are new to the cloud and wanted to explore more information, check out some articles below:

• Why are so many businesses are moving to the cloud?
• Who Coined ‘Cloud Computing’?
• History of Cloud computing
• Check out 15 Top Cloud Computing Services Provider Companies

This is a lengthy article. So, if you prefer to simply view a particular topic, you can use the table of contents below.

• Who am I?
• Starting my career as a software developer
• The beginning of my cloud computing journey
• My startup journey is just started
• My own company is officially established
• We got the first AWS certification
• A few more serverless projects
• Our most recent cloud computing projects
• Other works
• My future plan

Who am I?

Firstly, I’d like to introduce myself. My name is Hoang, I am the Co-founder and CTO of InnomizeTech. My title is CTO but I am a full-stack developer and software architect, passionate about Cloud Computing, Serverless, DevOps, Machine Learning, and IoT.

With 7+ years of experience in software development (web, desktop, and mobile), systems integration, Cloud computing, DevOps engineering, project management, and agile methodology, I thrive working both independently and collaboratively in a team and have strong leadership and communication skills from working directly with both technical and non-technical stakeholders.

Starting my career as a software developer

Before transitioning to Cloud computing, I worked for an outsourcing company in Vietnam for four years after I graduated from University.

• For the first two years, the primary programming language I used was C# (AWS .NET WebForm and AWS .NET MVC).
• For the next year, I transitioned to another role, leading a team working with system integration stuff using WSO2 and BizTalk, among other tools. I had a chance to work with our client’s CTO. We had a lot of fun and we did a great job.

The beginning of my cloud computing journey

In the third year, I started my first Cloud computing project. We had a project from a US-based client to build an application for activating mobile devices for various carriers. I was the technical lead of that project. After studying the requirements, we selected the following technology stack:

• Node.js for back-end, using the Sails.js framework
• MySQL database and using Sequelize for ORM. We used Liquibase to generate, and migrate the database schema using source control.
• Angular for the front-end application

Actually, it isn’t a real cloud-based application, since we used only a few cloud services to build the app. We have built the app and used the Elastic Beanstalk service for hosting. I have studied Jenkins and built a Jenkins job for analyzing code, building, testing, and deploying to the Elastic Beanstalk app. Here is the diagram of the CI/CD I built at that time:

We initially hosted the application on Elastic Beanstalk apps in order to apply our CI/CD script using AWS Beanstalk plugin on Jenkins for deployment. However, our client wanted to host it in their data center. Then we still use Beanstalk for our SIT/UAT environments, but we had to write documentation of around 10 pages to describe how to set up Web Servers, App Servers, NGINX, Node.js, RDS, etc. A lot of back-and-forth communication between our development team and infra team because of issues and errors. But finally, we have successfully deployed, and the app is up and running.

We kept working on that project for around a year. While developing new features, I also learned Ansible and built some playbooks to allow my client to deploy the app automatically without manual steps.

• Continuous Deployment — Deploying A Node.Js App To AWS EC2 Using Ansible

At that time, I and my co-workers established a technical club to study and share about cloud computing. We learning cloud computing concepts and doing some hands-on exercises with AWS and Microsoft Azure. You can visit our Facebook page to see our activities and great memories.

Before leaving the company, I studied and shared my CI/CD experiences. I have presented this presentation for my promotion to the Senior Engineer position (I have changed a few things to share with everyone).

My startup journey is just started

In August 2017, I started my own job. We had a project for building a core/shared backend API for various client applications including SPA, Mobile App, and Integration services. After clarifying requirements with our clients, we have selected the below technology stack:

• AWS Lambda uses Node.js runtime and Amazon API Gateway that utilize Auth, Caching, Logging, Monitoring Throttling, and Bursting Elastic features for backend services.
• We use the Serverless framework for building API, deploying and configuring AWS resources such as S3 bucket, SNS topic, Queue, and DynamoDB. At this time, we used JavaScript as the primary programming language.
• Amazon Cognito User Pool for authentication and authorization.
• Amazon Aurora for the relational database. We keep using Sequelize and Liquibase.
• Amazon Elastic Cache (Redis) for in-memory caching.
• Other AWS services we have used such as VPC, S3, SNS, SQS, CloudWatch, Route53, Certification Manager, AWS System Manager, AWS Parameter Store, X-Ray
• Angular for the frontend app. Static hosting with S3 and CloudFront.
• Jasmine, Protractor for unit test, integration test, and E2E test.
• We still use Jenkins for our DevOps server. But instead of defining the job manually on the admin console page, we wrote a Jenkinsfile that used the Mutliplebranch pipeline.
• And more…

We kept applying the same stack to a few projects for other clients and continuously improve and refactored our core modules/libraries. We have delivered and received positive feedback from our clients. The applications were running well, had lower costs, high availability, and were secure.

In relation to DevOps, we have all the stuff scripted and provided CLIs to allow building and deploying applications automatically. We’ve used CloudFormation to provision and configure infrastructure components such as VPC, RDS. We outlined steps to deploy/remove and send it over to our client, DevOps engineers, they can deploy a new environment using a few commands and then remove it completely without manual steps. Automation reduces human mistakes and increases productivity and quality.

My own company is officially established

In October 2018, we officially established InnomizeTech, our mission is to create awesome products that will make your everyday life easier. My aim is to build a professional team with passion, enthusiasm, and talent that can help bring more value to our customers and help them move fast and in the right direction.

The experienced engineering and development team at Innomize are committed, and passionate, and continually challenge themselves on the advancements and changes of the technologies we utilize to deliver the most up-to-date and innovative solutions.

We got the first AWS certification

I and my team have been learning, and building a lot of real-world cloud-based applications for many years. To prove to our clients that we had the experience to do their jobs, we studied and took some AWS exams and finally, our team got AWS certifications:

• 02 AWS Certified Solution Architecture — Associate
• 02 AWS Cloud Practioner
• 01 AWS Certified Developer — Associate

We have been with The Coral Edge our partner and primary client. Recently, they officially became the Select Consulting Partner with AWS and we are their Development Team.

A few more serverless projects

In early 2019, we had a contract with another client from the US, we will help their development team maintain and build new features for their micro-services (the integration component for their e-commerce system). They used Scala as their programming language, and Gradle as the build tool, their developer implemented some Gradle tasks to deploy the Lambda function, create the API gateway, and other required AWS resources. Here is the technology stack I can summarize:

• Each microservice has its own Lambda function and API Gateway as the original design. They had an NGINX server as a reserved proxy server that will route traffic to AWS API gateways based on the request path.
• http4s as the interface for Restful API.
• They used the Postgres database and also using Liquibase to manage their database schema and versioning. They used Doobie for functional JDBC for Scala.
• Sentry for error monitoring and reporting.
• They do use AWS CodeBuild, AWS CodeDeploy, and AWS CodePipeline to implement CI/CD but it isn’t fully implemented when we join their team.

When our developers work with their team, we have proposed some approaches to enhance and refactor their design:

• To remove their NGINX server and cost, we suggest they apply the Robust API pattern.
• We helped design and built a project template in order to add a new microservice easily. Here are some of our ideas:
• Defining a project template that uses AWS CodeStart which will allow us to create CodeCommit repositories, and define CodeBuild projects, and CodePipelines automatically when creating a new project. New microservices can be up and running by using a single command.
• We suggest using the AWS SAM framework for building Lambda functions and defining required AWS resources (i.e. Queue, Bucket, Topic, etc.). Previously, they deployed Lambda and other resources by implementing custom Gradle tasks.

Our most recent cloud computing projects

Recently, we have been working on some projects that apply Micro-service architecture. We continue the above technology stacks with some changes/improvements:

• We used NX Dev Tool to define our workspace on a Monorepo. We built our CI/CD pipeline that supports deploying only affected apps by a change instead of redeploying all services. Thanks to the NX team for their idea and awesome tool.
• Step Functions for our micro-service communication, automated workflow.
• NestJS for our Restful API that running on AWS Lambda.
• AWS CDK to provision and configure our infrastructure as code (IoC)
• Jest, Cypress for unit test, integration test, and E2E testing.
• Along with using Cognito we also use Okta, AuthO for authentication and authorization.
• CI/CD with AWS Develop Tools, Jenkins, Azure DevOps.
• Along with using Node.js runtime, we also use Java/Scala and .NET Core runtime.

Other works

We focused on AWS as the primary cloud vendor and wanted to become an active development service by partnering with them. Along with cloud computing, our team also has experience in other technologies. We work with the latest technologies specializing in Web Development, Mobile Development, Cloud Computing (AWS, Azure), DevOps, and Automation Testing. Refer to our Services section on our website for more information.

We continue working on some other projects such as:

• DevOps using CloudFormation, AWS CDK, Ansible
• Amplify framework: hosting, authentication, Mobile App with AppSync.
• Ruby on Rails app on AWS ECS.
• Headless upload bills to various sites with AWS ECS and Protractor (End to end automation test with Protractor).
• Serverless web crawler with Puppeteer on AWS Fargate that I recently wrote that post.
• React, Amplify, AppSync.
• Work with other AWS services such as AWS WAF, AWS Shield, Glue, ElasticSearch, Kinesis, and Athena.
• We helped some clients design and define DevOps, and CI/CD pipelines using AWS Developer Tools and Azure DevOps.

My future plan

My quote is “Learn to share and share to learn”, I am continually challenging myself on the advancements and changes of the technologies. I wanted to meet more people, talk, work, and help them. I do have a few things on my list so I am going to do:

• Build Innomize stronger, work with more clients, and collaborate with more people.
• Learn more things such as Machine Learning, Deep Learning, Transfer Learning, Convolution Neural Networks, and IoT.
• Build my own product.

I cannot describe anything about myself in this post, if you wanted to know more about me and our team, then you can get in touch with me via:

• Twitter
• LinkedIn
• DEV Community
• Github
• Medium
• Email

If you are looking for developers, an offshore team, or need consulting about the AWS cloud, Serverless architecture, and so on, then get in touch with us, we can help you!

Thank you for reading!

My Cloud computing journey was originally published in INNOMIZE on Medium, where people are continuing the conversation by highlighting and responding to this story.

In the previous part, I have shown how to build a sample mockup API for CRUD application as well as build a sample list page. In this part, we will continue to build CRUD application on creating, updating and deleting an entity using Ngrx.

Before continuing, I want to introduce to you a special thing is “Optimistic UI”.

What is Optimistic UI?

Optimistic UI is a pattern that you can use to simulate the results of a mutation and update the UI even before receiving a response from the server. Once the response is received from the server, the optimistic result is thrown away and replaced with the actual result ~ Apollo.

Let’s remember the previous approach for handling update operation (I call this approach is Synchronous UI):

• The end-user click update action.
• The end-user click update action.
• Make an HTTP request for the update operation.
• Displays a spinner (or loading image) for waiting for the success response from the server.
• After the server responses successfully, we continue to refresh data on UI to reflect with the updated result for consistency. Otherwise failed, display an error message.

And if we apply the Optimistic UI approach:

• The end-user click update action.
• Make an HTTP request for updating operation and the UI value is updated immediately assuming the response will be successful.
• After the server responses successfully, we do not anything. Otherwise failed, revert the UI value as the previous state.

As you see two above approaches, the Synchronous UI approach needs a step that showing a spinner for waiting for the server’s response and the UI just is updated after the server is responded. In most cases, there is no reason to wait for a successful response when we assume that all requests are expected to be successful. Like that, the Optimistic UI approach brings to the end-user a faster user experience, more smooth and stable.

Now, we will go dive to implement CRUD application with optimistic interaction.

As part 1, we have built the CRUD application to read Entity from the server and display it using Ngrx. Before we implement the update Entity feature, we need update code for some files as preparation.

Update the entity.service.ts file as:

Update entities.page.html to add some action links:

Update Entity

First, we update entity.actions.ts file to add some actions as:

Next, update the entity.reducer.ts file as:

Update the entity.effects.ts file as:

And update the entity.selectors.ts file as:

As you can see above the action file, we define an additional action is “Load Entity” for reading Entity detail from the server or the available store and display it for the user to edit. If you click edit on List of Entities page, just load data from the store that is loaded before. In case the user pastes the editing Entity page URL direct to the browser’s address, an HTTP request will be called to retrieving Entity detail instead of.

In the reducer file, you will see the “updateEntity” action will update immediately the state reflects with the updated entity even the server does not respond yet. And we need an action to revert the store to the previous state for case unsuccessful response. I used ngrx-undo to handle this case, the ngrx-undo provides a wrapper function to revert a specified action with the simple configuration, you can see I have used this library in the entity.effects.ts file for error handling section. To install this library you can refer to this link.

Ok, we will develop a page for Entity editing as below:

And don’t forget to define the route for Entity editing page:

{

    path: "edit/:id",

    component: EntityPage,

    resolve: {

        entity: EntityResolver

    }

}

Gives the result as:

Create an Entity

Now to create the Entity in the server, we need to perform HTTP POST operation and pass the data that needs to be inserted. We will take a “createEntity” action for that.

Update reducer and effect files to add some code as below:

And we can reuse the Entity editing page for creation operation, update code the entity.page.ts file as below:

Delete Entity

Similarly, we only need to define “deleteEntity” action and call the action on the delete method that we have added before:

Implement the delete method as below:

Done, enjoy the result as below:

Summary

Coming to the end of this guide, let us have a quick recap of what we did.

• Use Optimistic UI approach to make the application faster and smoother.
• Use the ngrx-undo library for reverting the store as the previous state on the unsuccessful response case.

Thank for reading!

Building a CRUD application with Angular using Ngrx (Part 2) was originally published in INNOMIZE on Medium, where people are continuing the conversation by highlighting and responding to this story.

Chart.js is an open-source JavaScript library which makes it very easy to build charts for your website. On this tutorial, I’d like to provide step by step guide how to combine chart.js with another angular package - ‘ng2-charts’.

Firstly, we need to understand about some properties of chart.js:

• data: Set of points of the chart, it should be MultiDataSet only for line, bar, radar and doughnut, otherwise SingleDataSet
• datasets: Data see about, the label for the dataset which appears in the legend and tooltips
• labels: x-axis labels. It’s necessary for charts: line, bar, and radar. And just labels (on hover) for charts: polarArea, pie, and a doughnut. Label is either a single string, or it may be a string[] representing a multi-line label where each array element is on a new line.
• type: Indicates the type of charts, it can be: line, bar, radar, pie…
• options: Chart options. See Chart.js documentation for more details.
• colors: Data colors, will use the default and|or random colors if not specified
• legend: If true show legend below the chart, otherwise not be shown

Let’s start to create a sample chart

1. We need to create an angular application with Angular CLI:

$ ng new myChart — routing

• This command is downloading the default Angular project template and all dependencies are being installed.

2. Install ng2-charts and chart.js dependency. Go to your root project and run commands

npm install ng2-charts — save

npm install chart.js — save

Now, you need to import ChartsModule into your app module or feature module. In this example, I will import to AppModule

import { BrowserModule } from "@angular/platform-browser";

import { NgModule } from "@angular/core";

import { ChartsModule } from "ng2-charts";

import { AppComponent } from "./app.component";

@NgModule({

  declarations: [AppComponent],

  imports: [BrowserModule, ChartsModule],

  providers: [],

  bootstrap: [AppComponent]

})

export class AppModule {}

ng2-charts chart types

There is one directive for all chart types: baseChart, and 8 types of charts: line, bar, radar, pie, polarArea, doughnut, bubble and scatter.

Let’s build a sample bar chart using below and update your app.component.html

<div class="chart-container" style="display: block">

<canvas baseChart

   [datasets]="barChartData"

   [labels]="barChartLabels"

   [options]="barChartOptions"

   [legend]="barChartLegend"

   [chartType]="barChartType">

</canvas>

</div>

And the last step, we need to init chart data for the chart. Update your app.component.ts as below

import { Component } from '@angular/core';

@Component({

  selector: 'app-root',

  templateUrl: './app.component.html',

  styleUrls: ['./app.component.scss']

})

export class AppComponent {

  constructor() {}

  public barChartOptions = {

    scaleShowVerticalLines: false,

    responsive: true

};

  public barChartLabels =['2006','2007','2008','2009','2010','2011','2012'];

  public barChartType = 'bar';

  public barChartLegend = true;

  public barChartData = [

   { data: [65, 59, 80, 81, 56, 55, 40], label: 'Series A' },

   { data: [28, 48, 40, 19, 86, 27, 90], label: 'Series B' }

  ];

}

Start your application and view your result in http://localhost:4200

$ ng serve

Noted: Detecting when the canvas size changes can not be done directly from the canvas element. Chart.js uses its parent container to update the canvas render and display sizes.

Build your first Chart using Angular & Ng2-charts was originally published in INNOMIZE on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this guide, we just focus on how to build a CRUD application with Angular (version 8) that has already installed the Ngrx, not on setting up an Angular application or installing the Ngrx library. You can follow the following guides to do those task:

• Set up an Angular application: https://codeburst.io/how-to-build-an-angular-app-with-angular-cli-in-a-couple-of-minutes-43089d3ab272
• Install the Ngrx library: https://ngrx.io/guide/store/install

If you have no idea what is Ngrx, I recommend you to read this to explore more about it. And if you have a basic knowledge of Redux, I think you will be easy to approach this library.

In this part, we will learn how to define actions, reducer, selectors and build a page to display all items that are selected from the store using the Ngrx.

In almost any application that we work with, we all encounter operations such as retrieving data, rendering it as a table view, create/update or delete a row on that view. And this data which is stored on the server is dealt with by the usage of HTTP requests (GET, POST, etc).

The first thing we need to do in here is where the data is stored that we will get through an HTTP API request. Because this article does not focus on this, I will use a fake backend server for mock. There are different ways to use a fake back-end server are:

• Use Angular In-Memory-Web-API
• Create a local JSON file and use it
• Or hard-code the data, and return this data.

But I have another easy way, say big thanks to https://getsandbox.com. Just provide some rows of code, we already have a RESTful API for mock or testing. I have created a mock RESTful API on https://getsandbox.com for CRUD on the Entity object as below:

Fake RESTful APIs: (CRUD)
GET:    https://entity.getsandbox.com/entities/     (Get all)
GET:    https://entity.getsandbox.com/entities/{id} (Get one)
POST:   https://entity.getsandbox.com/entities/     (Create)
PUT:    https://entity.getsandbox.com/entities/{id} (Update)
DELETE: https://entity.getsandbox.com/entities/{id} (Delete)

Ok, we have done on prerequisite items. Next, we’ll go together to define actions, reducer, and selectors to fetch Entity objects from the fake server I have created before and store it into the Ngrx store. If you have developed with a React Redux application before, all fundamental principles about Redux (action, reducer, selector, etc) will be similar to the Ngrx.

First, we will define a class that describes all properties of the Entity object we will deal with:

Let’s define some actions for loading entities as:

Next, we’ll define the Entity’s reducer. Here, we have to define an initial state and specify how the state is changed for each action is sent to the store.

Continue define selectors:

Create a service file to fetch data from the server:

To read data from the server, we need to create a side effect to make an HTTP request to the server and store data into the store through Ngrx actions. With React, Saga can handle side effects for us. And the Ngrx also has provided the Effect concept that is the same as the Saga. As the Ngrx documentation: “Effects are an RxJS powered side effect model for Store. Effects use streams to provide new sources of actions to reduce state based on external interactions such as network requests, web socket messages and time-based events.” So that we will use it for reading Entity data from the server.

Finally, creating an Angular page (with route is “/entities”) to display a list of the entity that are fetched from the server. For easy to navigate to entities page, we create a link for quick navigation in app.component.html:

In entities.page.ts, the store will dispatch loadEntities action and the effect will be triggered to fetch data from server and store data into the store. Since the data is loaded to the store, we only get data from the store and display it.

The entity template HTML file:

Note: Ensure that you have already imported CommonModule to use the async pipe.

Gives the result as:

The operation for reading Entity data from the server has completed. But you will see the entities page display empty page before render list of Entity objects. This is cause of waiting from the server for reading data and it should be better for the user’s experience if a spinner display instead of. That is a good suggestion since I know Angular Resolve. Resolve will wait for the data to be resolved before the page is finally navigated.

So I will define the resolver for entities page as below:

As above code, the entities page only is activated if the loaded flag is true. Also, I have added a filter operator to dispatch loadEntities action if the loaded flag is false. So the application only makes an HTTP request only the loaded flag is false, this reduces significantly backend traffic during the application navigation.

Update the entities.page.ts to comment or remove the following code:

// this.store.dispatch(fromEntityActions.loadEntities());

Continue to add the resolve to the route configuration:

Enjoy the result:

Summary

Coming to the end of this guide, let us have a quick recap of what we did.

• Use https://getsandbox.com/ to quick create a fake server for mocking and testing.
• Use createAction, createReducer, and createFeatureSelector for creating actions, reducer, selectors.
• Use Ngrx Effect for handle side effects to make an HTTP call for retrieving data from the server.
• Use Angular Resolver to ensure that data has already loaded before the page is navigated.

I hope this tutorial will be helpful to you :D

Building a CRUD application with Angular using Ngrx (Part 1) was originally published in INNOMIZE on Medium, where people are continuing the conversation by highlighting and responding to this story.