Step 1: Launch an EC2 Instance

• Log in to AWS Console
• Navigate to the AWS Console and go to EC2 > Instances.
• Click Launch Instance.
• Configure Instance Details
• Name your server (e.g., MyEC2Server).
• Choose an Amazon Machine Image (AMI):
• Amazon Linux 2023 or
• Ubuntu 22.04 LTS
• Select an Instance Type
• Choose one of the following:
• t2.micro – Free Tier (suitable for basic usage)
• t2.small – Slightly more performance
• Higher tiers if needed for performance
• Create or Use a Key Pair
• Choose an existing key pair or create a new one.
• Download the private key file (.pem or .ppk).
• Important: Save this key securely; it is required for SSH access.
• Launch the Instance
• Click Launch Instance and wait for it to start.

Step 2: Configure Security Group (Firewall Rules)

Once your instance is launched, configure the firewall settings to allow incoming traffic.

Access Security Group

• Go to EC2 > Instances.
• Select your running instance.
• Under the Security tab, click the linked Security Group name.

Edit Inbound Rules

• Click Inbound Rules, then Edit Inbound Rules.
• Click Add Rule to add the following entries:
• TCP 80 0.0.0.0/0
• TCP 4430.0.0.0/0
• TCP 56780.0.0.0/0(Optional: for n8n testing)

Save Changes

• Click Save Rules to apply the changes.

Verify Instance Status

• Go back to EC2 > Instances.
• Confirm the Instance State is Running

Step 3: Connect a Domain to Your EC2 Server

To make your application accessible via a custom domain like yourdomain.com or n8n.yourdomain.com, set up an A record in your domain's DNS settings.

Go to Your Domain Registrar

• Log in to your account on Namecheap, GoDaddy, Google Domains, etc.

Find DNS Settings

• Navigate to the DNS management section for your domain.

Create an A Record

FieldValue
Type: AHostn8n (or @ for root domain)
Value: Your EC2 Public IP (e.g., 12.61.155.33)
TTL: Auto or 5 minutes

This links your domain (e.g., n8n.yourdomain.com) to your EC2 server.

Wait for DNS Propagation

Changes usually propagate within 5 minutes to 1 hour.

Step 4: Install Docker, Docker Compose, NGINX, and Deploy n8n

Now that your EC2 server is live and your domain is connected, it’s time to install Docker and deploy n8n with NGINX and SSL.

sudo yum update -y
sudo yum install -y docker

Start and Enable Docker

sudo systemctl start docker
sudo systemctl enable docker

Add Your User to the Docker Group

sudo usermod -aG docker ec2-user
newgrp docker

Install Docker Compose

sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

Install NGINX and Certbot (SSL)

sudo dnf install -y nginx
sudo systemctl start nginx
sudo systemctl enable nginx

sudo dnf install -y certbot python3-certbot-nginx

Set Up n8n with Docker Compose

Create project directory:

mkdir ~/n8n
cd ~/n8n
nano docker-compose.yml

Paste the following into docker-compose.yml:

version: '3'
services:
  n8n:
    image: n8nio/n8n
    restart: always
    ports:
      - "5678:5678"
    environment:
      - N8N_BASIC_AUTH_ACTIVE=true
      - N8N_BASIC_AUTH_USER=admin
      - N8N_BASIC_AUTH_PASSWORD=yourStrongPasswordHere
      - WEBHOOK_URL=https://yourwebsitedomain/
    volumes:
      - n8n_data:/home/node/.n8n
volumes:
  n8n_data:

Replace yourStrongPasswordHere and yourwebsitedomain with your actual credentials and domain.

Start n8n:

docker-compose up -d

Configure NGINX as a Reverse Proxy

Create NGINX config:

sudo nano /etc/nginx/conf.d/n8n.conf

Paste the following config:

server {
    listen 80;
    server_name yourdomainaddress;

    location / {
        proxy_pass http://localhost:5678/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Replace yourdomainaddress with your actual domain (e.g., n8n.yourdomain.com)

Test and Reload NGINX

sudo nginx -t
sudo systemctl restart nginx

Secure with SSL Using Certbot

sudo certbot --nginx -d yourwebsitedomain

Thanks, For more details
Contact me at as1987137@gmail.com

Linkedin: https://www.linkedin.com/in/abdul-sami-a48b78234/

In this guide, I will cover the essentials, from setting up your first state machine to implementing error handling and choosing the right workflow type.

Step 1: Create a State Machine

• Log in to the AWS Management Console.
• Navigate to Step Functions and click Create State Machine.
• Choose a template or start from scratch.

You can select Standard Workflows for durable, long-running processes or Express Workflows for high-throughput, short-lived tasks

Step 2: Define Your Workflow

Here’s an example of a basic state machine that calls an AWS Lambda function:

{
  "StartAt": "LambdaTask",
  "States": {
    "LambdaTask": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME",
      "End": true
    }
  }
}

Step 3: Test Your Workflow

After defining the workflow, deploy it and test it with sample inputs. AWS Step Functions provide detailed execution logs to debug and optimize your state machine.

Error Handling

One of the most powerful features of AWS Step Functions is built-in error handling. You can define Retry and Catch blocks for each state to handle failures gracefully.

{
  "Type": "Task",
  "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME",
  "Retry": [
    {
      "ErrorEquals": ["States.TaskFailed"],
      "IntervalSeconds": 5,
      "MaxAttempts": 3
    }
  ],
  "Catch": [
    {
      "ErrorEquals": ["States.ALL"],
      "Next": "ErrorHandler"
    }
  ]
}

Best Practices and Real-Life Insights

• Choose the Right Workflow:
  Use Standard Workflows for processes like order processing or ETL jobs, where durability is key. Opt for Express Workflows for real-time data streaming or IoT events.
• Start with Templates:
  AWS Step Functions provide pre-built templates for common scenarios, such as integrating with AWS Lambda or processing S3 events. These templates are a great way to get started quickly.
• Handle Failures Thoughtfully:
  Adding retries and fallback logic can save your application from cascading failures.

Conclusion

AWS Step Functions bring order to the chaos of managing distributed systems. By leveraging state machines, built-in error handling, and tailored workflows, you can create scalable, reliable applications without writing excessive boilerplate code.

Setting Up RabbitMQ

First, let’s run RabbitMQ using Docker. Open your terminal and run:

docker run -d --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq

Writing the Go Application

go mod init rabbitmq-example

go get github.com/streadway/amqp

Create a Connection File

package main

import (
    "log"
    "github.com/streadway/amqp"
)

func failOnError(err error, msg string) {
    if err != nil {
        log.Fatalf("%s: %s", msg, err)
    }
}

func connectRabbitMQ() (*amqp.Connection, *amqp.Channel, amqp.Queue) {
    conn, err := amqp.Dial("amqp://guest:guest@localhost:5672/")
    failOnError(err, "Failed to connect to RabbitMQ")

    ch, err := conn.Channel()
    failOnError(err, "Failed to open a channel")

    q, err := ch.QueueDeclare(
        "hello", // name
        false,   // durable
        false,   // delete when unused
        false,   // exclusive
        false,   // no-wait
        nil,     // arguments
    )
    failOnError(err, "Failed to declare a queue")

    return conn, ch, q
}

func consumeMessages(ch *amqp.Channel, q amqp.Queue, consumerID int) {
    msgs, err := ch.Consume(
        q.Name, // queue
        "",     // consumer
        true,   // auto-ack
        false,  // exclusive
        false,  // no-local
        false,  // no-wait
        nil,    // args
    )
    failOnError(err, "Failed to register a consumer")

    for d := range msgs {
        log.Printf("Consumer %d received a message: %s", consumerID, d.Body)
    }
}

func checkQueueStatus(ch *amqp.Channel, queueName string) {
    q, err := ch.QueueInspect(queueName)
    failOnError(err, "Failed to inspect queue")

    log.Printf("Queue %s has %d messages and %d consumers", q.Name, q.Messages, q.Consumers)
}

Create the Main Application File:

package main

import (
    "log"
    "time"
)

func main() {
    conn, ch, q := connectRabbitMQ()
    defer conn.Close()
    defer ch.Close()

    numConsumers := 3
    for i := 1; i <= numConsumers; i++ {
        go consumeMessages(ch, q, i)
    }

    // Periodically check the status of the queue
    go func() {
        for {
            checkQueueStatus(ch, q.Name)
            time.Sleep(10 * time.Second) // Check every 10 seconds
        }
    }()

    log.Printf(" [*] Waiting for messages. To exit press CTRL+C")
    select {}
}

Running the Application

go run main.go rabbitmq.go

This will start three consumers that listen to the hello queue and log any messages they receive. Additionally, it will periodically log the status of the queue, including the number of messages and consumers.

RabbitMq and Go. Creating Push and Assignment System using Golang was originally published in DevOps.dev on Medium, where people are continuing the conversation by highlighting and responding to this story.

These are the following options found while reading the flag library …

1. Arg() and Args() functions return command line arguments

here is the below code to explain you

verbose := flag.Bool("v", false, "Verbose Mode Activated")

 // Int Flag Created
 n := flag.Int("i", 1, "Number of times")
 flag.Parse()

 args := flag.Args()

 name := "World"
 if len(args) > 0 {
  name = args[0]
 }

 if *verbose {
  name = "VERBOSE " + name
 }

 fmt.Printf("Hello, %s!\n", name)
 for i := 0; i < *n; i++ {
  fmt.Print(name)
 }

2. Bool and Boolfunc , both output a bool value but the latter takes a bool value as the output of a function

// Bool 
verbose := flag.Bool("v", false, "Enable verbose mode")

//Boolfunc 
isVerbose := flag.BoolFunc("v", "Enable verbose mode", func() bool {
    
    return true
})

3. Duration and DurationValue are both time and duration related functions in golang flag library, below is the implementation so you can understand

// Implementation for Duration
timeout := flag.Duration("timeout", 5*time.Second, "Timeout for the operation")
    flag.Parse()

    fmt.Printf("Timeout set to: %v\n", *timeout)

// Implementation for   DurationVar
var timeout time.Duration
    flag.DurationVar(&timeout, "timeout", 5*time.Second, "Timeout for the operation")
    flag.Parse()

    fmt.Printf("Timeout set to: %v\n", timeout)

4. Float64 and Float64Var

The Float64 function creates a flag "threshold" with a default value of 0.5, and Float64Var uses an existing variable "customThreshold" with a default value of 1.5.

// Float64
threshold := flag.Float64("threshold", 0.5, "Set the threshold value")

// Float64Var
var customThreshold float64
flag.Float64Var(&customThreshold, "customThreshold", 1.5, "Set a custom threshold value")

// Usage:
fmt.Println("Threshold:", *threshold)
fmt.Println("Custom Threshold:", customThreshold)

5. Func Function

The Func function defines a "config" flag, calling a function for validation. If the validation is successful, it sets the configPath variable

// Func
var configPath string
flag.Func("config", "Path to config file", func(value string) error {
    if isValidConfig(value) {
        configPath = value
        return nil
    }
    return fmt.Errorf("invalid config file path")
})

// Usage:
fmt.Println("Config Path:", configPath)

6. Float64 Flags

// Original Example
verbose := flag.Bool("v", false, "Verbose Mode Activated")
n := flag.Int("i", 1, "Number of times")
flag.Parse()
args := flag.Args()
name := "World"
if len(args) > 0 {
    name = args[0]
}
if *verbose {
    name = "VERBOSE " + name
}
fmt.Printf("Hello, %s!\n", name)
for i := 0; i < *n; i++ {
    fmt.Print(name)
}

Check out insightful articles on tech and programming by Me. Dive into engaging content that demystifies complex topics and enhances your coding journey. Don’t miss out on a blend of expertise and clarity — follow for regular updates!

Writing CLI Tools in Golang(with Native Libraries) was originally published in DevOps.dev on Medium, where people are continuing the conversation by highlighting and responding to this story.

In Linux System, You have multiple Log and Management Files.Let Suppose you want to extract some data from that file.You cannot do directly.

Let’s Suppose this is log file

03/22 08:51:06 TRACE  :...read_physical_netif: Home list entries returned = 7
03/22 08:51:06 INFO   :...read_physical_netif: index #0, interface VLINK1 has address 129.1.1.1, ifidx 0
03/22 08:51:06 INFO   :...read_physical_netif: index #1, interface TR1 has address 9.37.65.139, ifidx 1
03/22 08:51:06 INFO   :...read_physical_netif: index #2, interface LINK11 has address 9.67.100.1, ifidx 2
03/22 08:51:06 INFO   :...read_physical_netif: index #3, interface LINK12 has address 9.67.101.1, ifidx 3
03/22 08:51:06 INFO   :...read_physical_netif: index #4, interface CTCD0 has address 9.67.116.98, ifidx 4
03/22 08:51:06 INFO   :...read_physical_netif: index #5, interface CTCD2 has address 9.67.117.98, ifidx 5
03/22 08:51:06 INFO   :...read_physical_netif: index #6, interface LOOPBACK has address 127.0.0.1, ifidx 0
03/22 08:51:06 INFO   :....mailslot_create: creating mailslot for timer
03/22 08:51:06 INFO   :...mailbox_register: mailbox allocated for timer

if you want to get specific line that command would be easy

head -n 5 app.log 
# where 5 is the Line No and app.log is the File

But you cannot get only date or log number.
We have to Use AWK for this purpose

Print Full File

awk '{print}' app.log

Print One to Multiple Columns

awk '{print $1,$2}' app.log

Alternative Columns

awk '{print $1,$4}' app.log

Filter on Column

awk '/INFO/ {print $1,$2}' app.log

#Here INFO is the Filter in it 

Save to Specific log Other File

awk '/INFO/ {print $1,$2}' app.log > newlog.log

Count Specific Column Enteries

awk '/INFO/ {count++} END {print count}' app.log

Add Random Line to Count

awk '/INFO/ {count++} END {print "the count is", count}' app.log 

Real World Scenario

Let's assume, we have a log file of ip errors, we want to check, how many times specific IPs face Errors

awk '/192.168.1.1/ {count++} END {print "the count is", count}' app.log

Matching Time in log file

awk '$2 >= "08:51:06" && $2 <= "08:51:06" {print}' app.log

Are You Looking to Read Other Related Medium Articles

https://medium.com/@as1987137/spinning-up-multiple-databases-with-docker-mysql-cassandra-mariadb-and-postgresql-ea0ea356bd1a

https://medium.com/@as1987137/building-a-real-time-chat-application-with-websockets-using-aws-lambda-api-gateway-and-dynamodb-5c01399f1510

Awk is used where we have formatted and Arranged data like CSV or Column Values.

Understanding AWK command in the Linux Operating System. was originally published in DevOps.dev on Medium, where people are continuing the conversation by highlighting and responding to this story.

Setting Up Docker Containers

We’ll create Docker containers for each of the databases we want to run. Here’s how to do it:

1. MySQL

docker run -d --name mysql-container -e MYSQL_ROOT_PASSWORD=root_password -p 3306:3306 mysql

2. Cassandra

docker run -d --name cassandra-container -p 9042:9042 cassandra

3. MariaDB

docker run -d --name mariadb-container -e MYSQL_ROOT_PASSWORD=root_password -p 3307:3306 mariadb

4. PostgreSQL

docker run -d --name postgres-container -e POSTGRES_PASSWORD=root_password -p 5432:5432 postgres

Check Other Code and Implementation

https://github.com/AbdulSami455/Ec2_Database_Containers

Here is the Full Code :

GitHub - AbdulSami455/Real-Time-Chat-Application-With-WebSockets-APIGateway-Lambda-DynamoDb: Real Time Chat Application with WebSockets (AWS Services--Aws Lambda,API-Gateway,DynamoDb)

Step 1 :

Create DynamoDb

Step 2:

Set up Lambda Functions

Three Functions :

Function for Messaging

import json
import boto3
import os

dynamodb = boto3.client('dynamodb')


def lambda_handler(event, context):
    message = json.loads(event['body'])['message']
    paginator = dynamodb.get_paginator('scan')
    connectionIds = []

    apigatewaymanagementapi = boto3.client(
        'apigatewaymanagementapi',
        endpoint_url = "https://" + event["requestContext"]["domainName"] + "/" + event["requestContext"]["stage"]
    )

    for page in paginator.paginate(TableName=os.environ['WEBSOCKET_TABLE']):
        connectionIds.extend(page['Items'])

    for connectionId in connectionIds:
        apigatewaymanagementapi.post_to_connection(
            Data=message,
            ConnectionId=connectionId['connectionId']['S']
        )

    return {}

Function for Connection

#These all are Module Required for Working
import json
import boto3
import os

dynamodb = boto3.client('dynamodb')


#Main Lambda Function
def lambda_handler(event, context):
    connectionId = event['requestContext']['connectionId']

    dynamodb.put_item(
        TableName=os.environ['WEBSOCKET_TABLE'],
        Item={'connectionId': {'S': connectionId}}
    )

    return {}

Function for Disconnection

import json
import boto3
import os

dynamodb = boto3.client('dynamodb')

def lambda_handler(event, context):
    connectionId = event['requestContext']['connectionId']

    dynamodb.delete_item(
        TableName=os.environ['WEBSOCKET_TABLE'],
        Key={'connectionId': {'S': connectionId}}
    )

    return {}

Step 3 :

Configure Aws API Gateway

1. Create an API Gateway to handle WebSocket connections.
2. Define routes for WebSocket events (connect, disconnect, message).

Here are the Other Aws Projects :

Movie Library with Aws Cloudfront and S3

GitHub - AbdulSami455/MovieLibrary-with-Aws-Cloudfront-and-S3

In Today Digtial Age, Content and its seem Supply is Crucial Part of our Networks and Businesses.In this article, we’ll delve into the process of building a scalable and efficient movie library using AWS CloudFront and Amazon S3 as a Content Delivery Network (CDN)

In order to Understand, How I connect Aws s3 bucket with Aws Cloudfront which is independent CDN Network. You can see my Full Guideline on Github Repo
https://github.com/AbdulSami455/MovieLibrary-with-Aws-Cloudfront-and-S3

Let me Explain for You

The connection between AWS S3 and AWS CloudFront is established by configuring your CloudFront distribution to use your S3 bucket as the origin source. This means that CloudFront will pull content from your S3 bucket and distribute it across its network of edge locations. Any requests for your content are directed to CloudFront, which checks its cache and retrieves the content from your S3 bucket if needed.

AWS S3 provides the origin source for your content, while AWS CloudFront acts as the distribution layer that caches and delivers the content to users from edge locations. This integration not only accelerates content delivery but also reduces the load on your origin server, ensuring a seamless and efficient experience for users accessing your movie library.

Benefits of CDN

1. Faster Content Delivery
2. Improved Website Performance
3. Scalability and Reliability
4. Global Reach
5. Redundancy and High Availability

Want One-two-One Consulation on AWS?

Avail Now for 50% Discount

More About AWS

Read More Articles on AWS + Cloud Engineering

Product Api : https://medium.com/@as1987137/serverless-product-inventory-api-with-aws-services-f4f0191cdba2

Product Api with Authorization : https://medium.com/@as1987137/serverless-product-inventory-api-with-aws-services-f4f0191cdba2

Real Time Chat Application:

https://github.com/AbdulSami455/Real-TIme-Chat-Application-With-WebSockets-APIGateway-Lambda-DynamoDb

Step 1

Creating DynamoDB Table

if you are having any problem in creating dynamoDb. You can see Guide in my repo

https://github.com/AbdulSami455

Step 2

Creating Lambda Functions

import boto3
import json
import logging

from custom_encoder import CustomEncoder
logger = logging.getLogger()
logger.setLevel(logging.INFO)

dynamodbtablename = 'Product-inventory'
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table(dynamodbtablename)

getmethod = 'GET'
postmethod = 'POST'
patchmethod = 'PATCH'
deletemethod = 'DELETE'
healthpath = '/health'
productpath = '/product'
productspath = '/products'


def lambda_handler(event, context):
    logger.info(event)
    httpMethod = event['httpMethod']
    path = event['path']
    if httpMethod == getmethod and path == healthpath:
        response = buildResponse(200)
    elif httpMethod == getmethod and path == productpath:
        response = getProduct(event['queryStringParameters']['productid'])
    elif httpMethod == getmethod and path == productspath:
        response = getProducts()
    elif httpMethod == postmethod and path == productpath:
        response = saveProduct(json.loads(event['body']))
    elif httpMethod == patchmethod and path == productpath:
        requestbody = json.loads(event['body'])
        response = modifyProduct(requestbody['productid'], requestbody['updatekey'], requestbody['updatevalue'])
    elif httpMethod == deletemethod and path == productpath:
        requestbody = json.loads(event['body'])
        response = deleteProduct(requestbody['productid'])
    else:
        response = buildResponse(404, 'Not Found')
    return response

#Function to get Product from DynamoDb Database
def getProduct(productid):
    try:
        response = table.get_item(
            Key={
                'productid': productid
            }
        )
        if 'Item' in response:
            return buildResponse(200, response['Item'])
        else:
            return buildResponse(404, 'Product not found')
    except Exception as e:
        logger.error(str(e))
        return buildResponse(500, 'Internal Server Error')


#Get ALl Products from DynamoDb
def getProducts():
    try:
        response = table.scan()
        items = response.get('Items', [])
        return buildResponse(200, items)
    except Exception as e:
        logger.error(str(e))
        return buildResponse(500, 'Internal Server Error')

#Save Product in DynamoDB
def saveProduct(product):
    try:
        table.put_item(Item=product)
        return buildResponse(201, 'Product saved successfully')
    except Exception as e:
        logger.error(str(e))
        return buildResponse(500, 'Internal Server Error')

#
def modifyProduct(productid, updatekey, updatevalue):
    try:
        table.update_item(
            Key={'productid': productid},
            UpdateExpression=f"SET {updatekey} = :val",
            ExpressionAttributeValues={':val': updatevalue}
        )
        return buildResponse(200, 'Product updated successfully')
    except Exception as e:
        logger.error(str(e))
        return buildResponse(500, 'Internal Server Error')


def deleteProduct(productid):
    try:
        table.delete_item(Key={'productid': productid})
        return buildResponse(200, 'Product deleted successfully')
    except Exception as e:
        logger.error(str(e))
        return buildResponse(500, 'Internal Server Error')


def buildResponse(statusCode, body=None):
    response = {
        'statusCode': statusCode,
        'headers': {
            'Content-Type': 'application/json',
            'Access-Control-Allow-Origin': '*'
        }
    }
    if body is not None:
        response['body'] = json.dumps(body, cls=CustomEncoder)
    return response

Step 3

Setting Up API Gateway

Watch Full Guideness in This Repo

https://github.com/AbdulSami455/Serverless-APIS-with-Aws-APIGateway-Lambda-DynamoDB-and-Others

Step 4

Testing API on Postman

Future Possibilities

1. IAM Roles and Permissions
2. Aws CloudWatch
3. Authorize with Cognito

Step 1

• Click on “Create function.”
• Choose “Author from scratch” and provide a name for your function.

Step 2

• Select a runtime (e.g., Node.js, Python, etc.).
• Add the code for your API logic in the function.

Step 4

• Go to the AWS API Gateway service.
• Click on “Create API.”

Step 5

• Choose “REST API.”
• Select “New API” and provide a name for your API.
• Click “Create API.”

Step 6

• Once the deployment is successful, you’ll see an “Invoke URL” in the “Stages” section of your API.
• Use this URL to test your API by making HTTP requests to the defined resources and methods.

That’s it! You’ve now set up a serverless API using AWS API Gateway and Lambda. The API Gateway acts as the front-end to your API, and it forwards requests to your Lambda function to handle the business logic. Lambda automatically scales with the number of incoming requests, making it a cost-effective solution for building APIs that can handle varying traffic loads.