Overload control is important in service meshes because they often handle large volumes of traffic across multiple microservices. Without proper overload control mechanisms, a sudden surge in traffic to a particular service could cause it to become overwhelmed and fail, leading to service disruptions and degraded performance for end-users.

Some common mechanisms for overload control in service meshes include:

1. Circuit breaking: A technique that prevents requests from being sent to an overloaded service by detecting and tripping open the circuit when the service is unable to handle the load.
2. Rate limiting: A mechanism that limits the rate at which requests can be sent to a service, preventing it from being overwhelmed with too many requests.
3. Load shedding: A technique that reduces the amount of traffic directed to a service during periods of high load, prioritising more critical requests over less critical ones.

A very few Service Mesh technologies provide intelligent way of handling overload.

Following is the way proposed by WeChat in their paper which handles overload gracefully.

Business-oriented Admission Control

The idea behind business-oriented admission control is to prioritise service requests based on their importance to the overall business and user experience. For example, requests related to user authentication and payment processing may be given higher priority than requests related to messaging or content delivery.

This approach recognises that different services may have different levels of importance and impact on the user experience, and that overload control strategies should take this into account when managing traffic and allocating resources. By prioritising requests based on their business significance, the system can ensure that the most critical services and requests are handled first, even during periods of high load or resource constraints.

User-oriented Admission Control

This technique considers the priority of requests not only based on their business impact but also on the specific user making the request. For example, a request from a high-value user may be given a higher priority than the same request from a low-value user.

With user-oriented admission control, when the service is overloaded, requests with the same business priority are not necessarily all discarded or all consumed. Instead, the admission decision for each request is based on both its business priority and the user making the request. This approach can help to better balance the workload among different users and prevent the issue of partially discarding requests.

Adaptive admission control is a dynamic approach to load shedding in microservices that adjusts admission levels in response to changes in load status. This approach is particularly effective when services are overloaded, as the corresponding load shedding strategy is dependent on the volume of excess workload. The goal of adaptive admission control is to minimize the impact of load shedding on the quality of service while maintaining effective load shedding.

Adaptive Admission Control

In an adaptive admission control system, admission levels are automatically adjusted in response to changes in load status. When the overload situation is severe, admission levels are restricted to reject more incoming requests. Conversely, when the overload situation becomes mild, admission levels are relaxed. This allows for effective load shedding while minimizing the impact on the overall service.

Adaptive admission control is particularly important in complex microservice architectures, where the adjustment of admission levels of overloaded services must be automatic and adaptive. The DAGOR overload control system relies on adaptive admission control to maintain the performance of the microservice system during periods of high load.

Here is a step-by-step workflow for overload control based on above strategies:

1. User request arrives at the microservice system and is routed to the entry service.
2. Entry service assigns business and user priorities to the request and encapsulates these priorities into the request header.
3. Each service invokes downstream services according to the business logic.
4. When a service receives a request, it performs priority-based admission control based on its current admission level. The service periodically adjusts its admission level with respect to the load status.
5. When a service intends to send a subsequent request to a downstream service, it performs local admission control based on the stored admission level of the downstream service. The upstream service only sends out requests that are admitted by the local admission control.
6. When a downstream service sends a response to an upstream service, it piggybacks its current admission level to the response message.
7. When the upstream service receives the response, it extracts the information of admission level from the message and updates the corresponding local record for the downstream service accordingly.
8. If a service becomes overloaded, it adjusts its admission level to restrict more incoming requests. Conversely, when the overload situation becomes mild, the admission levels should be relaxed.
9. Requests with the same business priority may be partially discarded if necessary due to fluctuation in admission levels.
10. The adaptive admission control system ensures that admission levels are automatically and adaptively adjusted in response to changes in load status.

Happy Learning !!!!!

Microservices architectural pattern dominates the modern software development . It definitely has its own set of advantages.

In the context of microservices architecture, “polyrepo” refers to the practice of using multiple separate version control repositories for each individual microservice.

While using a polyrepo approach in microservices development can offer some benefits, such as loose coupling and independent release cycles for individual microservices, it also has potential disadvantages, which may include:

Complexity in managing dependencies: With each microservice having its own repository, managing dependencies among microservices can become more challenging. Changes in one microservice may require updates in multiple other microservices, leading to increased coordination efforts and potential versioning conflicts.

Overhead in managing multiple repositories: Managing multiple repositories may require additional tooling, processes, and overhead for tasks such as branching, merging, and managing release versions. It can also result in increased administrative overhead, such as managing access controls and permissions for multiple repositories.

Code duplication and inconsistency: Microservices are often designed to be highly decoupled and independent, but there may still be shared code, libraries, or configurations among microservices. With a polyrepo approach, there may be a risk of code duplication or inconsistency, as each microservice may independently implement and maintain similar functionality.

Lack of cross-repo visibility and traceability: In a polyrepo approach, it may be more difficult to have a unified view of the entire system and track changes across microservices. This can make it challenging to trace and understand the impact of changes on the system as a whole, and can hinder visibility into the overall system’s state.

Potential overhead in build and deployment processes: Managing the build, testing, and deployment processes across multiple repositories may require additional effort and tooling, as compared to a monorepo approach where all the code is stored in a single repository. This can increase the complexity and time required for managing the build and release processes.

Monorepo tries to solve these problems by storing all the source code for multiple projects or applications in a single version control repository.There are a couple of tools to use monorepo practise in software development. Google’s Bazel and Meta’s Buck are popular among all of these.

Here’s an example of how you could set up a monorepo using Bazel for an e-commerce application with a Python web server for the order system and a Golang server for the payment system:

Directory Structure: Create a root directory for your monorepo, and inside it, create directories for your backend components, including the order system and payment system.

e-commerce-monorepo/
  ├── order/
  │   ├── BUILD
  │   ├── src/
  │   └── ...
  ├── payment/
  │   ├── BUILD
  │   ├── src/
  │   └── ...
  └── ... 

Order System — Python Web Server: In the order directory, you can have your Python web server application for the order system. You can use a framework like Flask or Django to build your web server. Create a BUILD file in the order directory that describes how to build the order system using Bazel. Here's an example BUILD file for a hypothetical Flask-based order system:

py_binary(
    name = "order_server",
    srcs = glob(["src/*.py"]),
    deps = [],  # Define dependencies as needed
)

Payment System — Golang Server: In the payment directory, you can have your Golang server application for the payment system. You can use a framework like Gin or Revel to build your server. Create a BUILD file in the payment directory that describes how to build the payment system using Bazel. Here's an example BUILD file for a hypothetical Gin-based payment system:

go_binary(
    name = "payment_server",
    srcs = glob(["src/*.go"]),
    deps = [],  # Define dependencies as needed
)

Build Targets: To build the order system or payment system targets, you can use the bazel build command, followed by the label of the target you want to build. For example:

bazel build //order:order_server

bazel build //payment:payment_server

Bazel will automatically determine the correct build order based on the dependencies defined in the BUILD files and build the targets accordingly.

Happy Learning !!!

Apache Cassandra is a highly scalable, high-performance distributed database designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure. It is a type of NoSQL database.

Though Apache Cassandra has many good features that make it a highly scalable and high-performance database, this scalability and performance come with a cost. Sometimes this cost is too high to afford for a business.

Hence, I would like you to know some of the things about Apache Cassandra before using it in Production.

Deleting bulk data from Cassandra table makes reads very slow

In Cassandra, when you delete a record, it does not hard delete it and just creates a tombstone.

In the context of Cassandra, a tombstone is specific data stored alongside standard data. A delete operation does nothing more than inserting a tombstone. And when you read data from this table, Cassandra has to scan all non-deleted/live records plus tombstones making the reads slower. There is a configuration(gc_grace_seconds) per table after that tombstones get deleted. Also if your tombstones limit reaches a threshold value, you cannot read from your table. All reads are stopped till you get tombstones cleared on every Cassandra Node in the cluster.

If you model your tables incorrectly, your application is screwed

When designing a table in Cassandra, you have to specify Partition Key and Clustering Key. The partition key allows the database to quickly retrieve the data from the appropriate node. The database requires that all partition keys are restricted or none. All the partition key columns are required to compute the hash and locate the node containing the partition. If you want to fetch data from multiple partitions, brace yourself for ReadTimeoutExceptions.

Choose one of the two “strong consistency” or “high availability”

This should hold true for all distributed databases and not just Cassandra. But you have to choose one of these properties carefully, else either your application will not be available for some time or it will be available but return stale data.

Having too many tables in a cluster cause performance degradation

According to Datastax documentation, Once your database reaches the warning threshold(200 tables), it’s time to start monitoring and planning a re-architecture. If possible, remove unused and underutilized tables. The failure threshold is 500 tables.

SELECT….IN query — A big NO!!!!!!!!!!!!!!!!!!!!!!!!!

Under most conditions, using IN in relation to the partition key is not recommended. To process a list of values, the SELECT may have to query many nodes, which degrades performance and may cause timeouts.

Things I Wish I Knew Before Using Apache Cassandra was originally published in STUFF.TECHNOLOGY on Medium, where people are continuing the conversation by highlighting and responding to this story.

Recently, I started observing RequestTooLarge Exceptions in our API. After careful analysis, I got to know that there is a limit on the length of URL decided by most of the REST clients as well as servers. Though this limit is not standardized by HTTP specification, we have to respect this limit as we are dependent on standard REST clients and web servers to host and to call our API. The practical range of this limit is between 2Kb to 8Kb.

But wait, this is not the only limitation that GET has. It also have following limitations

1. URL Encoding is required before calling a GET API else be ready for some surprises in your API results. This encoding is required because your client can not encode the special characters like space, ampersand, brackets and what not automatically.
2. It is difficult to perform complex queries as URLs are not expressive as JSON or XML.

Now you may say , enough badmouthing about HTTP GET, tell us how we can solve these limitations ?

A very standard way of solving these limitations is using POST instead of GET. But POST is a way to create a resource while GET is for retrieving, how can I use it ? The answer is treat your “search” as a resource.

For Example,

Instead of GET /reports?filter=filter1&startDate=date1&endDate=date2 ,

Use POST /reports/search and give filter, startDate and endDate as request body. (Now, I know why ElasticSearch uses this format).

I understand that this may not be the exactly compliant with HTTP standards.And this may lead to some other problems like for POST, client will not cache request similar to GET, and POST being non-idempotent, client may assume that retrying same request may be harmful.

Hence, I would advise this should be used if your API are for reporting, heavy searches. Also, some companies are exploring HTTP’s new verb REPORT ( http://www.iana.org/assignments/http-methods/http-methods.xhtml) which solves problems of both GET and POST and fits this use case.

Happy API Designing !!!

Spring Boot provides a configurable and programmatic way to enable HTTPS on your web application. In this article, we will see each of these ways in detail.

Enable HTTPS using Configuration

1. You need to create a self-signed SSL certificate. The two most common formats used for keystores are JKS, specific for Java, and PKCS12, an industry standard format. JKS used to be the default choice, but now Oracle recommends to adopt the PKCS12 format. We will use PKCS12 format in the example.
2. Open your command prompt and enter the following command.

keytool -genkeypair -alias tomcat -keyalg RSA -keysize 2048 -storetype PKCS12 -keystore keystore.p12 -validity 3650

3. As an output of this command, you will be asked to enter the password for your keystore.

4. Once your password is accepted, you will be asked some information to enter like first name, last name, city, country but you can skip those.

5. Once you have entered all the required information, a keystore will be created. verify that using the following command

keytool -list -v -storetype pkcs12 -keystore keystore.p12

6. Now we have SSL Certificate ready, the only thing that is remaining is changing your application.properties/application.yml in your Spring Boot application.

server.port=50125 
security.require-ssl=true 
server.ssl.key-store-type=PKCS12 
server.ssl.key-store=classpath:keystore.p12 
server.ssl.key-store-password=password
server.ssl.key-alias=tomcat

Enable HTTPS Programmatically

1. Declare @Configuration class e.g. TomcatConfig.java in which you need to create an instance of the class org.apache.catalina.connector.Connector and set SSL(org.springframework.boot.web.server.Ssl) information to the instance of this Connector.
2. Create another class SslConnectorCustomizer which actually configures SSL using Tomcat’s API.

package com.demo.https;

import java.io.FileNotFoundException;

import org.apache.catalina.connector.Connector;
import org.apache.catalina.webresources.TomcatURLStreamHandlerFactory;
import org.apache.coyote.ProtocolHandler;
import org.apache.coyote.http11.AbstractHttp11JsseProtocol;
import org.apache.coyote.http11.Http11NioProtocol;
import org.apache.tomcat.util.net.SSLHostConfig;

import org.springframework.boot.web.server.Ssl;
import org.springframework.boot.web.server.SslStoreProvider;
import org.springframework.boot.web.server.WebServerException;
import org.springframework.util.Assert;
import org.springframework.util.ResourceUtils;
import org.springframework.util.StringUtils;

class SslConnectorCustomizer {

private final Ssl ssl;

private final SslStoreProvider sslStoreProvider;

SslConnectorCustomizer(Ssl ssl, SslStoreProvider sslStoreProvider) {
        Assert.notNull(ssl, "Ssl configuration should not be null");
        this.ssl = ssl;
        this.sslStoreProvider = sslStoreProvider;
    }

public void customize(Connector connector) {
        ProtocolHandler handler = connector.getProtocolHandler();
        Assert.state(handler instanceof AbstractHttp11JsseProtocol,
                "To use SSL, the connector's protocol handler must be an "
                        + "AbstractHttp11JsseProtocol subclass");
        configureSsl((AbstractHttp11JsseProtocol<?>) handler, this.ssl,
                this.sslStoreProvider);
        connector.setScheme("https");
        connector.setSecure(true);
    }

    protected void configureSsl(AbstractHttp11JsseProtocol<?> protocol, Ssl ssl,
                                SslStoreProvider sslStoreProvider) {
        protocol.setSSLEnabled(true);
        protocol.setSslProtocol(ssl.getProtocol());
        configureSslClientAuth(protocol, ssl);
        protocol.setKeystorePass(ssl.getKeyStorePassword());
        protocol.setKeyPass(ssl.getKeyPassword());
        protocol.setKeyAlias(ssl.getKeyAlias());
        String ciphers = StringUtils.arrayToCommaDelimitedString(ssl.getCiphers());
        if (StringUtils.hasText(ciphers)) {
            protocol.setCiphers(ciphers);
        }
        if (ssl.getEnabledProtocols() != null) {
            for (SSLHostConfig sslHostConfig : protocol.findSslHostConfigs()) {
                sslHostConfig.setProtocols(StringUtils
                        .arrayToCommaDelimitedString(ssl.getEnabledProtocols()));
            }
        }
        if (sslStoreProvider != null) {
            configureSslStoreProvider(protocol, sslStoreProvider);
        }
        else {
            configureSslKeyStore(protocol, ssl);
            configureSslTrustStore(protocol, ssl);
        }
    }

private void configureSslClientAuth(AbstractHttp11JsseProtocol<?> protocol, Ssl ssl) {
        if (ssl.getClientAuth() == Ssl.ClientAuth.NEED) {
            protocol.setClientAuth(Boolean.TRUE.toString());
        }
        else if (ssl.getClientAuth() == Ssl.ClientAuth.WANT) {
            protocol.setClientAuth("want");
        }
    }

protected void configureSslStoreProvider(AbstractHttp11JsseProtocol<?> protocol,
                                             SslStoreProvider sslStoreProvider) {
        Assert.isInstanceOf(Http11NioProtocol.class, protocol,
                "SslStoreProvider can only be used with Http11NioProtocol");
        TomcatURLStreamHandlerFactory instance = TomcatURLStreamHandlerFactory
                .getInstance();
        instance.addUserFactory(
                new SslStoreProviderUrlStreamHandlerFactory(sslStoreProvider));
        try {
            if (sslStoreProvider.getKeyStore() != null) {
                protocol.setKeystorePass("");
                protocol.setKeystoreFile(
                        SslStoreProviderUrlStreamHandlerFactory.KEY_STORE_URL);
            }
            if (sslStoreProvider.getTrustStore() != null) {
                protocol.setTruststorePass("");
                protocol.setTruststoreFile(
                        SslStoreProviderUrlStreamHandlerFactory.TRUST_STORE_URL);
            }
        }
        catch (Exception ex) {
            throw new WebServerException("Could not load store: " + ex.getMessage(), ex);
        }
    }

private void configureSslKeyStore(AbstractHttp11JsseProtocol<?> protocol, Ssl ssl) {
        try {
            protocol.setKeystoreFile(ResourceUtils.getURL(ssl.getKeyStore()).toString());
        }
        catch (FileNotFoundException ex) {
            throw new WebServerException("Could not load key store: " + ex.getMessage(),
                    ex);
        }
        if (ssl.getKeyStoreType() != null) {
            protocol.setKeystoreType(ssl.getKeyStoreType());
        }
        if (ssl.getKeyStoreProvider() != null) {
            protocol.setKeystoreProvider(ssl.getKeyStoreProvider());
        }
    }

private void configureSslTrustStore(AbstractHttp11JsseProtocol<?> protocol, Ssl ssl) {
        if (ssl.getTrustStore() != null) {
            try {
                protocol.setTruststoreFile(
                        ResourceUtils.getURL(ssl.getTrustStore()).toString());
            }
            catch (FileNotFoundException ex) {
                throw new WebServerException(
                        "Could not load trust store: " + ex.getMessage(), ex);
            }
        }
        protocol.setTruststorePass(ssl.getTrustStorePassword());
        if (ssl.getTrustStoreType() != null) {
            protocol.setTruststoreType(ssl.getTrustStoreType());
        }
        if (ssl.getTrustStoreProvider() != null) {
            protocol.setTruststoreProvider(ssl.getTrustStoreProvider());
        }
    }

}

3. Create class SslStoreProviderUrlStreamHandlerFactory. This is an URLStreamHandlerFactory that provides an URLStreamHandler for accessing an SSLStoreProvider’s key store and trust store from a URL.

package com.demo.https;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;
import java.net.URLStreamHandler;
import java.net.URLStreamHandlerFactory;
import java.security.KeyStore;

import org.springframework.boot.web.server.SslStoreProvider;

class SslStoreProviderUrlStreamHandlerFactory implements URLStreamHandlerFactory {

private static final String PROTOCOL = "springbootssl";

private static final String KEY_STORE_PATH = "keyStore";

static final String KEY_STORE_URL = PROTOCOL + ":" + KEY_STORE_PATH;

private static final String TRUST_STORE_PATH = "trustStore";

static final String TRUST_STORE_URL = PROTOCOL + ":" + TRUST_STORE_PATH;

private final SslStoreProvider sslStoreProvider;

SslStoreProviderUrlStreamHandlerFactory(SslStoreProvider sslStoreProvider) {
        this.sslStoreProvider = sslStoreProvider;
    }

@Override
    public URLStreamHandler createURLStreamHandler(String protocol) {
        if (PROTOCOL.equals(protocol)) {
            return new URLStreamHandler() {

@Override
                protected URLConnection openConnection(URL url) throws IOException {
                    try {
                        if (KEY_STORE_PATH.equals(url.getPath())) {
                            return new KeyStoreUrlConnection(url,
                                    SslStoreProviderUrlStreamHandlerFactory.this.sslStoreProvider
                                            .getKeyStore());
                        }
                        if (TRUST_STORE_PATH.equals(url.getPath())) {
                            return new KeyStoreUrlConnection(url,
                                    SslStoreProviderUrlStreamHandlerFactory.this.sslStoreProvider
                                            .getTrustStore());
                        }
                    }
                    catch (Exception ex) {
                        throw new IOException(ex);
                    }
                    throw new IOException("Invalid path: " + url.getPath());
                }
            };
        }
        return null;
    }

private static final class KeyStoreUrlConnection extends URLConnection {

private final KeyStore keyStore;

private KeyStoreUrlConnection(URL url, KeyStore keyStore) {
            super(url);
            this.keyStore = keyStore;
        }

@Override
        public void connect() throws IOException {

}

@Override
        public InputStream getInputStream() throws IOException {

try {
          ByteArrayOutputStream stream = new ByteArrayOutputStream();
                this.keyStore.store(stream, new char[0]);
                return new ByteArrayInputStream(stream.toByteArray());
            }
            catch (Exception ex) {
                throw new IOException(ex);
            }
        }

}

}

And that’s it! Do let us know if it works for you or either way. Happy learning!

How To Enable HTTPS In Spring Boot Application was originally published in STUFF.TECHNOLOGY on Medium, where people are continuing the conversation by highlighting and responding to this story.