How to configure WSO2 API Manager 3.0 Analytics HA cluster
Introduction
WSO2 API Manager is a 100% open source, full API lifecycle management product that supports various different deployment architectures including monolithic, microservices and hybrid-cloud. It comes with a pre-configured analytics runtime which is designed specifically to analyze various events occurring at the API management runtime. In this post, I will be discussing how to set up a minimum high available deployment of the WSO2 API Manager analytics server along with the API Manager.
How Analytics Work?
WSO2 API Manager consists of various components that act together to provide the full API lifecycle management capability. The API Gateway is the main component that receives the API traffic from consumers and it is the main component interact with the analytics runtime. Analytics runtime has 2 main profiles called worker profile and dashboard profile.
- Worker profile — This is the component which receives analytics related event streams from the gateway/s and does the real-time event processing and publishes data to tables and other consumers through alerts/notifications
- Dashboard profile — This component is mainly reading the analyzed information from the database tables and visualize them in dashboards which are pre-built
The high-level architecture of API Manager works along with Analytics runtime is depicted in the below figure.
WSO2 API Manager analytics runtime utilizes a real-time event processing engine called siddhi to process events that are coming from the API gateway/s. This engine will calculate statistics of various aspects of API operations and store them in a set of tables that holds data per SECONDS, MINUTES, HOURS, DAYS, MONTHS, YEARS. These summary tables allow the visualization graphs (dashboard) to showcase these statistics in various granularity levels up to second.
Configuring minimum HA deployment of WSO2 APIM Analytics
Even though analytics is not in the critical path of your customer interaction with business data, it is an essential part of the overall business operations. Having a proper analytics deployment which is always available provides the ability to operate your API manager deployment in better quality unless otherwise. Having a single node of worker profile and dashboard profile would work until you lose the worker profile. If your analytics requirement is not much critical, you can go with that approach.
If you are really serious about the analytics of the API manager deployment, then the minimum recommendation is to deploy API manager analytics in an active/passive HA deployment. In this deployment, there are 2 worker profiles of APIM Analytics deployed in a cluster. In this minimum HA setup, one node is assigned as the active node while the other node is assigned as the passive node. Only the active node processes the incoming events and publishes the outgoing events. Internally, the active node publishes the events to the passive node, but the passive node does not process or send any events outside as mentioned earlier. In a scenario where the active node fails, the passive node is activated, and it starts receiving events and then publishes them from where the active node left off. Once the terminated (previously active) node restarts, it operates in the passive state. In the passive node, the data bridge port and other ports related to sources remain closed, and events are not fetched from pulling sources such as Kafka, JMS, etc. until the node becomes active.
Configuration steps
You need to download the WSO2 API Manager 3.0 and WSO2 API Manager Analytics 3.0 from the download page mentioned below.
https://wso2.com/api-management/#
You can extract the WSO2 API Manager 3.0 into a folder and create 3 copies of the WSO2 API Manager Analytics 3.0 binary and extract them to 3 different folders. You can name them as worker-1, worker-2, and dashboard.
Configure the API Manager 3.0 Analytics worker profiles
WSO2 API Manager Analytics is built using the WSO2 streaming integrator (stream processor in the past) runtime. You can follow the steps mentioned in the below documentation link to configure the 2 worker nodes of the analytics cluster (worker-1 and worker-2).
https://docs.wso2.com/display/SP440/Minimum+High+Availability+Deployment
There are a few things you have to do differently than in the above link.
- In step 2, when configuring the state.persistence section, you should use the class name which is mentioned below.
- In step 4, you should use different ports for eventSyncServer setting in 2 nodes (e.g. 9893 for worker-1 and 9894 for worker-2) if you are running both the instances in the same server. If those are running on 2 separate servers, this is not required.
- To start two instances in the same machine, the listenerConfigurations under the wso2.transport.http and siddhi.stores.query.api namespaces in the <WSO2AM_ANALYTICS_HOME>/conf/worker/deployment.yaml file must be updated to listen to different ports. The offset property under the ports section of the wso2.carbon section found in the <SP_HOME>/conf/worker/deployment.yaml should also be changed in one SP instance to avoid conflicts when starting both servers.
In addition to the above changes, you have to configure the datasources shared by the worker instances and the dashboard instance as well as with the API manager instance.
Configure the analytics database within worker profiles
Open the <API-M_ANALYTICS_HOME>/conf/worker/deployment.yaml file and edit the AM_DB section and APIM_ANALYTICS_DB section.
Edit the AM_DB sections and point to your desired type of database.
A sample for MySQL is shown below.
Edit the APIM_ANALYTICS_DB section. A sample for MySQL is shown below.
If your deployment does not consist of any Microgateways, you do not need to configure the WSO2AM_MGW_ANALYTICS_DB to an external database as you can use the default embedded H2 database.
Configure WSO2AM_MGW_ANALYTICS_DB to the same database as APIM_ANALYTICS_DB in the <API-M_ANALYTICS_HOME>/conf/worker/deployment.yaml file.
Create the AM_USAGE_UPLOADED_FILES table in the APIM_ANALYTICS_DB database.
Once you are done with the configurations af the 2 worker instances, you can start the workers by executing following command from the <APIM_ANALYTICS_HOME> directory
sh bin/worker.sh
If the cluster is correctly configured, the following CLI logs can be viewed without any error logs:
In the active node:
[2018–09–09 23:56:54,272] INFO {org.wso2.carbon.stream.processor.core.internal.ServiceComponent} — WSO2 Stream Processor Starting in Two Node Minimum HA Deployment
[2018–09–09 23:56:54,294] INFO {org.wso2.carbon.stream.processor.core.ha.HAManager} — HA Deployment: Starting up as Active Node
In the passive node:
[2018–09–09 23:58:44,178] INFO {org.wso2.carbon.stream.processor.core.internal.ServiceComponent} — WSO2 Stream Processor Starting in Two Node Minimum HA Deployment
[2018–09–09 23:58:44,199] INFO {org.wso2.carbon.stream.processor.core.ha.HAManager} — HA Deployment: Starting up as Passive Node
Configure the API Manager 3.0 Analytics dashboard profile
Open the <APIM_ANALYTICS_HOME>/conf/dashboard/deployment.yaml file. Here the <APIM_ANALYTICS_HOME> directory is where you have extracted the 3rd analytics instance.
Edit the APIM_ANALYTICS_DB and AM_DB sections and point to your desired type of database.
A sample for MySQL is shown below.
Edit the APIM_ANALYTICS_DB section. A sample for MySQL is shown below.
Point the following data sources to external databases (optional). None of the following databases need DB scripts. The tables will be automatically created.
- WSO2_DASHBOARD_DB (dashboard profile)
- BUSINESS_RULES_DB (dashboard profile)
- WSO2_PERMISSIONS_DB (worker + dashboard)
- GEO_LOCATION_DATA (Only if you need geo-location based statistics.)
Once you are done with the configuration, you can start the dashboard profile by executing the below command from the <APIM_ANALYTICS_HOME> directory.
sh bin/dashboard.sh
Even though you start the analytics dashboard instance, you won’t be able to log into the server until you start the API Manager instance. The reason is that, by default, APIM Analytics is configured to use the API Manager component as the Identity Provider (IdP). You can find these configurations under “auth.configs” section within the /conf/dashboard/deployment.yaml fie of the <APIM_ANALYTICS_HOME> directory.
Configure the API Manager 3.0
This is the easiest step of this deployment.
Open the <APIM_HOME>/repository/conf/deployment.toml file and uncomment the following section as shown below.
Edit the AM_DB sections and point to your desired type of database.
A sample for MySQL is shown below.
Edit the analytics section as mentioned below in the same file.
Once you are done with the configuration of the above file, you can start the API Manager server by executing the below command from <APIM_HOME> directory.
sh bin/wso2server.sh
The overall deployment architecture looks like the below figure.
If you need to extend this deployment into a more distributed deployment, the architecture would look like the below figure.
Testing the deployment
Once you are done with the deployment. You can test the deployment by sending a few requests by following the quick start guide.
https://apim.docs.wso2.com/en/latest/getting-started/quick-start-guide/
Once you follow the above mentioned steps and sent a couple of requests, you can log into the dashboard profile and view the 2 dashboards available in the below URL.
https://localhost:9643/analytics-dashboard
You can test the active/passive behavior by shutting down the active worker node. You will observe that the passive node becomes active. Once you restart the previously active node, you can observe that it will become the passive node. During this process, the APIM node will also switch from worker-1 to worker-2 once the active node gets switched.
