Cloud Pak for Integration on OpenShift deployment decisions: Part 1 OpenShift Platform
Cloud Pak for Integration deployments require planning decisions; this article is the beginning of a series that will focus on production deployment decisions.
This series of articles will examine decisions related to the cloud platform, RedHat OpenShift, Cloud Pak for Integration foundations and shared services, and the Cloud Pak component software, including Integration Tracing, API Management, IBM Messaging, IBM Event Streams, Application Connect, High-Speed Data Transfer.
Note this series of articles will focus on decisions for planning related to deployment of the Cloud Pak for Integration software. Some of this varies depending on the target platforms and the software that is planned. In this first in the series we will deal with the generic decsions when discussing platform and OpenShift delving more deeply in subsequent postings on the variations across platforms. For hands-on commands and sequences of steps product documentation should be referenced. Documentation references here will be for Cloud Pak for Integration 2021.4 and OpenShift 4.8. References can be found at the end.
Firstly, let's look at the platform and architecture for deployment on Openshift. Figure 1 is an example of an OpenShift deployment on AWS (Variations of this can be found on both AWS and OpenShift sites Reference [1] the quick deployment guide by AWS is one such example).
Whether the cloud is private, IBM Cloud, Azure, …each cloud platform, will have fundamentally the same conceptual requirements, but their implementation will vary slightly. For example, in the case of RedHat OpenShift Kubernetes Service (ROKS) on IBM cloud, the control plane sits in an IBM-managed zone as illustrated in Figure 2; you will find this to be similar in most managed services situations such as with OpenShift Dedicated Services.
In considering these figures, there are several decisions that we can begin to discuss. The most obvious of these is the size of the Cluster. You will need to decide the number of nodes that you will need to manage the planned workload. Remember Cloud Pak for Integration production configurations is our focus; as such, we can expect three master nodes for the control plane to confirm a high availability, and as depicted. Deploying OpenShift across multiple zones is essential for continuing service in case of failure. For the Infrastructure, Worker, and Storage planes, the sizing is dependent on planned workloads. Experience has shown that reasonable starting sizes for Cloud Pak for Integration deployments are three infrastructure nodes, five Worker nodes, and three Storage Nodes.
Note: All comments on sizing are exmplary and must be validated against load and the planned components. The compute nodes available to your cloud of choice can also influence the sizing. IBM Sales, CSM, or Technical Sales teams can help you with sizing.
But here are some of the top considerations that can influence these node sizing decisions. When it comes to sizing on the cloud, it is worth keeping in mind starting small and scaling fast should always be an option (this can be complicated with private cloud where traditional hardware availability considerations will still influence the availability of sizing).
- Consider the workload that will run on the infrastructure plane. Workloads such as Cluster Logging and Monitoring will increase the size and activity of the workload in the infrastructure plane (see Reference [2] for a more extensive list of "infrastructure" components). When deployed in separate infrastructure nodes, these components will not incur OpenShift subscription costs. A typical starting point for infrastructure configurations is three smaller compute nodes, such as 8 vCPU, 32 GB of RAM, and 120GB of disk. Depending on your deployment preferences, infrastructure nodes can be single or multiple groups. For example,
1. Two nodes for Default Router and Registry,
2. Two nodes for External Router, and
3. Three nodes for Logging and Monitoring. - When configuring cluster logging, you will need to set the sizing for the Cluster logging, in particular, consideration to the retention policy for application, infrastructure, and audit logging: the replica and resource limits for the elastic search, Kibana, and Fluentd. For Elasticsearch, you will also need an RWO storage class to allocate a storage size for each elastic search replica. (see reference [3]).
- The Monitoring Stack requires planning and consideration of which monitoring components to configure. Planning for the resources of each monitoring component will have resource limit planning and potential storage considerations (see Reference [4]). For example, Prometheus storage has unique concerns, and RedHat documentation recommends for performance reasons the use of local storage(see Reference [5],[6]). New in OpenShift 4.8, there are monitoring capabilities for the user-defined projects. Suppose this monitoring stack is your primary tool for monitoring, alerts, etc. In that case, it seems worth considering the monitoring configuration for the projects established for Cloud Pak for Integration components. (see reference [7]).
- Storage sizing is essential to deployment planning. You will need storage classes of Block(RWO), File (RWX), and Object (RWX). Object storage is used predominantly for backups that are relevant for production setups. But the storage needed depends on the planned components, so take a close look. The Instrustrure nodes will have their storage requirements on top of Cloud Pak for Integration needs. Imagine an example like Cloud Pak for Integration with Cluster Logging and Integration Tracing and API Management components installed. Use the default production template sizing from the Cloud Pak for Integration Platform Navigator. Your sizing may be similar to that shown in figure 3 (This is exemplary for illustrative purposes and based on a specific workload, backup sizing excluded from example). It illustrates the variety of storage needs. For Openshift thinking on storage Optimization, see Reference [8]. For security reasons, you will
- How many Cloud Pak for Integration components will you run in the Cluster? An increased number of components means a more substantial system requirement which could mean more Worker nodes. Planning your workloads carefully and consulting your architects is recommended.
So far, we have discussed workload sizing decisions and have mentioned the concept of groups of nodes. It makes sense to take the next step in creating machine sets, consider the machine groupings and plan the labels for workload association to machine sets. Utilizing OpenShift machine sets is an easy way to automate your scaling. Considerations include which groups should auto-scale based on workload distribution and possible storage implications in scaling. For some tips regarding scaling and machine sets, see Reference [9].
The last item we will discuss for planning the platform's deployment is security. Application and Role-Based Authentication and authorization are somewhat missing from the above architecture figures. But security is an essential conversation that you must plan and have. Always consider security planning carefully, and it applies to the whole stack network, storage, Cluster, projects, pods, and so on. Initial security decisions are some of the most important. A full identity provider should be used for securing authentication and the groups mapped for authorization. Utilize fine-grained access controls only giving groups access to which they require access. Avoid having many groups or users with Cluster scoped permissions. Some security decisions should be easy such as preferring RHCOS as the default operating system for nodes or encrypting the etcd datastore.
"It's primary goal is to provide a secure operating system platform for running Kubernetes, OpenShift services, and containerized workloads running on an aggragated platform."
- p.g 37 The OpenShift Security Guide Book (Reference [10])
Some security decisions will depend on your enterprise policies and agreements with your cloud provider. Some that may need consideration include
- For audit logging, do you require to send your audit to a log storage or Security Information and Event Management (SIEM) system? Do note in the security Guide Book that the cluster logging is logging stack is considered best effort for non-repudiation. Review also the audit policy and retention requirements.
- Will you want full disk encryption enabled?
- What groups should have authorization for managing the Cluster? See page 175, The OpenShift Security Guide Book, Best practices for RBAC management, Reference [10].
- Less common, but do you require an external egress server for managing calls external to the Cluster?
- What timing server will you use for encryption? Review pg 254–255 The OpenShift Security Guide Book, Network Time and Date Encryption, Reference [10].
Note: Later in this series, when discussing the different Cloud Pak for Integration components, we will revisit security specific to the component. Security related to the Platform Navigator will be delt with in the next article.
With these platform decisions in mind, it is time to look at the Cloud Pak for Integration. In particular, in this article, we will look at the Operator Configuration and the Platform Navigator Operator Installation. Figure 4 provides a reminder of the Cloud Pak for Integration Components.
The Cloud Pak for Integration Planning documentation covers these topics very well, so we will continue to highlight the decisions required to discuss the options; Considerations include:
- Do I have any specific High availability requirements that will influence deployment or sizing? (See Reference [11])
- When installing the operators, set the scope to All Namespaces, expected for production. But as discussed in Reference [12], there are reasons such as shared non-production Clusters for project scoped installation.
- Do I need to plan which nodes the workload runs? While node selection is possible, it is good to understand the best reasons for node assignment. See Reference [13]. Generally, this is probably not needed for initial workload deployments, and it is best to let OpenShift make it workload assignments where possible. It is a somewhat advanced requirement.
- When installing the platform navigator, consider if more than one replica is needed; this is unlikely. If MQ is to be deployed, you will want to enable the MQ Dashboard, and finally, consider the project name that you will deploy the platform navigator. Use a common prefix for projects where Cloud Pak for Integration components are installed, keeping them grouped in the projects list. For example cp4i, cp4i-tracing, cp4i-api.
Note: While many of these planning decisions should be acted on if the preferred decision is not clear, additional nodes can be created later, and workloads moved as appropriate. Do not let indecision impede your getting started but as your deployment grows you can always revisit the early deployment decsions and how to manage the workload of the cluster.
Summary
Setting up a Cloud Pak for Integration on an OpenShift environment can be straightforward. But the reality of production systems remains true; we must plan, think and understand our enterprise's requirements to ensure reliable platforms to run our business. Figure 5 is a summary of the decisions to make.
It is worth noting things that must be in the environment have not been addressed here, for example, the need for load balancers. Consider the questions here early where possible they are mostly options but have the potential to impact your deployment.
Next in Series
We will take a specific look at API Management deployment decisions, including Integration Tracing. Figure 6 illustrates the deployed components in the example.
References
[1] "IBM Cloud Pak for Integration on the AWS Cloud"
https://aws-quickstart.github.io/quickstart-ibm-integration/
[2] "OpenShift Container Platform infrastructure components" https://docs.openshift.com/container-platform/4.8/machine_management/creating-infrastructure-machinesets.html
[3] Installing OpenShift Logging
https://docs.openshift.com/container-platform/4.8/logging/cluster-logging-deploying.html#cluster-logging-deploying
[4] Understanding the Monitoring Stack
https://docs.openshift.com/container-platform/4.8/monitoring/understanding-the-monitoring-stack.html
[5] Configuring the monitoring stack
https://docs.openshift.com/container-platform/4.8/monitoring/configuring-the-monitoring-stack.html#prerequisites
[6]Scaling the Cluster Monitoring Operator
https://docs.openshift.com/container-platform/4.8/scalability_and_performance/scaling-cluster-monitoring-operator.html#scaling-cluster-monitoring-operator
[7] Enabling monitoring for user-defined projects
https://docs.openshift.com/container-platform/4.8/monitoring/enabling-monitoring-for-user-defined-projects.html#granting-users-permission-to-monitor-user-defined-projects_enabling-monitoring-for-user-defined-projects
[8] Optimizing Storage
https://docs.openshift.com/container-platform/4.8/scalability_and_performance/optimizing-storage.html
[9] Recommended Cluster Scaling Practices
https://docs.openshift.com/container-platform/4.8/scalability_and_performance/recommended-cluster-scaling-practices.html
[10] The OpenShift Security Guide Book
https://access.redhat.com/articles/5059881?extIdCarryOver=true&sc_cid=701f2000001Css5AAC
[11] High availability considerations
https://www.ibm.com/docs/en/cloud-paks/cp-integration/2021.4?topic=installation-high-availability-considerations
[12] Structuring your deployment
https://www.ibm.com/docs/en/cloud-paks/cp-integration/2021.4?topic=installation-structuring-your-deployment
[13] Node placement considerations
https://www.ibm.com/docs/en/cloud-paks/cp-integration/2021.4?topic=installation-node-placement-considerations
