Azure Red Hat Openshift — An Honest Return-on-Experience
It’s amazing how far we are today, as Microsoft has started embracing open-source technologies, and an alliance with Red Hat has evolved over the last couple of years.
In my previous article, I shared how these two big plants, Microsoft and Red Hat, committed to collaborating and providing their end-users with better, simpler, yet more robust experiences to play with OpenShift and other Red Hat products.
The managed OpenShift on Azure takes things to the next level with amazing benefits, such as simplifying how containerized applications can integrate with a broad set of Azure services. Unhopefully, some limitations can arise in this case, which we need to consider. Let’s check that out.
NB. I am emotionally neutral towards ARO and looking to share my point of view.
ARO Capabilities
The usage of ARO has its own benefits and capabilities for the organization. Some of them are described below:
- Self-Service deployment: No need to deal with Ansible playbooks anymore to provision a new cluster or apply cluster upgrades.
- Full Cluster provisioning in less than 20 minutes. “az openshift create" will trigger a terraform deployment.
- Scale-In/Out the worker nodes easily and in less than 5 minutes. (Up to 20 worker nodes)
- Fully managed clusters: No more VMs to operate and no more patching.
- Well integrated with Azure Entra ID (aka. Azure AD), and its amazing features such as B2B, B2C, AccessControl, etc.
- Cluster monitoring: “Azure Monitor for Containers” is finally integrated with Azure Red Hat OpenShift. (Technical Preview). [more details]
- Single Sign-On is provided natively, and Multi-Factor Authentication is here to enhance security with additional factors.
- Native Integration with a number of Azure Services, such as Azure CosmosDB, Azure SQL DB, Azure Storage, etc.
- Dynamic Storage Provisioning: Two storage classes are provided. Persistent Volumes are based on Azure Disks (General Purpose v1) — No need to manage the storage capacity anymore ;)
- Encrypted Persistent Volumes: Data at rest is encrypted by Microsoft. So, it’s for Microsoft to manage your encryption. (Security Issue here !!)
- Regional Availability: ARO was limited to some specific regions, such as the United States, Europe, and Canada. A few weeks after its launch, Microsoft announced that ARO was released in more than seven regions. For an updated list check this link.
ARO Drawbacks
ARO does also presents some flexibility issues, which may not hurt you if you are not looking for a specific cluster’s tuning.
Do you have the fear of being ignorant of those drawbacks? Please dive below
- Nodes’ Scheduling: OpenShift cluster nodes are not enabled for labeling by customers, which will prevent the user from customizing the scheduler or applying affinity/anti-affinity rules to the running pods.
- Master API/Console is fully exposed to the outside world. The user has no control over the public Load Balancer or even on the Network Security Groups to limit/secure the access.
- Application Load Balancer is public and has no control to limit access or to add a Web Application Firewall appliance to secure your application layer. Luckily, a workaround is possible using Egress Network Security Policies to limit access.
“ARO Private Cluster” will be released soon by Microsoft to remediate to this issue, which will provide you with better security.
- Autoscaling for the cluster nodes is not supported yet.
- Loss of control on the Storage Classes: The creation of new storage tiers is not supported; This feature is generally useful to fulfil with specific high-performance needs or even to lower the costs.
- SSH Connectivity is not permitted to any node entity of the cluster: As said by Microsoft, this is intentionally done by them to ensure a certain level of SLA on the platform.
- Low SLA, 99.9% availability is granted by Microsoft but is not sufficient for most of the production workloads.
So, if the availability zone you are running ARO on goes down, you will encounter some downtime, which may impact your business. There is no possibility of provisioning a Multi-AvailabilityZone cluster until this date. - ETCD database is not encrypted.
- Docker.IO: We can not configure admission policies to deny pulling images from DockerHub (Untrusted images).
- Higher costs as you are paying for computing resources for both of the worker nodes and Master/Infra nodes. But, you can cut off the costs up to 40%, by enabling virtual machines’ reservation.
- No Kubernetes Operators’ support till the date of publishing this article. This is however scheduled for Q2, 2020.
- Custom Resource Definition — CRD, is not supported in the current release.
- Virtual Machine reservations are mandatory. This prerequisite will be removed soon by Microsoft.
- Cluster Monitoring/Logging: Inability to link Azure ARO to Azure Monitor/Logs; No information will be provided to you regarding how healthy your cluster is.
- Service Mesh not supported in the current release.
- You can not deploy Egress routers (You need privileged security contexts)
- Bootstrap project templates are not possible,
Do you need something else from ARO?
The answer is just NO !!! 😳
Conclusion
Please note that ARO is still in a “growing” phase, which means some of the missing features and services can still be expected when the proposition gets more mature.
Microsoft already promised to offer in Mai 2020 a new ARO experience based on OpenShift 4.x and be sure that the limitations we discussed here will be all tackled by that date.
Please, let me have your comments :)
