Two clouds in practice: a comparison of SAP Commerce Cloud v1 and v2
Please note that this piece was written in January 2019. A lot of water has passed under the bridge since that. I’m planning to write a sequel where I examine the situation in 2020. Please stay tuned.
Introduction
SAP (like all others) is moving its services from on-premise to the cloud. This includes also SAP Commerce, the eCommerce platform formerly known as Hybris. But there are two versions of the SAP Commerce Cloud, and they are significantly different from each other. This post is about the practical and technical aspects of these. With quite a few successful projects on SAP Commerce Cloud v1 and also on SAP Commerce Cloud v2 under my belt I can give my insight my insight and advice of what should be considered when selecting the cloud platform.
SAP Commerce Cloud v1
SAP Commerce Cloud v1 (CCv1 or “SAP Commerce Cloud on SAP Infrastructure” as SAP calls it) is not strictly speaking a cloud service. Instead, it is a standardized set of Linux virtual machines following the basic SAP Commerce architectural setup running in an SAP data center.
By default SAP offers three environments in CCv1: development, staging and production (and a VPN to access these). A typical development environment contains one of each server type: Hybris application server, web server, SOLR search server, Hybris datahub and HANA database. In a production environment on the other hand there is usually a minimum of two web servers, two front-end application servers, one admin application server, three SOLR servers (one master and two slaves), two datahub servers and a cluster of HANA database servers. This sizing is good for your standard B2B eCommerce site with moderate traffic, but for a public B2C site this might prove to be too small. Staging environment has similar set up as production environment although the database server cluster might be smaller.
Of these environments, development environment is managed by customer or partner. Staging and production environments are managed by SAP. In practice this means that customer or partner can SSH to the virtual machines and perform administrative tasks on development environment (not on the database server though, that is managed by SAP). In the other environments only SAP personnel have full access to the servers. In these environments the customer and the partner can access the web user interfaces of SAP Commerce, and any deployments and server restarts need to be requested from SAP via the SAP Support portal. As a standard SAP reserves a three hour time window for deployments. Access to server logs in staging and production environments is provided either via Splunk, Kibana or Slog (depending on the data center and contract).
This brings our first challenge to light: as deployments to staging and production environments require making service requests to SAP, there is no possibility of building a continuous integration (CI) pipeline to those environments. A CI pipeline can be built against the development environment, which might be good enough in some simpler setups. However, as the production environment has far more complex architecture we recommend ordering an additional QA environment with production-like server architecture but with full SSH access to the servers. We have done so in many of our CCv1 projects and it has proven to be a good practice. For example, some bugs or integration issues are only reproducible in a multi-server environment, and troubleshooting these can be very time consuming if you need to request every configuration change and reboot from SAP.
CCv1 recommendation 1: get an additional QA environment with similar server setup as production but with full access.
So the server setup setup is pretty standard: Linux boxes with SAP Commerce. But wait, what’s the database technology? Yes, in CCv1 it’s always SAP HANA database. SAP HANA is far less well-known among developers than for example MySQL is. Although most of the time a SAP Commerce developer does not need to worry much about the database technology, sometimes you have to do troubleshooting in the database directly. SAP HANA database is very heavy on resources and hard to install on a local development machine, so it makes no sense for every developer to have it. Using MySQL (or whatever you have used to) in local development is just fine, but it has proven a good practice for someone in the team to have also a local HANA database in case a need for any database troubleshooting should arise. When production is on fire it is too late to start wondering how to install a HANA database locally.
CCv1 recommendation 2: have someone in the development team with a local HANA database.
When a CCv1 project is getting near and beyond go-live the co-operation with SAP cloud team becomes very intense. The main channel of operation is the SAP support portal, which is accessed with SAP S-accounts. It has proven to be vital to have dedicated persons in the development team to handle the co-operation with SAP, so that the communication is smooth and things get resolved without unnecessary complications. SAP has 24/7 support which means that the persons handling the issues at their side change all the time, so clear, precise and coherent communication is a must.
CCv1 recommendation 3: have dedicated persons in the development team with S-accounts to handle co-operation with SAP.
SAP Commerce Cloud v2
SAP Commerce Cloud v2 (CCv2 or “Commerce Cloud in the Public Cloud”) is something completely different. This is more of a true cloud solution running on Microsoft Azure using modern technology such as Kubernetes. There is a nice web UI for managing all your environments, including production.
You’ll get the same set of environments as in CCv1: development, staging and production. Unlike CCv1, you won’t get a detailed architectural diagram showing the server setup of each environment. You can see the minimum set of server capacity SAP promises to deliver in CCv2 here. SAP can also scale up the production environment, but this does not happen automatically (i.e. no auto scaling Hybris, at least just yet). Instead, scaling is manual and needs to be requested separately in advance from SAP in case you are expecting traffic peaks to your site. The database is always a Microsoft Azure SQL database, which is provided as a service from Azure cloud. Kibana is provided for server monitoring and logs.
CCv2 recommendation 1: have someone in the development team with a local MS SQL database. Just like in CCv1 and the HANA database, it is good to have someone in your local team with similar database setup as in the actual environment. MS SQL should be close enough to Azure SQL.
In a way CCv2 is more open than CCv1. The customer or partner is able to create new builds and deploy them to all environments, including production. Builds are created by connecting your Git repository to the cloud platform. You only need to have a special manifest.json file in the root of your repository. After connecting your repository to the CCv2, you can create new builds by clicking a button in CCv2 user interface. Once a build is finished, you can click another button to deploy the build to an environment.
Sounds good, eh? Not so fast, literally: making a new build in CCv2 is slow: a simple build taking a few minutes on your local development machine will take around half an hour on CCv2. Every build is created from scratch on an empty Kubernetes pod and creating such pod seems to take most of the time. After the pod has been created the contents of your bin/custom directory in your connected repository are cloned onto the pod and a build (i.e. ant clean all) is executed. Any advanced build options like ant customize cannot be performed. The manifest.json file mentioned before contains the information about which extensions to include and what to put to local.properties before the build. Luckily you are able to download the build logs even while the build is running, so that you can see the progress yourself.
CCv2 recommendation 2: have a separate Git repository for deployments and connect that to the CCv2. You can automate moving changes from your actual development repository to the deployment repository with e.g. a Jenkins job. Local development environment and cloud environment are somewhat different. You can even perform some customization (ant customize is not allowed in the cloud build itself, remember?) before pushing code to the deployment repository.
Once the build is ready, it is time to deploy the build to an environment. This is where it gets interesting. Basically you just click the “Deploy to Environment” button, choose the target environment, database operation (initialize, update or do nothing) and wait. And then wait some more. Our experience is that deployments with initialization or update take up to 1.5 hours. During this time there is no progress indicator or log to follow, although once we get to the initialization stage we can follow the logs of that in Kibana. We have also seen deployments get stuck completely and we have had to contact SAP support to have those manually removed.
Also noteworthy to mention is that a fully automated CI pipeline (e.g. nightly builds) is currently not possible to any of the CCv2 environments, since creating builds and deploying those to the environments requires logging in to the cloud portal with a personal S-account. This is likely to change in the near future since as far as we know an API for builds and deployments has been a much requested feature in CCv2.
Our experience is that CCv2 is still in an early stage of life although there are already live production systems on it. Still, we can see the potential that it offers and we are really looking forward to that. From the things on the roadmap especially the integration options look really interesting — we’ve all had enough of datahub, haven’t we (even though that too is available in CCv2)? Add the SAP Cloud Platform Extension Factory and project Kyma and we are very excited.
Migrating from on-premise to cloud
On a very high level migrating from an existing on-premise SAP Commerce implementation to either one of the cloud platforms requires three steps:
- Upgrade SAP Commerce to a suitable version. The lowest version supported by the CCv2 is 6.7. SAP does not specify the lowest supported version for CCv1, but I doubt very old SAP Commerce versions are supported there either.
- Adjust your build process to be compatible with chosen cloud platform. The package structure required by CCv1 is strictly specified here. As mentioned above, CCv2 deployments are done from Git repository directly, but that needs to have a compatible structure as well. I would say that CCv1 package structure is probably quite close to what you have already (i.e. the deployment package includes the SAP Commerce binaries as well). Adjusting to CCv2 would require more work.
- Migrate your database and media. Database migrations are always difficult and this is no exception: this would be the most critical and difficult part of your whole cloud migration project. In the likely case you are not using Microsoft SQL server (as in CCv2) or SAP HANA database (as in CCv1) already, you need to migrate your existing database to either one of these. The migration process needs to be so smooth that you can do this during the cut-over from existing system to the cloud while your service is down.
Summary
As discussed above, the SAP Commerce Cloud v1 and v2 are two quite different ways of running SAP Commerce (Hybris). CCv1 has been around for some time now and is stable, but it has its limitations. CCv2 is new and more of a true cloud solution. It also has its limitations, but the these are mostly because of its early stage of life. SAP is heavily investing in CCv2 and we expect it to be really ready for prime time soon. And once the things on the roadmap become a reality CCv2 is the clear winner of these two.
