Case Study for Android CI -> CD -> CD = Continuous * ( Integration, Delivery, Deployment ) Part 4

Designed By Hafiz Waleed Hussain

WOW, we got one more day so it’s time to make this day awesome by learning something new. 🙂

Hello friends, hope you are doing good. So today we are going to start our part 4.

Motivation:

Here motivation is same which I already shared with you guys in part1.

Plan:

So, for my readers below is my plan for this case study.

  1. Part 1
  • Optional Prerequisites
  • What is Continuous Integration ( CI), Continous Delivery ( CD), Continous Deployment ( CD )?
  • Version control system ( VCS )
  • Continuous integration server
  • Build System
  1. Part 2
  • Application code on Github
  • Git structure for the code
  • Unit Tests
  • Functional Tests
  • Gradle build system and it’s commands
  • Jenkins integration
  1. Part 3
  • Webhook
  • Unit Test Report
  • Unit Test Code Coverage Report
  • Lint rules ( Skip )
  • Artifacts
  • Release notes
  1. Part 4
  • Functional / UI / Espresso Unit Test Report
  • Functional / UI / Espresso Unit Test Code Coverage Report
  • Email
  • Notifications may be on slack, hip chat
  • Deliver APK using Fabric
  • Deployment on play store beta
  • Tag creation after publish
  • Merge code with the master branch ( Skip )
  1. Part 5
  • Case study ( App has three different flavors )
  • Git structure for the code
  • Unit Tests
  • Functional Tests
  • Acceptance Tests
  • Gradle build system and it’s commands

Code for part 4 is available on Github branch part4 and part5.

Functional / UI / Espresso Unit Test Report:

As we already know to run Functional / UI / Espresso tests we used command “connectedDebugAndroidTest”. Once we run this command Gradle will generate a report of Functional / UI / Espresso tests. Which we can see in app/build/reports/androidTests/connected/index.html as shown below:

If you open above index.html into your browser you can see your Functional / UI / Espresso tests report.

Now it’s time to integrate this report with our Jenkins server just like we did with our Unit Tests reports in the previous post of this series.
Also if you guys remember. I removed “connectedDebugAndroidTest” command from our Jenkins configuration to make focus only on Unit Test. Now I am going to add this command first into our Jenkins configuration.

Before going to start a setup of Functional / UI / Espresso tests report. We need to run our Jenkins build and we will see is Jenkin build is working properly after adding this new command.

Bad news, we got a failure on Jenkins as shown below:

Now we need to install one plugin to resolve this issue. Its name is Android Emulator Plugin.

After successful installation opens you Jenkins Build Configurations and scroll to Build Environment Section. Here select a checkbox with a text “Run an Android emulator during build”.

Here I am going to make things as simple as possible. Otherwise, this is the step which I feel is difficult in whole CICDCD process of Android. Here select “Run existing emulator” as shown above.

But from now I am not using this above Plugin instead I am going with simplest and easiest way. I will run an emulator on my machine and after that once I will run my build on local Jenkins that will automatically detect my emulator and will run my all Functional / UI / Espresso tests on this already running emulator.

Maybe you are confused why I am not using Android Emulator Plugin. Basically, there are some concepts which maybe advance for a lot of readers like how Emulator work without GUI servers ( headless build machines) or how to use an emulator on remote servers. So all these concepts need a whole separate tutorial. That is why I decided to show you Android Emulator Plugin so if someone wants to implement that concept he/she should have some starting point. Hope you guys understand. Second I am also not good so I don’t think so I can write in an easy way.

Come back to our topic. Now uncheck the “Run an Android emulator during build” option as shown below:

Now I start my emulator on my local machine as shown below:

Now this time when we will run our Jenkins build. We will not get any error instead everything will work properly.

If you try to watch parallel our emulator and Jenkins build. You can observe test will run on an emulator as shown below:

Now its time to integrate Functional / UI / Espresso tests reports with our Jenkins build. (Test report path: app/build/reports/androidTests/connected/index.html )

After saving new configuration run your build and enjoy a new testing report on your Jenkins as shown below:

Functional / UI / Espresso Test Coverage Report:

As we already install Jacoco plugin in our last post. Also, we added some new gradle scripts in our build.gradle file. Here we are going to get a benefit from our last post hard work :). To get a code coverage of Functional / UI / Espresso tests we need to give one simple command “createDebugCoverageReport”. Once that command successfully executes we can see our Code coverage report on the path app/build/reports/coverage/debug/index.html as shown below:

If you open above index.html into your browser you can see your Functional / UI / Espresso Code Coverage report.

It’s time to integrate Functional / UI / Espresso test Coverage Report with our Jenkins. So need to add this new command into our gradle task on Jenkins build as shown below:

If you observe after adding this new command I removed my other command “connectedDebugAndroidTest”. Basically, this command will run first our tests and later will generate a code coverage report. Next, integrate this report with our Jenkins.

After saving these changes again run Jenkins build and enjoy your hard work by getting new Functional / UI / Espresso test Code Coverage Report :) as shown below:

Now we completed our Functional / UI / Espresso test report and code coverage report. HURRAY.

Email:

Now we are going to integrate email with Jenkins. By adding this we will send an email once Jenkins will complete a build or maybe in case of failure we can send an email. For this first install Email-ext plugin.
There is a possibility that plugin is already installed. After the installation of this plugin. Go into Configuration System Settings.

Now scroll and find out E-mail Notification section as shown below:

Here I am using temporary Gmail account for sending a notification. You can use Gmail, Yahoo or any other one. Second, in case of Gmail if you are using two-way authentication then you need to do some extra steps. If you are doing the first time then, in my opinion, create a new Gmail account and use that without two-way authentication as shown below:

Now, its time to test our Email Notification is working or not. For that select a checkbox with label “Test configuration by sending test e-mail” as shown in above or below image:

Now click Test Configuration and wait for the response. After waiting some seconds I got a below error:

In my case, I need to enable less secure app login from my Gmail account. For that you can use below link:
https://myaccount.google.com/u/1/lesssecureapps

After giving permission to less secure app try again. In my case, Jenkins sent successfully test notification.

Email is successfully setup with Jenkins. Now, its time to add email notification in our Jenkins build. So when the build failed, abort or success we should get an email from Jenkins. For that open, the Jenkins build configuration and scroll to Post Build Section. There add Email Notification as shown below:

After save, everything starts working. Now, I am going to run build and will do intentionally failed. So, we can confirm are we getting an email or not.

Hurray, I got the email which is saying your build is failed as shown below:

I am not going to show you what type of email you will get in case of success. :)

Notifications may be on slack, hip chat:

Here I am going to implement only slack. If your company is using HipChat you can install HipChat plugin on Jenkins and follow the wiki of the plugin. You can integrate within 30 mins.
In case of Slack, install “Slack Notification” Plugin on JIRA as shown below.

Next, open a slack app and search for app “Jenkin” after that install Jenkin app as shown below:

After installation of Jenkins app on Slack. That will open a configuration page on a browser as shown below. Only try to follow this and your project will be ready with slack notifications.

After following above instruction I integrated Slack successfully with my project. I am going to share with you how I follow their instructions as shown below:

As you can see in above image. I got the testing successful message from Jenkins.
Now, I am going to run again my build and once that is complete I will share with you all notifications which I will get on slack.

Hurray, Slack Notification implementation is complete now.

Deliver APK using Fabric:

Here I have an assumption. Everybody knows what is Fabric and how you use to send a build from IDE. Now, here I am going to use Jenkins to send a build to our QA in case of success.
In step one we need to add our QA email and release notes into our build.gradle file as shown below:

If you want to add more then one emails, you can add with a comma as shown below:

ext.betaDistributionEmails = "qa1@gmail.com,qa2@gmail.com,qa3@gmail.com"
ext.betaDistributionReleaseNotes = "Release notes are available on Jenkins :)"

Next, we need to add a new gradle command into our Jenkin Build Gradle Tasks and its name is “crashlyticsUploadDistributionDebug” as shown below:

After save, run build and enjoy the Fabric automation from Jenkin:).

After a successful build, Jenkins send a build email to QA as shown above.

Deployment on play store beta:

Now, I am happy we are near to our last steps of CI, CD, CD for Android. Once everything is successfully complete on Jenkins we want to publish our app on Google Play Store. Most developers are used to generate a signed APK and upload manually. In this section, we will learn how to generate a Signed APK from Jenkins and Upload directly to Google Play Store without Human intervention. For this step, I am going to use one of my app which is Unpublished by me a long time ago on Play Store. So I will do that app again publish so I can show you guys real example and problems which you may face and how to resolve.
Note: According to my knowledge you can implement this step only for old or already published apps. For the new app, you need to do published manually the first time and after that, you can use Jenkins.

First, need to install Plugin Google Play Android Publisher as shown below:

After successful installation. Add Post Build Action “Upload Android APK on Google Play” as shown below:

Now, its time to populate above fields.
First, we are going with Specific Credentials option as shown above image. So click on “Add” -> “Jenkins”.

After clicking Jenkins you will see a form as shown below:

Next click on a Kind field and select “Google Service Account from private key” as shown below:

Next, we have two options. JSON key or p12 key. So I am going with JSON key. Next click on JSON key Choose file button and select JSON key:). Now the question is what is this JSON key and from where we will get.
Basically, now we need to create a connection between our Jenkins and Google Play Store. So this JSON key is something like credential for Jenkins to upload our APK to Google Play. Its mean this JSON key we will get from Google Play.

So do a login on Google Play Developer Console.

Next Click Settings -> Developer Account -> API Access as shown below:

Now Click on “Creare Service Account” as shown below:

After clicking a button, you will get a dialog as shown below:

Now navigate to Google API Console by clicking above dialog link and that will open for you a new tab. Now after this step we have two paths. One if someone already has some project on Google API Console then he can use that project to create a Service Account. Second, you can create a new project on Google API Console and use this new project for Service Account generation. So, I am going with the second approach.

After successful creation of a project. Select this new project as shown below:

Now search a Google Play Android Developer API or maybe scroll down you can see as shown below:

Now do Enable this API as shwon below:

Next go back to Dashboard and click on Credential as shown below:

Next, create a credential for Service Account Key as shown below:

After Click you will see a screen as shown below:

Select a new service account as shown below:

Once you will select a New Service Account. You will see a new screen give Service Account Name which you want and ignore Role field as shown below:

After clicking on a Create Button. You will get a dialog to save a JSON file. So save that file on your local machine and this file is important to secure.

Now you can close Google API Console Tab and go back to Google Play Console and if you have a popup as shown below then click Done.

Next, once you will click a Done button. You will see your API Access project automatically Link with the newly created project and everything is ready but in case if you are not lucky due to some reason. Then you need to do some extra steps.
Like my project is associated with Google Console API project but my new Service Account key is not attached as shown below:

In this case, go into “Users & Permission” and sent an invitation on email which we got when we created a Google Service JSON file on Google API Console.

If you forgot the email no problem open your Google API Console and go into Settings there you can see your email.

Now sent an invitation on this email from Google Play Console “User & Permission” Tab.

After Successful Addition you can see this new user in your “Users & Permission” Tab as shown below:

Now go back to Jenkins Credential page where we stop and add this JSON file into our Google Service Account Credential as shown below:

After click Add we will be back on our Jenkins Configuration page as shown below:

As you can see in above image. I selected my new Google Play Account credential.

Next, I need to give an APK file path. But here we have a problem. We already know we are saying Debug APK as an artifact, not a Signed APK and we know we can only Publish Signed APK on Google Play. So it means we need to a generate a Signed APK but before going to that part. We can complete other configurations in “Upload Android APK to Google Play” configuration. For this blog I am going to Publish in Beta Track and to 100% users as shown below:

Now save this configuration with An APK file path Error.

So now install a new plugin on Jenkins and its name is “Android Signing” as shown below:

Now open Jenkins project Configuration and scroll to “Build” Step and add “Sign Android APK” as shown below:

Now we need to give a Key Store file here. Basically, this is the file which we used to generate a Signed APK from Android Studio or from Terminal. Only one issue Jenkins will not work with .jks or .keystore extension file. Second, we need to add this as a certificate. So its time to start Add credential process.
Open Jenkins Dashboard and click on Credentials as shown below:

Now click on System in Credentials.

Next click on Global credentials.

Now Click Add Credential. After the click, you will see a below screen on your browser. From this screen click on a Kind field and select option “certificate” as shown below:

Next, there are two options. We are going with Upload PKCS#12 certificate. Its mean select that option as shown below:

Now click on Upload certificate button and you will get a new tab on a browser as shown below:

It’s time to give PKCS#12 certificate to Jenkins. But question is what is this certificate and from where we will get this certificate.

In simple words, “The PKCS#12 or PFX format is a binary format for storing the server certificate, any intermediate certificates, and the private key in one encryptable file.” Now we need to convert our application key-store file to PKCS#12 file. This is the same key-store file which we used to make Signed APK. All Published APK’s always have this file maybe some people are using name.jks or some are using a name.keystore but both will work.
Now, I am going to show you all steps from selection of my keystore to conversion to PKCS#12.

Next, we need to use a command which will convert our file.

keytool -importkeystore -srckeystore keystore.jks -srcstoretype JKS -deststoretype PKCS12 -destkeystore keystore.p12

A command is really simple and self-explanatory so I am going to start a conversation without explaining this command.

First is my path of a keystore file and second is a path where I want to save my .p12 file. Next click Enter.

Here give your password which you want for .p12 and click Enter.

Re-enter the same password and click Enter.

Here give your current password for .jks file and click enter.

Once all credentials are given to command and there is no issue you will get a .p12 file on your given path as shown below.

Congratulations. Now we can see our .p12 file. It’s time to upload this certificate as shown below:

After Upload, give a password and ID and click OK as shown above.

Once certificate creation is complete you can see this new certificate in your credentials as shown below:

Now go back to your Jenkins Project Configuration and scroll to Build Step. Here you will see Key Store automatically select your new certificate as shown below. (In case of you have more than one certificates then you need to select on your own).

Next, skip Key Alias field and come to “APKs to Sign” field. Basically here this plugin is asking a path of an unsigned APK and then this plugin will generate a signed APK from this unsigned APK. As we already know currently we are working with Debug APK because we are giving an assembleDebug command. So here we need to take a decision. We can change all debug keywords from our current Jenkins Configuration or we can create a new Jenkin Build by copying current build. I am going with the second approach.
Note: I am going to make a new project because in this way everything will be revised easily. :)

So go on Jenkins Dashboard and click on a new item as shown below:

Next, enter your build name and add current Jenkins build name into “Copy from” field. So Jenkins will copy for us same configuration in the new build as shshownelow:

Now click OK and then you can see two builds in our Jenkins Dashboard as shown below:

Now next open our new build and replace all paths and commands according to release build as shown below:

Note: In Release case, I am only making a Release build. No testing no fabric to make things simple. But if you want you can add testing reports as we already learn about testing in last part.
Next, give unsigned-release build path to Sign Android APK plugin as shown below:

Next, In Post-build Actions, as you can see above I am giving a new release apk path basically this APK is a signed APK which is generated by a Sign Android APK plugin. Now we are ready to Publish our this APK into Google Play Store.
Note: I am removing all test reports in release jenkns build.

Now give this Signed APK Path to “Upload Android APK to Google Play” Plugin as shown below:

Hurray, next click save and go to Jenkins Dashboard and try to click on a “Build Now” as shown below:

Once build is complete we got a below console message.

Now there is a one good news and one bad news.
Good news is our plugins worked properly. As we can see we generated a succesfully Signed APK, we successfully create a connection with Google Play. Only problem is as I already discuss with you guys I am going to use my one current app which already available on Play Console but on Jenkins I got a message. Google Play Plugin not able to find out a package name. So now I am going to change our current package name to Play stor package name and then we will try again. So guys I am going to change this package name in branch part5 on GitHub and second I am going to use part5 as a branch on Jenkins.

After saving. Again click on “Build Now” and wait for the result :).

Now this time everything is succesful except we faced one issue. As an Android Developer, we already know evey new build should have a greater version code from the previous one. Here we faced the same issue Google Play is saying you are uploading wrong version code. So we need to update our version code to 8 because my last version code is 7. But the problem is I don’t want to do manually update this version code otherwise next time I need to do this step again manually. So what we can do to achieve this automation step. That is really simple. Only write a small Grovvy funtion into your app.build file and call that function to versionCode as shown below:

def getCustomVersionCode() {
def date = new Date()
return Integer.valueOf(System.env.BUILD_NUMBER ?: date.format('yyMMddHHmm'))
}

Everything is fixed now. Basically, in this function, I am saying to Gradle. If you got the Build Number from Jenkins then put that number into version code else give current time as a version number. Its mean now if I run my code on Android Studio I will get some unique number and once this code runs on Jenkins I will get the version number from Jenkins.

Confesion: To my all readers in this post I am creating intentionally some issues. I can complete this post without creating these issues but I know without knowing these issues and their proper fixes you are not able to use this into your production also maybe you need to spend days on Google to figure it out fixes of these issues.

Next, there is one more problem which most probably some of my readers will face and that is Jenklins Build Number. For example, currently, your version code in Google Play Store is 22 but once you will try to Publish from Jenkins most probably you will face again version code issue. Because Jenkins Build Number at that time you have 1 or maybe 2. So to fix that problem you can add that offset into your function getCustomVersionCode() or you can install a Jenkins plugin “Next Build Number” as shown below:

Now I am going to assume my Google Play APK currently has 22 version code. So now I am going to change my Jenkins Build Number to 23 as shown below:

Now click Submit and click “Build Now”. You will saw a magic as shown below:

Now, as I already shared with you guys, for this tutorial I am using my current app and that app currently already have build in beta. So I am going to change my release track from beta to alpha. But you guys can use Beta, Alpha or Production.

Now after changing this change into my build I clicked “Build Now” and I got a below message from Jenkins.

HURRAY, HURRAY, HURRAY. Our app is successfully published on Google Play Store Alpha Channel as shown below:

I am going to “Build Now” again to confirm version code is automatically increasing or not.

Tag creation after publish:

Once we publish our APK we always create a Tag on our git. Here I am going to automate that step also. Its mean every time when we do a published from Jenkins, Jenkins will create a new tag in case of success.
To achieve this step we need to add a Jenkins Action Git Publisher in “Post-build Action” as shown below:

Now its time to populate Git Publisher.
The first check “Push Only if Build Succeeds”.
Next, we need to add a Tag as shown below:

In Tag to Push we need to give a Tag Name. Every organization has their own rule for Tag names but for this tutorial, I am going to use Build Number with App name like “Silence-Shout-23”. So this way, we can manage our tags easily. For example, we need to fix some bug in version code 25. I can easily check out Silence-Shout-25 and fix in this code. So we achieve the generic tag name management. Congratulations.

Now save these new changes and “Build Now” again.

Hurray, now we can see in above image. APK published successfully with tag “Silence-Shout-40”. Now you can check your new tag will be on GitHub as shown below:

Now I am going to run 3–4 times to show you a long list of tags :).

On Google Play Store we can see 41 as deactivated and 42 as an active alpha build as shown below:

Merge code with the master branch:

Now I am skipping this step.

Conclusion:

I hope you guys enjoy this part. In this part, we saw practical examples of
. Functional / UI / Espresso Unit Test Report
. Android Emulator Plugin
. Espresso Test run on local machine emulator from Jenkins
. Functional / UI / Espresso Test Coverage Report
. Email Notification integration with Jenkins
. Slack Notification integration with Jenkins
. Deliver APK using Fabric with Jenkins
. Integration between Google Play Store and Jenkins
. Creating a Google Play Service Account on Google API console
. Conversion from .keystore file to .p12 certificate
. Signed APK generation from Unsigned-Release-APK
. Deployment APK on Play Store ( Alpha, Beta, Production Channel) with Jenkins
. Auto Increment of Version Code
. Auto Tag creation of git

OK, Guys. We will meet again in next part so, for now, BYE BYE.