Analyse your code with SonarQube
Sonarqube is a software for checking your code for bugs, security problems and plain bad writing.
In a few minutes it can tell you where in your code you have duplicated chunks or where you might be causing a bug by not calling a package correctly. When used frequently (as a part of a CI pipeline, for example) it can help maintain your code to a good standard.
In this tutorial we will look at installation using Docker and running an initial analysis. The analysed project will be the pandas code as it is:
- open source
- very widely used in data science
- large enough and old enough to have at least some kinks SonarQube can point out.
For any errors during execution, please see the Errors section down below.
How to set up SonarQube
Docker files enable us to execute software on various systems. This is what we will use to install SonarQube. If for now you are unfamiliar or uncomfortable with Docker, you can use a standard installation as well. You can find the instructions for all installations here.
Pull the images
The Docker command to pull the latest version of the image at the time of writing is
docker pull sonarqube:9.4-communityThis will pull a docker image named sonarqube, version 9.4-community.
You will also need the SonarScanner, which you can install as a program or as another Docker container.
docker pull sonarsource/sonar-scanner-cliYou can see instructions for the SonarScanner installation here.
The various SonarQube versions live here.
Run a SonarQube container
The command below creates a container which will show you the analysis of your code.
docker run -d — name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:latestHere is what each part of the command means:
- docker → shows you are using docker to execute the rest of the commands.
- run → create a container
- -d → run in a detached state. If you do not put this command in, the command line window will remain attached to the container and you will be able to see all the messages written in the container by the programs executing in it. This is useful in debugging mode, not so much when there are no issues.
- - -name (two dashes with no space) → this will be name of the container. It needs to be unique for the system. Once the container is created, next time you can use it by calling
docker start name_of_container- -e sets environment variables. As discussed here, “This will disable some checks that a production level SonarQube determines are necessary with its ElasticSearch integration and could otherwise break your build. Generally, these are memory checks to make sure there’s enough available for smooth operation.”
- -p matches machine port 9000 (first number) to container port 9000(second number). This means that whatever the container writes on its own 9000 port will show up on our machine’s 9000 port. In this case that is where you can go and check the results of the analysis. (localhost:9000)
- sonarqube:latest — lastly, the image from which the container should be created. If it does not exist locally it will be pulled from the Docker Hub.
Analyse a project
Once you have started a container, go to localhost:9000. You might need to give it 2 mins after starting the container to make sure everything is up and running. It will ask you for a user name and password. Both are admin. Then it will ask you to change your password. Once you have done that, you will see the screen below or something similar, depending on which SonarQube version you are using.
We will go with a manual project creation out of the options included here, since that has no dependencies.
The next screen requires a project display name (for this tutorial “pandas”) and project key, which is automatically populated with the project name.
The question of how to analyse (next page) we’ll answer with “Locally”.
Each time we analyse we will need to supply the Sonar Scanner with a key, so we set that key name here as well. Take note of what the generated key ends up being.
The second part of the page asks for what type of project it is. Since it is a Python one we will chose the “Other” version. On the next pop up choose your system version as well.
The next prompt will tell you what you need to do to get the Sonar Scanner, if you have not yet.
The required command on a Windows machine when working with Sonar Scanner downloaded as a zip file is:
sonar-scanner.bat -D”sonar.projectKey={name of the project}” -D”sonar.sources=.” -D”sonar.host.url=http://localhost:9000" -D”sonar.login={key from earlier page}”Inside the root of the folder you want to analyse put the settings below in a file named sonar-project.properties. Here you can specify where your testing directory is, the encoding of the files, settings for what gets flagged as a problem, etc.
You can see the various configs here.
Results page
The results page for pandas library, commit 1ce21664848a1128ea1456d4fd24d76f045a74df, looks as below:
There will be a follow-up article to this one going into a bit more detail as to what the issues detected represent (and how much of an issue they are).
Errors when using SonarQube
Elastic search
You might get an error related to Elastic Search during your first run. Elastic search is a system for storing data. When the Docker image tries to create a container, it takes the sonarqube zip file from https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-${SONARQUBE_VERSION}.zip. I assume as a part of it the zip file uses Elastic search, probably to store metrics and flags.
To sort out this error try:
- switching to a different docker image — the one you have chosen might be too much for your system.
- if you are on Windows and you are using Hyper-V as your backend (possible if you have a Windows Pro version), try switching to wsl as the backend. You can see here how to do that. It has explanations on how to install wsl and a Linux distribution afterwards.
The local host does not give you any data
When executing the command to create the container, you cannot just run docker run sonarqube . You have to put in the mapping of the ports in there as well, using the argument -p 9000:9000.
Notes
“SONARQUBE” is a trademark belonging to SonarSource SA. You can read more about them and approved logo usage on their page.
