Running your tests on the command line with Gradle
In addition to running your tests in Android Studio you can also run them from the command line. This is useful for automated build scripts, CI servers, or when Android Studio is crashing on launch and needs to be reinstalled.
Use the command ./gradlew test
to run all tests.
$ ./gradlew test
Incremental java compilation is an incubating feature.
:app:preBuild UP-TO-DATE
:app:preDebugBuild UP-TO-DATE
:app:checkDebugManifest
:app:preReleaseBuild UP-TO-DATE
:app:prepareComAndroidSupportAnimatedVectorDrawable2501Library
:app:prepareComAndroidSupportAppcompatV72501Library
:app:prepareComAndroidSupportDesign2501Library
:app:prepareComAndroidSupportRecyclerviewV72501Library
:app:prepareComAndroidSupportSupportCompat2501Library
:app:prepareComAndroidSupportSupportCoreUi2501Library
:app:prepareComAndroidSupportSupportCoreUtils2501Library
:app:prepareComAndroidSupportSupportFragment2501Library
:app:prepareComAndroidSupportSupportMediaCompat2501Library
:app:prepareComAndroidSupportSupportV42501Library
:app:prepareComAndroidSupportSupportVectorDrawable2501Library
:app:prepareComAndroidSupportTransition2501Library
:app:prepareDebugDependencies
:app:compileDebugAidl
:app:compileDebugRenderscript
:app:generateDebugBuildConfig
:app:generateDebugResValues
:app:generateDebugResources
:app:mergeDebugResources
:app:processDebugManifest
:app:processDebugResources
:app:generateDebugSources
:app:incrementalDebugJavaCompilationSafeguard
:app:compileDebugJavaWithJavac
:app:compileDebugJavaWithJavac - is not incremental (e.g. outputs have changed, no previous execution, etc.).
:app:incrementalDebugUnitTestJavaCompilationSafeguard UP-TO-DATE
:app:preDebugUnitTestBuild UP-TO-DATE
:app:prepareDebugUnitTestDependencies
:app:compileDebugUnitTestJavaWithJavac
:app:processDebugJavaRes UP-TO-DATE
:app:processDebugUnitTestJavaRes UP-TO-DATE
:app:compileDebugUnitTestSources
:app:mockableAndroidJar
:app:assembleDebugUnitTest
:app:testDebugUnitTest
:app:checkReleaseManifest
:app:prepareReleaseDependencies
:app:compileReleaseAidl
:app:compileReleaseRenderscript
:app:generateReleaseBuildConfig
:app:generateReleaseResValues
:app:generateReleaseResources
:app:mergeReleaseResources
:app:processReleaseManifest
:app:processReleaseResources
:app:generateReleaseSources
:app:incrementalReleaseJavaCompilationSafeguard
:app:compileReleaseJavaWithJavac
:app:compileReleaseJavaWithJavac - is not incremental (e.g. outputs have changed, no previous execution, etc.).
:app:incrementalReleaseUnitTestJavaCompilationSafeguard UP-TO-DATE
:app:preReleaseUnitTestBuild UP-TO-DATE
:app:prepareReleaseUnitTestDependencies
:app:compileReleaseUnitTestJavaWithJavac
:app:processReleaseJavaRes UP-TO-DATE
:app:processReleaseUnitTestJavaRes UP-TO-DATE
:app:compileReleaseUnitTestSources
:app:assembleReleaseUnitTest
:app:testReleaseUnitTest
:app:test
BUILD SUCCESSFUL
Total time: 6.517 secs
If we dig into the output we can see that both :app:testDebugUnitTest
and :app:testReleaseUnitTest
were executed.
Unfortunately that is all we see and it is buried in a long list of build steps. A quick glance at the output might leave you wondering if all of your tests ran or even any at all.
If there is a failing test we at least get a bit more output about that specific test case.
$ ./gradlew test
Incremental java compilation is an incubating feature.
:app:preBuild UP-TO-DATE
:app:preDebugBuild UP-TO-DATE
:app:checkDebugManifest
:app:preReleaseBuild UP-TO-DATE
:app:prepareComAndroidSupportAnimatedVectorDrawable2501Library UP-TO-DATE
:app:prepareComAndroidSupportAppcompatV72501Library UP-TO-DATE
:app:prepareComAndroidSupportDesign2501Library UP-TO-DATE
:app:prepareComAndroidSupportRecyclerviewV72501Library UP-TO-DATE
:app:prepareComAndroidSupportSupportCompat2501Library UP-TO-DATE
:app:prepareComAndroidSupportSupportCoreUi2501Library UP-TO-DATE
:app:prepareComAndroidSupportSupportCoreUtils2501Library UP-TO-DATE
:app:prepareComAndroidSupportSupportFragment2501Library UP-TO-DATE
:app:prepareComAndroidSupportSupportMediaCompat2501Library UP-TO-DATE
:app:prepareComAndroidSupportSupportV42501Library UP-TO-DATE
:app:prepareComAndroidSupportSupportVectorDrawable2501Library UP-TO-DATE
:app:prepareComAndroidSupportTransition2501Library UP-TO-DATE
:app:prepareDebugDependencies
:app:compileDebugAidl UP-TO-DATE
:app:compileDebugRenderscript UP-TO-DATE
:app:generateDebugBuildConfig UP-TO-DATE
:app:generateDebugResValues UP-TO-DATE
:app:generateDebugResources UP-TO-DATE
:app:mergeDebugResources UP-TO-DATE
:app:processDebugManifest UP-TO-DATE
:app:processDebugResources UP-TO-DATE
:app:generateDebugSources UP-TO-DATE
:app:incrementalDebugJavaCompilationSafeguard UP-TO-DATE
:app:compileDebugJavaWithJavac UP-TO-DATE
:app:incrementalDebugUnitTestJavaCompilationSafeguard UP-TO-DATE
:app:preDebugUnitTestBuild UP-TO-DATE
:app:prepareDebugUnitTestDependencies
:app:compileDebugUnitTestJavaWithJavac
:app:processDebugJavaRes UP-TO-DATE
:app:processDebugUnitTestJavaRes UP-TO-DATE
:app:compileDebugUnitTestSources
:app:mockableAndroidJar UP-TO-DATE
:app:assembleDebugUnitTest
:app:testDebugUnitTest
com.ecgreb.mytestapp.ExampleUnitTest > thisTestShouldFail FAILED
java.lang.AssertionError at ExampleUnitTest.java:23
3 tests completed, 1 failed
:app:testDebugUnitTest FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':app:testDebugUnitTest'.
> There were failing tests. See the report at: file:///usr/local/code/my-test-app/app/build/reports/tests/debug/index.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 1.622 secs
Wouldn’t it be nice to see how many tests were run, which passed, which failed, and how many were skipped? With a small change to your build.gradle
file you can.
tasks.withType(Test) {
testLogging {
exceptionFormat "full"
events "started", "skipped", "passed", "failed"
showStandardStreams true
}
}
Add this block to your build.gradle
then re-run your tests on the command line. Go ahead — I’ll wait.
$ ./gradlew test
Incremental java compilation is an incubating feature.
:app:preBuild UP-TO-DATE
:app:preDebugBuild UP-TO-DATE
:app:checkDebugManifest
:app:preReleaseBuild UP-TO-DATE
:app:prepareComAndroidSupportAnimatedVectorDrawable2501Library
:app:prepareComAndroidSupportAppcompatV72501Library
:app:prepareComAndroidSupportDesign2501Library
:app:prepareComAndroidSupportRecyclerviewV72501Library
:app:prepareComAndroidSupportSupportCompat2501Library
:app:prepareComAndroidSupportSupportCoreUi2501Library
:app:prepareComAndroidSupportSupportCoreUtils2501Library
:app:prepareComAndroidSupportSupportFragment2501Library
:app:prepareComAndroidSupportSupportMediaCompat2501Library
:app:prepareComAndroidSupportSupportV42501Library
:app:prepareComAndroidSupportSupportVectorDrawable2501Library
:app:prepareComAndroidSupportTransition2501Library
:app:prepareDebugDependencies
:app:compileDebugAidl
:app:compileDebugRenderscript
:app:generateDebugBuildConfig
:app:generateDebugResValues
:app:generateDebugResources
:app:mergeDebugResources
:app:processDebugManifest
:app:processDebugResources
:app:generateDebugSources
:app:incrementalDebugJavaCompilationSafeguard
:app:compileDebugJavaWithJavac
:app:compileDebugJavaWithJavac - is not incremental (e.g. outputs have changed, no previous execution, etc.).
:app:incrementalDebugUnitTestJavaCompilationSafeguard UP-TO-DATE
:app:preDebugUnitTestBuild UP-TO-DATE
:app:prepareDebugUnitTestDependencies
:app:compileDebugUnitTestJavaWithJavac
:app:processDebugJavaRes UP-TO-DATE
:app:processDebugUnitTestJavaRes UP-TO-DATE
:app:compileDebugUnitTestSources
:app:mockableAndroidJar
:app:assembleDebugUnitTest
:app:testDebugUnitTest
com.ecgreb.mytestapp.ExampleUnitTest > mainActivity_shouldNotBeNull STARTED
com.ecgreb.mytestapp.ExampleUnitTest > mainActivity_shouldNotBeNull PASSED
com.ecgreb.mytestapp.ExampleUnitTest > addition_isCorrect STARTED
com.ecgreb.mytestapp.ExampleUnitTest > addition_isCorrect PASSED
:app:checkReleaseManifest
:app:prepareReleaseDependencies
:app:compileReleaseAidl
:app:compileReleaseRenderscript
:app:generateReleaseBuildConfig
:app:generateReleaseResValues
:app:generateReleaseResources
:app:mergeReleaseResources
:app:processReleaseManifest
:app:processReleaseResources
:app:generateReleaseSources
:app:incrementalReleaseJavaCompilationSafeguard
:app:compileReleaseJavaWithJavac
:app:compileReleaseJavaWithJavac - is not incremental (e.g. outputs have changed, no previous execution, etc.).
:app:incrementalReleaseUnitTestJavaCompilationSafeguard UP-TO-DATE
:app:preReleaseUnitTestBuild UP-TO-DATE
:app:prepareReleaseUnitTestDependencies
:app:compileReleaseUnitTestJavaWithJavac
:app:processReleaseJavaRes UP-TO-DATE
:app:processReleaseUnitTestJavaRes UP-TO-DATE
:app:compileReleaseUnitTestSources
:app:assembleReleaseUnitTest
:app:testReleaseUnitTest
com.ecgreb.mytestapp.ExampleUnitTest > mainActivity_shouldNotBeNull STARTED
com.ecgreb.mytestapp.ExampleUnitTest > mainActivity_shouldNotBeNull PASSED
com.ecgreb.mytestapp.ExampleUnitTest > addition_isCorrect STARTED
com.ecgreb.mytestapp.ExampleUnitTest > addition_isCorrect PASSED
:app:test
BUILD SUCCESSFUL
Total time: 6.428 secs
There isn’t that better? Now at a glance we can see which tests ran and the result of each. Twice. Once for debug
and once for release
. Of course you can customize the logging level to taste by adding or removing an event
in the task filter.
One last note — if you want to run your tests for only one build type (this can be important for a large test suite) you can use ./gradlew testDebug
or ./gradlew testRelease
.
$ ./gradlew testDebug
Incremental java compilation is an incubating feature.
:app:preBuild UP-TO-DATE
:app:preDebugBuild UP-TO-DATE
:app:checkDebugManifest
:app:preReleaseBuild UP-TO-DATE
:app:prepareComAndroidSupportAnimatedVectorDrawable2501Library
:app:prepareComAndroidSupportAppcompatV72501Library
:app:prepareComAndroidSupportDesign2501Library
:app:prepareComAndroidSupportRecyclerviewV72501Library
:app:prepareComAndroidSupportSupportCompat2501Library
:app:prepareComAndroidSupportSupportCoreUi2501Library
:app:prepareComAndroidSupportSupportCoreUtils2501Library
:app:prepareComAndroidSupportSupportFragment2501Library
:app:prepareComAndroidSupportSupportMediaCompat2501Library
:app:prepareComAndroidSupportSupportV42501Library
:app:prepareComAndroidSupportSupportVectorDrawable2501Library
:app:prepareComAndroidSupportTransition2501Library
:app:prepareDebugDependencies
:app:compileDebugAidl
:app:compileDebugRenderscript
:app:generateDebugBuildConfig
:app:generateDebugResValues
:app:generateDebugResources
:app:mergeDebugResources
:app:processDebugManifest
:app:processDebugResources
:app:generateDebugSources
:app:incrementalDebugJavaCompilationSafeguard
:app:compileDebugJavaWithJavac
:app:compileDebugJavaWithJavac - is not incremental (e.g. outputs have changed, no previous execution, etc.).
:app:incrementalDebugUnitTestJavaCompilationSafeguard UP-TO-DATE
:app:preDebugUnitTestBuild UP-TO-DATE
:app:prepareDebugUnitTestDependencies
:app:compileDebugUnitTestJavaWithJavac
:app:processDebugJavaRes UP-TO-DATE
:app:processDebugUnitTestJavaRes UP-TO-DATE
:app:compileDebugUnitTestSources
:app:mockableAndroidJar
:app:assembleDebugUnitTest
:app:testDebugUnitTest
com.ecgreb.mytestapp.ExampleUnitTest > mainActivity_shouldNotBeNull STARTED
com.ecgreb.mytestapp.ExampleUnitTest > mainActivity_shouldNotBeNull PASSED
com.ecgreb.mytestapp.ExampleUnitTest > addition_isCorrect STARTED
com.ecgreb.mytestapp.ExampleUnitTest > addition_isCorrect PASSED
BUILD SUCCESSFUL
Total time: 4.017 secs
Happy testing!