API Testing Patterns for the Play Framework
A robust suite of end-to-end tests for your APIs gives you the confidence to iterate and move quickly. With APIs being easier and faster to test than web frontends, the invaluable benefits of testing come at a low cost of investment.
Two of the key challenges when creating API tests are expressing the desired system-level behaviors precisely and, more technically focused, deciding how to cover external dependencies.
After a few years of working with the Play Framework I’d like to share a few of the testing patterns I am using to both solve the challenges just mentioned and create tests effortlessly and productively.
Sample code to accompany this blog post, including a vagrant dev environment, is available on my github.
Expressive, Behaviour-focused Tests
I am a DDD practitioner (and author) who obsesses over linguistic precision. Before I start coding tests I try to explain as expressively as possible the behaviour of the feature I am writing tests for.
For that reason I love ScalaTest’s FreeSpec. It allows me to write free-form, arbitrarily-nested, syntax-free specifications like the following:
class notifications_latest extends APITest {
import TestData._
val dbScript = "/api/notifications.sql""When requesting the latest notifications" - {
startApi()
val httpRes = await(WS.url(s"http://localhost:$port/notifications/latest").get())"A 200 HTTP resonse is returned" in {
assert(httpRes.status === 200)
}"The most recent notifications are returned as JSON in the response body" in {
val notifications = Json.parse(httpRes.body).as[Seq[APINotification]]
assert(notifications === testNotifications)
}
}
...
}
Free-form certainly has it’s drawbacks compared to structured languages like given, when, then but overall it has consistently allowed me to write expressive descriptions that are easy to comprehend whenever I return to the code.
Behaviour First, Implementation Details Second
I want the person who comes along after me to understand what my tests are doing and why. For this reason I obsessively follow the regime of putting all plumbing and test data at bottom of test files. What you saw in the last code is typical of what you’d see in any test file I write — behaviour first.
In the previous code snippet you can see that there is a static import: TestData._ Each test file has a nested TestData class that contains all the test data. Only when you’ve understood the behaviours I am writing tests for do I want to throw low-level setup and implementation details at you.
Here is an example TestData nested class implementation taken from the code sample. I could not sleep at night knowing that when people open my test files, that clutter is what they will be greeted with.
class notifications_latest extends APITest {
...object TestData {
// These correspond to the insertions in "$dbScript"
val testNotifications = Seq(
APINotification("test room1", "user 1", "This room is so amazing", "2015-01-01 00:00:00.0"),
APINotification("test room2", "user 2", "This room is so amazing", "2015-01-01 00:00:00.0"),
APINotification("test room3", "user 3", "This room is so amazing", "2015-01-01 00:00:00.0"),
APINotification("test room4", "user 4", "This room is so amazing", "2015-01-01 00:00:00.0"),
APINotification("test room5", "user 5", "This room is so amazing", "2015-01-01 00:00:00.0"),
APINotification("test room6", "user 6", "This room is so amazing", "2015-01-01 00:00:00.0"),
APINotification("test room7", "user 7", "This room is so amazing", "2015-01-01 00:00:00.0"),
APINotification("test room8", "user 8", "This room is so amazing", "2015-01-01 00:00:00.0"),
APINotification("test room9", "user 9", "This room is so amazing", "2015-01-01 00:00:00.0"),
APINotification("test room10", "user 10", "This room is so amazing", "2015-01-01 00:00:00.0")
)
}
}API Lifecycle ManagementTo avoid writing the same boilerplate setup and tear down in every test file I create a base class - expediting the creation process and eliminating the fast-acting lethargy induced by disconcerting thoughts of having to write laborious test plumbing.
The APITest base class from my sample code is shown below. It creates a test server for each test and manages that server's lifecycle, carrying out activities like ensuring the environment is clean and uncontaminated from previous tests.trait APITest extends FreeSpecLike with Matchers with BeforeAndAfterAll with FutureAwaits with DefaultAwaitTimeout {val port: Int = 1000 + Random.nextInt(8999)
val dbScript: Stringlazy val server: TestServer = CreateTestApplicationWithTestDB(port, dbScript)def startApi() = {
info(s"Starting API test: ${getClass.getName} on port: ${server.port}")
info(s"Running /db.sql against $dbUrl")
DBScriptRunner.runDbScript(dbUrl, username, password, "/db.sql")
info(s"Running $dbScript against $dbUrl")
DBScriptRunner.runDbScript(dbUrl, username, password, dbScript)
}override def afterAll() {
server.stop()
}
}
This sample application is backed by a MySQL database. Accordingly, my APITest base classes cleans out completely the “apitests” database. It runs the main database configuration script (db.sql) and a specific setup script used for each test - the abstract dbScript val.
Note: I use the ibatis script runner library to set up and tear down the database. You can see in the sample code how I put it to use inside DBScriptRunner. I purposely avoid using code from the main application so that the tests have as little coupling to the production code as possible to avoid behaviour not being tested.
Conveniently, as shown in the first code sample, each api test I write just has to implement APITest, provide the name of a database script it would like to run, and then call startApi() (shown above) and it’s ready to start testing behaviour….
…well almost. There’s still an issue with external dependencies. In this case, how does the running application know to use the “apitests” database?
Stubbing External Dependencies
Play Framework allows you to override dependencies in your tests. You can benefit from this immensely by stubbing or redirecting external dependencies, like databases and external APIs.
There’s always a trade-off when stubbing external dependencies. The more of your production code that is stubbed during tests, obviously, the less you are testing.
When stubbing dependencies in API tests, I try to create very thin layers that contain only the low level code for talking to external dependencies. HTTP clients are common example rather than an entire class that builds requests, parses responses etc.
In the sample application I am injecting a test database. Instead of pointing the application to the main “chatroomz” database running on a MySQL server somewhere. I wanted to point it to the “apitests” database running on a local MySQL server.
Note: I could have used an in-memory database to increase performance, and sometimes that’s a rewarding decision. However, in-memory databases don’t behave like the real implementations, so at the cost of slower tests I prefer to use the real database implementation until the performance penalty gets too high.
Overriding Dependency Injection Bindings During Testing
The way I point the application to an alternative database is to override the dependency-injection binding. In Play 2.4, Guice is the default DI framework. Here’s how the sample code wires up it’s dependencies in the main application using a custom module:
class DependencyWiring extends AbstractModule {
override def configure() {
bind(classOf[Database]).toInstance(Database.forConfig("chatroomz"))
bind(classOf[NotificationsRetriever]).to(classOf[DatabaseNotificationsRetriever])
}
}With very little effort I can override the Database binding, or any binding, in an API test and point it to another instance. All that is required is a call to configure() on an application builder that builds the test version of the application:
object CreateTestApplicationWithTestDB {
import api.configuration.TestDatabase._
val m = // test database configuration (see sample code for full details)
val dbConfig = ConfigFactory.parseMap(m)def apply(port: Int, dbScript: String) = {
val db = Database.forConfig("chatroomz", dbConfig)
val app = new GuiceApplicationBuilder()
.configure(Map("logger.root" -> "ERROR", "logger.application" -> "ERROR"))
.overrides(bind[Database].toInstance(db)) // <- override binding in main app
.buildval server = TestServer(port, app)
server.start()
server
}
When creating the test application you just use a custom Guice application loader and specify the override. All of the other dependencies declared in the main application will remain intact.Note: Play’s official documentation contains plenty of detailed advice about dependency injection and test overriding.
Note: An easier way to point the test application at the "apitests" database would have been to pass in custom configuration into Play’s FakeApplication. But that doesn’t work for other dependencies like an embedded Elasticsearch test node or a HTTP client.
Other Dependencies
Other dependencies I have found myself stubbing during API tests are datastores, like Elasticsearch, HTTP clients and interestingly Play's Actor System.If you find yourself needing to wait for events to occur in your tests, and cannot see beyond the much-maligned Thread.sleep(), consider injecting an Actor System and using it’s event stream to get notifications out of your running app.
On a recent task I had to wait for a background process to index some data into Elasticsearch from the main datastore before I could test the indexed data was served up by the API.
In the application I was able to fire an IndexingComplete event which the API test waited for before performing the search. I had to leak a bit of the innards of the application, which I’d prefer not to, but it saved having to put painstakingly-generous sleeps to avoid having flaky tests.
Full Sample Code Available — Comments Welcome
As I've mentioned, for a working demonstration there's a small sample application on my github that shows all of these concepts in practice. I’d still love to hear your thoughts and see what patterns you use. Feel free to send a pull request if you have any good ideas for addition or enhancement to the sample app.
