How to Extract and Parse Query Params in Akka HTTP
In our free Scala and Akka HTTP course we build an API for a Todo application. Original I know…
In this tutorial we will extend the API’s functionality by adding an endpoint that will allow us to search for todos.
The endpoint will accept two query parameters, we’ll extract, parse them and return the appropriate todo list.
Clone the course repo and checkout the branch 8.3-test-update-route.
Feel free to look around, when you’re ready we’ll start coding!
Implementing search in the repository
Our first step is to implement the search functionality in the repository.
If you look at our models, we’ve been creating a model for the data that each endpoint receives.
Let’s add one for our search, we want to search by text or by the done field, if the text is specified we’ll check both the title and description.
We want both values to be optional to keep our search flexible.
The model looks like:
case class SearchTodo(text: Option[String], done: Option[Boolean])Next, open the TodoRepository file, and add the following method to the trait:
def search(searchTodo: SearchTodo): Future[Seq[Todo]]It expects the model we just created, and it’ll return a sequence of todos.
The project won’t compile now as we have to implement this new method in a couple of places. Let’s start with the simplest one.
We need to update the FailingRepository in the test folder, it lives inside the TodoMocks trait. Add the following method to it:
override def search(searchTodo: SearchTodo): Future[Seq[Todo]] = Future.failed(new Exception("Mocked exception"))The entire file now looks like:
import scala.concurrent.Future
trait TodoMocks {
class FailingRepository extends TodoRepository {
override def all(): Future[Seq[Todo]] = Future.failed(new Exception("Mocked exception"))
override def done(): Future[Seq[Todo]] = Future.failed(new Exception("Mocked exception"))
override def pending(): Future[Seq[Todo]] = Future.failed(new Exception("Mocked exception"))
override def save(createTodo: CreateTodo): Future[Todo] = Future.failed(new Exception("Mocked exception"))
override def update(id: String, updateTodo: UpdateTodo): Future[Todo] = Future.failed(new Exception("Mocked exception"))
override def search(searchTodo: SearchTodo): Future[Seq[Todo]] = Future.failed(new Exception("Mocked exception"))
}
}Now we can implement the actual search login in the InMemoryTodoRepository class. We will filter the todo list step by step to keep things simple. This is a dummy in-memory implementation after all.
Override the new method:
override def search(searchTodo: SearchTodo): Future[Seq[Todo]] = Future.successful {
}We know we need to return a Future, so we'll keep using the pattern used throughout the class with Future.successful. Let's implement the function now.
First, we’ll fold over the text field, if it’s empty it means that we don’t have to filter the todo list with it, so we’ll return the todo list. If it’s not empty, we’ll check if the title or the description contains the text:
val textTodos = searchTodo.text.fold(todos)(text => todos.filter { todo =>
todo.title.contains(text) || todo.description.contains(text)
})Second, we’ll do the same over the done field, but this time we filter over the textTodos:
searchTodo.done.fold(textTodos)(done => textTodos.filter(_.done == done))Our entire function looks like:
override def search(searchTodo: SearchTodo): Future[Seq[Todo]] = Future.successful {
val textTodos = searchTodo.text.fold(todos)(text => todos.filter { todo =>
todo.title.contains(text) || todo.description.contains(text)
})
searchTodo.done.fold(textTodos)(done => textTodos.filter(_.done == done))
}Adding the search route
With our search functionality in place, it’s time to expose it by adding a new route.
We want to respond to GET requests made to the /todos/search route, and we'll accept two query parameters as mentioned earlier, done and text.
Let’s go to the Router file, and inside it take a look at the TodoRouter class. To listen to the desired route, we'll need to add it next to the "pending" endpoint:
... ~ path("pending") {
get {
handleWithGeneric(todoRepository.pending()) { todos =>
complete(todos)
}
}
} ~ path("search") {
get {
}
}With that we are listening to GET requests under the path /todos/search, now we need to handle the parameters.
To accomplish that we can use the parameters directive. We can look at the official documentation for some guidance.
It takes a list of the parameters we want to accept, by following the first example in the documentation the usage would look like:
parameters('text, 'done) { (text: String, done: String) =>
// search!
}But that would mean that both the text and done values we are provided with are strings, and that's not what we want.
We want the text value to be an Option[String] and the done to be an Option[Boolean]. We could always do the parsing/conversation manually, but let's see if Akka HTTP provides us with what we need.
According to the docs we can use the ? function to get an Option[String]:
parameters('text.?, 'done) { (text: Option[String], done: String) =>
// the text is now Option[String]
}But what about done?
Let’s start by making it a boolean, the docs show how to convert color to an Int by using the as method, let's see if we can use it to convert to a Boolean:
parameters('text.?, 'done.as[Boolean]) { (text: Option[String], done: Boolean) =>
// done is a Boolean now!
}How can we make that Boolean an optional value? The docs don't mention this specific use-case, but we can see that we can use the * method after using as like "distance".as[Int].*. Let's see if we can use the ? method instead to accomplish what we need:
parameters('text.?, 'done.as[Boolean]) { (text: Option[String], done: Option[Boolean]) =>
// finally, we got Option[Boolean]
}Now we can create an instance of our SearchTodo model. We could do it manually, but can't we do it automatically?
It turns out we can by using the as function and the model's apply method:
parameters('text.?, 'done.as[Boolean].?).as(SearchTodo) { searchTodo: SearchTodo =>
// we've got our model now!
}Let’s finish implementing the route. After getting the model, we can follow the usual pattern of handling errors with the generic directive, and then calling the search method we implemented early on in the repository:
... ~ path("search") {
get {
parameters('text.?, 'done.as[Boolean].?).as(SearchTodo) { searchTodo =>
handleWithGeneric(todoRepository.search(searchTodo)) { todos =>
complete(todos)
}
}
}
}Once we get the todos for the search, if any, we will respond to the request with them.
In the Main object, we start our application with a couple of todos, let's run it and send some search requests to our API.
I’m using HTTPie, but feel free to use curl or a client of your choice.
http localhost:9000/todos/search?done=true :
http localhost:9000/todos/search?done=false :
http localhost:9000/todos/search?text=Buy :
Feel free to play around with it, and think of any edge cases we’re not covering yet.
Testing the new route
Manual testing is fun at the beginning, you get to see working what you’ve built with your own eyes, but as your application evolves it stops being practical, or feasible.
So let’s add some tests for our search route. If you’re not familiar with how to test Akka HTTP don’t worry, it’s straightforward and there are some tests in place already, take a look the test folder next to the main one.
We want to make sure some scenarios work as expected:
- search by done
- search by text
- search by both
Let’s get started.
Create a new file under test/scala named TodoRouterSearchSpec:
import akka.http.scaladsl.model.StatusCodes
import akka.http.scaladsl.server.ValidationRejection
import akka.http.scaladsl.testkit.ScalatestRouteTest
import org.scalatest.{Matchers, WordSpec}class TodoRouterSearchSpec extends WordSpec with Matchers with ScalatestRouteTest {
import de.heikoseeberger.akkahttpcirce.FailFastCirceSupport._
import io.circe.generic.auto._}
We have the imports we’ll need by the time we finish our tests, we will use ScalaTest’s WordSpec styles, and we import Circe to be able to encode and decode our models to and from JSON.
We’ll instantiate a couple of todos for test purposes, just like our Main ones:
private val doneTodo =
Todo("2", "Buy milk", "The cat is thirsty!", done=true)
private val pendingTodo =
Todo("1", "Buy eggs", "Ran out of eggs, buy a dozen", done=false)private val todos = Seq(doneTodo, pendingTodo)
Let’s start with our first test:
"A TodoRouter" should { "search todos by done field" in {
// what goes here?! }
}
We’ll follow the patterns used in the other tests, first we’ll instantiate an in-memory repository and a router:
val repository = new InMemoryTodoRepository(todos)
val router = new TodoRouter(repository)Then we’ll make two requests, one to search for done todos and another one for pending todos. We’ll assert that we get the correct status code and response:
Get("/todos/search?done=true") ~> router.route ~> check {
status shouldBe StatusCodes.OK
val respTodos = responseAs[Seq[Todo]]
respTodos shouldBe Seq(doneTodo)
}
Get("/todos/search?done=false") ~> router.route ~> check {
status shouldBe StatusCodes.OK
val respTodos = responseAs[Seq[Todo]]
respTodos shouldBe Seq(pendingTodo)
}Our test case looks like:
"search todos by done field" in {
val repository = new InMemoryTodoRepository(todos)
val router = new TodoRouter(repository) Get("/todos/search?done=true") ~> router.route ~> check {
status shouldBe StatusCodes.OK
val respTodos = responseAs[Seq[Todo]]
respTodos shouldBe Seq(doneTodo)
} Get("/todos/search?done=false") ~> router.route ~> check {
status shouldBe StatusCodes.OK
val respTodos = responseAs[Seq[Todo]]
respTodos shouldBe Seq(pendingTodo)
}
}
For our next test we’ll perform two requests as well, one for the title and another one for the description:
"search todos by text" in {
val repository = new InMemoryTodoRepository(todos)
val router = new TodoRouter(repository)
Get("/todos/search?text=dozen") ~> router.route ~> check {
status shouldBe StatusCodes.OK
val respTodos = responseAs[Seq[Todo]]
respTodos shouldBe Seq(pendingTodo)
}
Get("/todos/search?text=Buy") ~> router.route ~> check {
status shouldBe StatusCodes.OK
val respTodos = responseAs[Seq[Todo]]
respTodos shouldBe todos
}
}Quite similar to the first one, but this time we send the text query parameter instead.
We’ll add one more test for our happy path, where we’ll search for both:
"search todos by both" in {
val repository = new InMemoryTodoRepository(todos)
val router = new TodoRouter(repository)
Get("/todos/search?done=true&text=egg") ~> router.route ~> check {
status shouldBe StatusCodes.OK
val respTodos = responseAs[Seq[Todo]]
respTodos shouldBe Seq.empty
}
}Feel free to add more test cases and play around with them.
Validating the params
You might be wondering: “What if we get invalid data?”. And you’d be right to worry, we can’t trust users to send us valid data.
But if Akka HTTP is creating the model for us, how can we validate it?
We can always validate it after it’s created, but wouldn’t it be nice if Akka HTTP could handle that and once we get the model we can use it straight away?
Well, of course it’s possible… Otherwise I wouldn’t have said that! 😉
Joking aside, there are multiple ways to accomplish this, we’ll use a simple one in this tutorial.
We’ll make use of Scala’s require (if checking the link, look for the last occurrences when searching for require) method.
Akka HTTP will use those constraints to reject the request if they aren’t fulfilled.
We will cover two cases:
- if the string is specified it can’t be empty, e.g.
/todos/search?text=should be rejected - at least one parameter should be specified
The model has to be updated to include those validations, let’s do that next:
case class SearchTodo(text: Option[String], done: Option[Boolean]) {
require(text.fold(true)(_.nonEmpty), "If specified, the text can't be empty")
require(text.nonEmpty || done.nonEmpty, "At least one parameter has to be specified")
}We can even write our own error message, cool!
Let’s write a test to verify the first case:
"not search with empty text" in {
val repository = new InMemoryTodoRepository(todos)
val router = new TodoRouter(repository) Get("/todos/search?text=") ~> router.route ~> check {
rejection shouldBe a[ValidationRejection]
}
}
What’s new here is how we’re asserting that the request should be rejected. Akka HTTP’s testkit let us know with the rejection method.
We assert that the value the rejection method returns should have a type of ValidationRejection, which is what Akka HTTP would return.
Run the test and make sure it passes.
But how does that rejection look when actually making a request? Let’s run the Main object and see for ourselves:
http localhost:9000/todos/search?text= :
We see a 400 Bad Request which is what we'd expect, but note that we also get our error message requirement failed: If specified, the text can't be empty. Great!
Our last test will make sure that our search endpoint won’t be used to search for all the todos, for that we have a different endpoint. (No specific reason 😁)
Quite similar to the previous one, but this time we specify no parameters. Let’s test it with an HTTP client:
http localhost:9000/todos/search :
Conclusion
We learned how to extract, parse and even validate query params in our API!
Akka HTTP provides a lot of utilities that cover most use-cases, and if they don’t, you have the option to extend the library yourself with powerful primitives.
Hope you enjoyed the tutorial and we’ll see you soon!
Originally published at https://www.codemunity.io.
