Building an app with elixir-3

previous post, source code

Disclaimer: I’m a beginner in elixir, so this is not an ideal way of doing things in elixir. I’m just writing my experience on learning elixir.

So in the last post we ran some CRUD operation to our database. In this post we’ll write some functions to get latest tweets of different languages via twitter search api. First create a languages table to store all the languages we want to fetch tweets.

iex(1)> LanguageTrends.Query.create_table("languages")
{:ok,
 %RethinkDB.Record{data: %{"config_changes" => [%{"new_val" => %{"db" => "language_trends",
         "durability" => "hard", "id" => "cf275a3c-6984-47a6-9ad2-a292917024a0",
         "indexes" => [], "name" => "languages", "primary_key" => "id",
         "shards" => [%{"nonvoting_replicas" => [],
            "primary_replica" => "elixir_elixir_local_v5q",
            "replicas" => ["elixir_elixir_local_v5q"]}],
         "write_acks" => "majority"}, "old_val" => nil}],
    "tables_created" => 1}, profile: nil}}

Now lets add some languages with their hashtag to the table

iex(1)> LanguageTrends.Query.insert_document("languages", %{"name" => "elixir", "hashtag" => "elixirlang"})
{:ok,
 %RethinkDB.Record{data: %{"deleted" => 0, "errors" => 0,
    "generated_keys" => ["8499795f-dd1b-4d25-9501-43550f0942b1"],
    "inserted" => 1, "replaced" => 0, "skipped" => 0, "unchanged" => 0},
  profile: nil}}

Add some more languages like the above. I’ve added few mores and lets see all the languages I’ve added

iex(2)> LanguageTrends.Query.get_document("languages")
{:ok,
 %RethinkDB.Collection{data: [%{"hashtag" => "clojure",
     "id" => "b08e8940-7c74-4bad-9a8f-e1259ba8ecc5", "name" => "clojure"},
   %{"hashtag" => "rustlang", "id" => "759cd575-df54-425d-abc9-116c46517610",
     "name" => "rust"},
   %{"hashtag" => "scala", "id" => "fc17ad49-928b-4156-9229-82cfa3a85ede",
     "name" => "scala"},
   %{"hashtag" => "kotlin", "id" => "651a6716-9b53-4caa-91a8-05c9e05f55e4",
     "name" => "kotlin"},
   %{"hashtag" => "swift", "id" => "0649e075-ae83-48a4-8b21-2cea0e0b2124",
     "name" => "swift"},
   %{"hashtag" => "haskell", "id" => "d387f0b2-a7c6-489d-8757-a16164ac6742",
     "name" => "haskell"},
   %{"hashtag" => "golang", "id" => "b1666a91-7a8f-4cd8-8ab4-9cb22c93e02a",
     "name" => "go"},
   %{"hashtag" => "crystallang", "id" => "4988fb3a-cbae-4086-8fbd-813a3e3ff1fe",
     "name" => "crystal"},
   %{"hashtag" => "elmlang", "id" => "6bb447b6-3bc4-485b-be4d-3d0ca42cba11",
     "name" => "elm"},
   %{"hashtag" => "elixirlang", "id" => "8499795f-dd1b-4d25-9501-43550f0942b1",
     "name" => "elixir"},
   %{"hashtag" => "julialang", "id" => "e744a5f9-055c-41fa-b2ad-473905175a92",
     "name" => "julia"], profile: nil}}

Next we’ll create a file twitter.ex in lib/language_trends folder where we’ll write all the api call to twitter. We’ll use a library called twittex to make the api call. Add the library name in mix.exs file

defp deps do
  [
    {:rethinkdb, github: "hamiltop/rethinkdb-elixir"},
    {:twittex, github: "almightycouch/twittex"},
    {:poison, "~> 2.0", override: true}
  ]
end

I’ve also added poison library (a json parser) because twittex and rethinkdb require different version of poison which conflicts at compile time, so we added it manually and locked the version with option override so that other library can’t upgrade it to another version.\.

Now open twitter.ex file and add this

defmodule LanguageTrends.Twitter do
  import Twittex.Client
end

First I defined the module name LanguageTrends.Twitter and imported Twittex.Client from twittex library. Lets write a function for fetching tweets by hashtag

def search_with_hashtag(hashtag, since_id \\ nil, count \\ 100) do
    search("#" <> hashtag <> "-filter:retweets", since_id: since_id,
    count: count, result_type: "recent")
end

Above function need one required argument and two optional argument. First argument is hashtag that’ll be used to search tweets. count is to determine how many tweets we want to retrieve in our api call (Twitter returns max 100 tweets in a single call). And we’ll use since_id later. Within the function we are calling search function of twittex library passing four arguments. First we are adding pound # symbol in front of hashtag argument and “-filter:retweets” at the end to filter out the retweets. <> is used for string concatenation in elixir. Fourth argument is result_type, which is “recent” to fetch only recent tweets. Let’s run this function now

Response of search api call

We passed “elixirlang” as hashtag, 0 as since_id and 1 as count to get only 1 recent tweets. Twitter api has returned a lot of data in response. We need two information from this response, one is max_id and the other is the count of statuses. So let’s extract those two information

def search_with_hashtag(hashtag, since_id \\ nil, count \\ 100) do
    search("#" <> hashtag <> "-filter:retweets", since_id: since_id,
    count: count, result_type: "recent")
    |> (fn {:ok, x} -> %{"since_id" => get_in(x, ["search_metadata",
    "max_id"]), "statuses" => Enum.count(get_in(x, ["statuses"]))}
    end).()
end

We are piping a anonymous function to the output of our search method call. Anonymous function in elixir written in following format

fn x, y -> x + y end

Anonymous function starts with fn keyword followed by arguments, function body comes after -> symbol and finish with end keyword. In our code the anonymous function is wrapped in bracket and then we added .() to invoke the function. The search method return the response in a tuple {:ok, list} format where second element is the list with all the data. So we passed that to anonymous function in x variable and we are returning a map from the function. Within the map we are defining first key as “since_id” to store the id of the last tweet and to get that from the response body we’re using get_in function from Kernel module. get_in function takes a collection from which we want to extract value as first argument and a list of the key structure as second argument to get the value of a key. In our response the “max_id” key is under the “search_metadata” key, so we pass a list as ["search_metadata", "max_id"] to get that value. And the second key is “statuses” where we’ll store the total number of tweets returned to us. The statuses key in twitter api response is a map which is enumerable or iterable. Elixir has a Enum module with lots of function to work with collections. Here we are using count function of Enum module which returns the number of elements of a collection. We used get_in function again to get the collection instatuses key from api response and pass it to count function. Now let’s run the function now

iex(2)> LanguageTrends.Twitter.search_with_hashtag("elixirlang", 0, 100)
%{"since_id" => 863782766407806976, "statuses" => 100}

We can see it returned a map with two keys we defined earlier and their respective values. We’ll use the since_id in next api call to get all the tweets posted after the last tweet from this api call so that we don’t get same tweets again.

That’s it for today. In next post we’ll fetch all the languages hashtag from database and call search api to get the tweets count for each of the hashtag.

You can find the source code here.

Happy brewing some elixir.