TLDR; Click here to cut the crap!
The Background
Here in The Plant, we’re required to write our work-log every day. Basically, describing what you’ve done today and creating an issue on Github, and then your team lead gives it a score. I’m too lazy for that and I didn’t record anything for the past July… So now I have to create over 20 issues, it’s tedious and may take me half an hour.
“What if it can be automated?”
I’m sure 👉you have the same moment if you’re a developer. Developers’ time is so precious that we can’t afford for 30 minutes repeating ourselves, but would spend hours automate things! Even it’s a one-time job, or it shouldn’t be automated at all!
I didn’t write my work-log not just because I’m super lazy. I recently shifted my role from a tech lead(a team of 3–5 people) to a product manager(QOR: the SDK for E-Commerce & CMS, written in Go). I was busy reading, learning, arguing, planning… And suffering! I felt unproductive, and most of the time, I didn’t know what to do. So these were the obstacles that I didn’t keep my work-log.
So I really need to get out of it. I’m too far away from my comfort zone, I guess I can retreat back for a while?🙈🙉🙊
Interacting with the Github API
Yes, I’ve got enough good excuses to get my hands dirty. I’d like to write some code to help me quickly create 20 Github issues, that’s it! I’m sure I could do it with Ruby pretty quickly. But that’s boring. I’m always interested in Elixir, but I’ve never had a chance to use it in production. Now I have 20 issues to create, maybe I try out its fancy concurrency feature. To avoid hitting the request threshold, maybe I should set up a pool. To resend a failed request, maybe there should be a supervisor, I can play with the famous OTP framework. These all seems very interesting.
Interacting with the Github API is supposed to be easy. Usually, using my email and password to exchange an access token, so that I can use the API freely. But I’m using 2-factor authentication, that could be a problem. Luckily, Github has sophisticated docs. I managed to use the basic authentication with an X-GitHub-OTP header to create an access token. With this access token, my program can create an issue on Github on behalf of me.
I use curl to make a POST request to get the access token, now I need to make HTTP request in Elixir. httpoison is a good option. A pitfall here. The Github API uses JSON format. Naturally, I would use a Map for the parameters. But httpoison doesn’t automatically convert a Map into a JSON object. So I need to use poison for help.
headers = %{“Authorization” => “token your_access_token”}
body = %{“title”: “Hello world!”} |> Poison.encode!
HTTPoison.post!("https://...", body, headers)The format of an issue title is:
[score]/[hour]/[date]/[content]Now I need some dates, from July 1 to 31, except the weekends. It’s intuitive in Ruby.
d1 = Date.new 2016, 7, 1
d2 = Date.new 2016, 7, 31
(d1 .. d2).each { |d| puts d unless d.sunday? or d.saturday? }But how do I do that in Elixir? Elixir just added Date, DateTime, NaiveDateTime in version 1.3, which is great, but it doesn’t support ranges.
Well, I don’t need a DateRange to do the iteration. There are 31 days in July, so I can do:
def july do
1..31
|> Enum.map(fn i ->
{:ok, d} = Date.new 2016, 7, i
d
end)
|> Enum.filter(fn d -> !weekend?(d) end)
end
def weekend? date do
weekday = date |> Date.to_erl |> :calendar.day_of_the_week
weekday > 5
endSee that :calendar, it’s an Erlang library. You can use Erlang in Elixir seamlessly!(That says, Elixir Date doesn’t have a convenient way to tell whether a date is a weekday or not…)
Okay! Now I can create my Github issues.
Implementing ExclusiveDateRange and DateRange
Shall I make concurrent requests now? No, it doesn’t interest me now, as Elixir doesn’t support date ranges, which bothers me! Using date ranges is common, there must be handy libs out there. But I don’t care, I want to implement my own, even a naive one.
Elixir does support Range, but just for integers, e.g, (1..10).
A Range is represented internally as a struct.
A Range implements the Enumerable protocol, which means all of the functions in the Enum module is available
Great, I have the basic idea about how to implement the date range:
* It should be a struct(../2 feels like magic to me)
* It should implement the Enumerable protocol
The struct is fairly simple:
defmodule DateRange do
defstruct first: nil, last: nil
end
d1 = ~D[2016-07-01]
d2 = ~D[2016-07-31]
%DateRange{first: d1, last: d2}# => %DateRange{first: ~D[2016-07-01], last: ~D[2016-07-31]}
It simply holds the first and last element of a range.
Now I want to be able to do this:
# iterate through a date range, and print the dates
date_range |> Enum.each(fn date -> date |> inspect |> IO.puts end)That is to implement the Enumerable protocol for the DateRange module. What I need to do it to implement the following 3 functions:
count(enumerable) # Retrieves the enumerable’s size
member?(enumerable, element) # Checks if an element exists within the enumerable
reduce(enumerable, acc, fun) # Reduces the enumerable into an element
count/1 and member?/2 is straightforward. For reduce/3, you can think about a date range being transformed to a list of dates. Confusing? Keep reading, you’ll get a better understanding later.
Now I have to think about the next date(how to iterate through a date range). I need to be very careful about which month has 30 days and which month has 31 days. And the special February.
But there’s an easier way. Date.new/3 returns a tuple of {:ok, date} or {:error, :invalid_date}. So I can simply increase the day field, and if I get an error, then increase the month field, if I get another error, then increase the year field.
def next %Date{year: y, month: m, day: d} do
case Date.new(y, m, d + 1) do
{:ok, nd} -> nd
{:error, _} ->
case Date.new(y, m + 1, 1) do
{:ok, nd} -> nd
{:error, _} ->
{:ok, nd} = Date.new(y + 1, 1, 1)
nd
end
end
endHere’s the complete implementation. Read the gist if need syntax highlighted.
defmodule ExclusiveDateRange do
defstruct first: nil, last: nil
def new(first, last) do
%ExclusiveDateRange{first: first, last: last}
end
defimpl Enumerable, for: ExclusiveDateRange do
def reduce _dr, {:halt, acc}, _fun do
{:halted, acc}
end
def reduce(dr, {:suspend, acc}, fun) do
{:suspended, acc, &reduce(dr, &1, fun)}
end
def reduce(%ExclusiveDateRange{first: d, last: d}, {:cont, acc}, _fun) do # when first == last, ends the recursion
{:done, acc}
end
def reduce(%ExclusiveDateRange{first: d1, last: d2}, {:cont, acc}, fun) do
reduce(%ExclusiveDateRange{first: next(d1), last: d2}, fun.(d1, acc), fun)
end
defp next %Date{year: y, month: m, day: d} do
case Date.new(y, m, d + 1) do
{:ok, nd} -> nd
{:error, :invalid_date} ->
case Date.new(y, m + 1, 1) do
{:ok, nd} -> nd
{:error, :invalid_date} ->
{:ok, nd} = Date.new(y + 1, 1, 1)
nd
end
end
end
def member?(%ExclusiveDateRange{first: d1, last: d2}, value) do
days1 = date_to_gregorian_days d1
days2 = date_to_gregorian_days(d2) - 1
days3 = date_to_gregorian_days value
{:ok, Enum.member?(days1..days2, days3)}
end
def count(%ExclusiveDateRange{first: d1, last: d2}) do
days1 = date_to_gregorian_days d1
days2 = date_to_gregorian_days d2
{:ok, days2 - days1}
end
defp date_to_gregorian_days date do
date |> Date.to_erl |> :calendar.date_to_gregorian_days
end
end
endThe problem with the above implementation is:
* it’s exclusive
* it doesn’t support reversed ranges
Take a look at the next/1 function again, now think about if it has to return the previous date, it’d be much more complex.
What if dates can be represented as integers?
Then a date range would actually be an integer range, then we can reuse Elixir’s Range. Isn’t it great?
:calendar.date_to_gregorian_days/1 converts a Date into an integer, and if you add 1 to the integer, it increases 1 day. You can also decrease it. Now, we don’t have to worry about manipulating year, month, and day.
Read the gist if need syntax highlighted.
defmodule DateRange do
defstruct first: nil, last: nil, first_gregorian_days: nil, last_greogorian_days: nil
defmodule Operator do
def first <~> last do
DateRange.new first, last
end
end
defmodule Helpers do
def date_to_gregorian_days date do
date |> Date.to_erl |> :calendar.date_to_gregorian_days
end
def gregorian_days_to_date int do
{:ok, date} = int |> :calendar.gregorian_days_to_date |> Date.from_erl
date
end
end
def new(first, last) do
d1 = Helpers.date_to_gregorian_days first
d2 = Helpers.date_to_gregorian_days last
%DateRange{first: first, last: last, first_gregorian_days: d1, last_greogorian_days: d2}
end
defimpl Enumerable, for: DateRange do
def reduce(%DateRange{first_gregorian_days: d1, last_greogorian_days: d2}, acc, fun) do
reduce(d1, d2, acc, fun, d1 <= d2)
end
defp reduce(_x, _y, {:halt, acc}, _fun, _up) do
{:halted, acc}
end
defp reduce(x, y, {:suspend, acc}, fun, up) do
{:suspended, acc, &reduce(x, y, &1, fun, up)}
end # normal ranges
defp reduce(x, y, {:cont, acc}, fun, true) when x <= y do
d = Helpers.gregorian_days_to_date x # NOTE it yeilds a date, not an integer
reduce(x + 1, y, fun.(d, acc), fun, true)
end
# reversed ranges
defp reduce(x, y, {:cont, acc}, fun, false) when x >= y do
d = Helpers.gregorian_days_to_date x # NOTE it yeilds a date, not an integer
reduce(x - 1, y, fun.(d, acc), fun, false)
end
defp reduce(_, _, {:cont, acc}, _fun, _up) do
{:done, acc}
end
def member?(%DateRange{first_gregorian_days: d1, last_greogorian_days: d2}, value) do
val = Helpers.date_to_gregorian_days value
{:ok, Enum.member?(d1..d2, val)}
end
def count(%DateRange{first_gregorian_days: d1, last_greogorian_days: d2}) do
if d1 <= d2 do
{:ok, d2 - d1 + 1}
else
{:ok, d1 - d2 + 1}
end
end
end
end
The reduce/3 and reduce/5 are completely borrowed from Range. In reduce/5, an integer is converted to a Date. Because when enumerating a DateRange, we naturally expect a Date, not a integer.
See the DateRange.Operator, I also implemented a <~> operator, which is similar to the ../2 macro, you can have (1..10). If you import the DateRange.Operator module, you can write ~D[2016–07–01] <~> ~D[2016–07–31].
Just like you can have a reversed integer range, e.g (10..1), you can have a reversed DateRange, like ~D[2016–07–31] <~> ~D[2016–07–01].
Note: the code in this post is not well-tested, use at your own risk.
A Date Comparison Bug
While playing with the Date, I found a bug.
iex(1)> d1 = ~D[2016-08-01]
~D[2016-08-01]
iex(2)> d2 = ~D[2016-08-02]
~D[2016-08-02]
iex(3)> d3 = ~D[2016-07-02]
~D[2016-07-02]
iex(4)> d1 < d2
true
iex(5)> d3 < d1
falseJose prefers to say Ecto.Dates are not comparable, though. So I guess it’s not a bug, and I didn’t create an issue for the Elixir team.
Summary
There are a few things I learned:
* Maybe the best way to relax is to do something I’m good at or interested in.
* Protocols provide a good way to achieve extensibility.
* Read the docs.
* Don’t be too lazy.😜
Please hit the ♥ below if you found this post useful, so that others can read it.
