datetime mindfuck
It is rare as a white, british, middle class, straight, oxbridge educated man to find an adversity to succeed in spite of. Dates, times and timezones are my adversity, I suppose I should be grateful for the challenge, but I’m not, I’m sad and confused.
You see when you build an application which makes heavy use of timezones on London (as TutorCruncher does), you spend half the year (October to March) in glorious conviction everything is ok, then March hits and odd things start going wrong. No sooner have you smoothed out those problems (and finished apologising to the effected customers) than October arrives and somehow something else goes wrong.
From October to March the London timezones is UTC+0
(or GMT
as it's affectionately known), but from March to October we're on British Summer Time UTC+1
(or maybe -1
, at this point I've lost all clue). March-October lulls one into being lazy and assuming there's no timezone offset, October-March breaks that but not by enough to always be immediately obvious. And it's always the bugs which don't jump out at you that end up screwing you.
If I was American, or Japanese or even French these things wouldn’t happen to me. As a developer I’d have grown up knowing you can never assume the timezone offset was zero. I’d be a better person.
Okay enough fluff, lets talk about some specific non sequiturs with datetimes.
mindfucks with python
I’m a massive fan of python, I realised why recently when I read the transcript of a speech by Guide Von Rossum: he explains he doesn’t think programming languages are a way of communicating with computers (“computers can take care of themselves”), but rather a way for progammers to communicate with each other (and their future selves). To me that explains why python is so elegant, simple and communicative.
However unfortunately I would say dates & times are python’s ugly side: an area where you might expect it to excel but where it comes up significantly short.
Some specific examples:
the tzinfo argument and dst timezones
The canonical way to apply a timezone to datetimes is with the tzinfo
argument, but don't you dare use it with timezones which have day light saving time.
Using a timezone with a dst offset fucked everything right up.
Joking aside this is real problem with python. Either dst
timezones should work when used with the tzinfo
key word argument, or they should throw a dirty great exception. The current status quo of silent failure really isn't good.
Four ways to get a unix timestamp
The zen of python is that there should be one and only one clear way of doing any given task. Two minutes on StackOverflow shows that there are at least 4 commonly used approaches to converting a datetime to a unix timestamp (number of seconds since 1970–01–01). What’s worse they’re subtly different, and easy to make a mistake with:
TZ”
environment variable allows you to force the system timezone)As you might imagine the situation doesn’t get any better if dt
has a timezone.
(Hint the first and last answers are correct, the middle two are off by an amount equal to the utc offset of the current timezone, eg. they’re be right if we happened to be in UTC
.)
Four ways to change timezones
Changing timezones for datetimes is another minefield, there are four ways of changing the timezone on a datetime.
Their names replace
, localize
, astimezone
, normalize
have absolutely no meaning for me. I end up either experimenting with which to use or resorting to reading the pytz
docs yet again.
At least some of these actually raise exceptions when things go wrong. Other than that I’m even more confused now than when I started writing this blog, again we see that replace(tzinfo=...)
breaks completely and silently.
database timezone support & django
Using django you’d be forgiven for thinking that with timezones “enabled” (settings.SE_TZ = True
) datetimes saved to the database would hold timezone info. In fact they don’t.
All that’s saved in the database is a timestamp. Save a date in America/Toronto
to the database and then get it back from the database and it'll be in django's default timezone (settings.TIME_ZONE
). If you want to save information about what timezone the datetime was originally it; that's your responsibility.
So many choices!
Theunix timestamp example above hints at another big problem: to many standard libraries related to dates and times:
- datetime — nowadays the main option for virtually all date and time calcs.
- time — often required for
time.sleep()
andtime.time()
but much less sophisticated thandatetime
. - calendar — I don’t really use it and I’m not sure why it still exists. Presumably it has some functions people use but I don’t know what they are and why they haven’t been moved into
datetime
or an external library.
All the problems above (and a lack of functionality for advanced timezones and datetime parsing) has lead to the rise of a host of external libraries:
- pytz — fairly ubiquitous for proper timezone support. Sensibly pytz doesn’t try to do too much more.
- dateutils — very useful for parsing datetimes and datetime arithmetic, also does complex stuff with timezones and thereby overlaps with pytz.
- arrow — “better dates and times for Python” clever datetime parsing and humanisation methods.
- pendulum — “PYTHON DATETIMES MADE EASY” apparently a successor to arrow, not clear why it’s better.
- maya — tag line: “datetimes for humans”; tag line could have alternatively been “because writing my own was more fun than contributing to datetime, time, calendar, datetutils, arrow, or pendulum”
- django timezone — I guess when django started much of the datetime support was missing from the standard library so django decided to implement a lot itself. Still today it has it’s own way of doing human readable datetimes, including implementing much of php’s datetime rendering logic itself!!!
- momentjs — of course not part of python but if you’re writing web applications you’ll eventually end up having to work with datetimes in javascript and will likely resort to moment, thereby adding yet another enormous library with a complex and unique API to understand, use and likely misuse.
The solution
So what’s the solution to all this? Perhaps yet another library? Please no.
First: agree which external external libraries you need. I would say pytz
and dateutils
.
Then agree on sane and consistent ways of doing common tasks:
If you need extra human readable dates eg. “an hour ago” and your not using django I would recommend leaving all that stuff to momentjs. If you’re not building a web application and/or need such stuff in python I guess you’ll need arrow, pendulum or maya, but I’ve no idea which is preferable.
And Lastly
Need any more evidence that datetimes in python are not all well: