datetime mindfuck

It is rare as a white, british, middle class, straight, oxbridge educated man to find an adversity to succeed in spite of. Dates, times and timezones are my adversity, I suppose I should be grateful for the challenge, but I’m not, I’m sad and confused.

You see when you build an application which makes heavy use of timezones on London (as TutorCruncher does), you spend half the year (October to March) in glorious conviction everything is ok, then March hits and odd things start going wrong. No sooner have you smoothed out those problems (and finished apologising to the effected customers) than October arrives and somehow something else goes wrong.

From October to March the London timezones is UTC+0 (or GMT as it's affectionately known), but from March to October we're on British Summer Time UTC+1 (or maybe -1, at this point I've lost all clue). March-October lulls one into being lazy and assuming there's no timezone offset, October-March breaks that but not by enough to always be immediately obvious. And it's always the bugs which don't jump out at you that end up screwing you.

If I was American, or Japanese or even French these things wouldn’t happen to me. As a developer I’d have grown up knowing you can never assume the timezone offset was zero. I’d be a better person.

Okay enough fluff, lets talk about some specific non sequiturs with datetimes.

mindfucks with python

I’m a massive fan of python, I realised why recently when I read the transcript of a speech by Guide Von Rossum: he explains he doesn’t think programming languages are a way of communicating with computers (“computers can take care of themselves”), but rather a way for progammers to communicate with each other (and their future selves). To me that explains why python is so elegant, simple and communicative.

However unfortunately I would say dates & times are python’s ugly side: an area where you might expect it to excel but where it comes up significantly short.

Some specific examples:

the tzinfo argument and dst timezones

The canonical way to apply a timezone to datetimes is with the tzinfo argument, but don't you dare use it with timezones which have day light saving time.

Completely wrong result!

Using a timezone with a dst offset fucked everything right up.

Joking aside this is real problem with python. Either dst timezones should work when used with the tzinfo key word argument, or they should throw a dirty great exception. The current status quo of silent failure really isn't good.

Four ways to get a unix timestamp

The zen of python is that there should be one and only one clear way of doing any given task. Two minutes on StackOverflow shows that there are at least 4 commonly used approaches to converting a datetime to a unix timestamp (number of seconds since 1970–01–01). What’s worse they’re subtly different, and easy to make a mistake with:

unix time stamp confusion (The “TZ” environment variable allows you to force the system timezone)

As you might imagine the situation doesn’t get any better if dt has a timezone.

(Hint the first and last answers are correct, the middle two are off by an amount equal to the utc offset of the current timezone, eg. they’re be right if we happened to be in UTC.)

Four ways to change timezones

Changing timezones for datetimes is another minefield, there are four ways of changing the timezone on a datetime.

Their names replace, localize, astimezone, normalize have absolutely no meaning for me. I end up either experimenting with which to use or resorting to reading the pytz docs yet again.

Confusion switching timezone

At least some of these actually raise exceptions when things go wrong. Other than that I’m even more confused now than when I started writing this blog, again we see that replace(tzinfo=...) breaks completely and silently.

database timezone support & django

Using django you’d be forgiven for thinking that with timezones “enabled” (settings.SE_TZ = True) datetimes saved to the database would hold timezone info. In fact they don’t.

All that’s saved in the database is a timestamp. Save a date in America/Toronto to the database and then get it back from the database and it'll be in django's default timezone (settings.TIME_ZONE). If you want to save information about what timezone the datetime was originally it; that's your responsibility.

So many choices!

Theunix timestamp example above hints at another big problem: to many standard libraries related to dates and times:

• datetime — nowadays the main option for virtually all date and time calcs.
• time — often required for time.sleep() and time.time() but much less sophisticated than datetime.
• calendar — I don’t really use it and I’m not sure why it still exists. Presumably it has some functions people use but I don’t know what they are and why they haven’t been moved into datetime or an external library.

All the problems above (and a lack of functionality for advanced timezones and datetime parsing) has lead to the rise of a host of external libraries:

• pytz — fairly ubiquitous for proper timezone support. Sensibly pytz doesn’t try to do too much more.
• dateutils — very useful for parsing datetimes and datetime arithmetic, also does complex stuff with timezones and thereby overlaps with pytz.
• arrow — “better dates and times for Python” clever datetime parsing and humanisation methods.
• pendulum — “PYTHON DATETIMES MADE EASY” apparently a successor to arrow, not clear why it’s better.
• maya — tag line: “datetimes for humans”; tag line could have alternatively been “because writing my own was more fun than contributing to datetime, time, calendar, datetutils, arrow, or pendulum”
• django timezone — I guess when django started much of the datetime support was missing from the standard library so django decided to implement a lot itself. Still today it has it’s own way of doing human readable datetimes, including implementing much of php’s datetime rendering logic itself!!!
• momentjs — of course not part of python but if you’re writing web applications you’ll eventually end up having to work with datetimes in javascript and will likely resort to moment, thereby adding yet another enormous library with a complex and unique API to understand, use and likely misuse.

The solution

So what’s the solution to all this? Perhaps yet another library? Please no.

First: agree which external external libraries you need. I would say pytz and dateutils.

Then agree on sane and consistent ways of doing common tasks:

(infuriatingly medium doesn’t let you paste a single file from a gist so you get the tests too, sorry)

If you need extra human readable dates eg. “an hour ago” and your not using django I would recommend leaving all that stuff to momentjs. If you’re not building a web application and/or need such stuff in python I guess you’ll need arrow, pendulum or maya, but I’ve no idea which is preferable.

And Lastly

Need any more evidence that datetimes in python are not all well:

Yup, 30.4% coverage for python stdlib datetime.py module.