10 things you need to know about Date and Time in Python with datetime, pytz, dateutil & timedelta

Dates and Time probably sounds like an easy concept, until you have to deal with users and data from around the world.

Then you realize, it’s actually complicated!

TL;DR: the cheatsheet

1. Parse date strings

import datetime
import pytz
import dateutil.parser
format = '%Y-%m-%dT%H:%M:%S%z'
datestring = '2016-09-20T16:43:45-07:00'
d = dateutil.parser.parse(datestring) # python 2.7
d = datetime.datetime.strptime(datestring, format) #3.2+

2. ISO-8601 date string to UTC datetime object

import datetime
import pytz
import dateutil.parser
format = '%Y-%m-%dT%H:%M:%S%z'
datestring = '2016-09-20T16:43:45-07:00'
d = dateutil.parser.parse(datestring) # python 2.7
d = d.replace(tzinfo=utc) - d.utcoffset()
>>> datetime.datetime(2016, 9, 20, 23, 43, 45, tzinfo=<UTC>)

3. UTC Timestamp (now)

import datetime
import pytz
# define epoch, the beginning of times in the UTC timestamp world
epoch = datetime.datetime(1970,1,1,0,0,0)
now = datetime.datetime.utcnow()
timestamp = (now - epoch).total_seconds()
>>> 1505329554.617216
# subtracting datetime objects result in a datetime.timedelta object which can be expressed in seconds.
# this works because both datetime are implicitly in UTC time

4. UTC Timestamp from naive date string

import datetime
import pytz
# naive datetime 
d = datetime.datetime.strptime('01/12/2011', '%d/%m/%Y')
>>> datetime.datetime(2011, 12, 1, 0, 0)
# add proper timezone for the date
pst = pytz.timezone('America/Los_Angeles')
d = pst.localize(d)
>>> datetime.datetime(2011, 12, 1, 0, 0, tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
# convert to UTC timezone 
utc = pytz.UTC
d = d.astimezone(utc)
>>> datetime.datetime(2011, 12, 1, 8, 0, tzinfo=<UTC>)
# epoch is the beginning of time in the UTC timestamp world
epoch = datetime.datetime(1970,1,1,0,0,0,tzinfo=pytz.UTC)
>>> datetime.datetime(1970, 1, 1, 0, 0, tzinfo=<UTC>)
# get the total second difference
ts = (d - epoch).total_seconds()
>>> 1322726400.0

5. Add timezone to a naive datetime

import datetime
import pytz
# naive datetime
d = datetime.datetime.strptime('01/12/2011 16:43:45', '%d/%m/%Y %H:%M:%S')
>>> datetime.datetime(2011, 12, 1, 16, 43, 45)
# add proper timezone
pst = pytz.timezone('America/Los_Angeles')
d = pst.localize(d)
>>> datetime.datetime(2011, 12, 1, 16, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)

DON’T use datetime.replace to set a timezone:

import datetime
import pytz
### DON'T DO THIS, THIS CODE IS WRONG!!!
d = datetime.datetime.utcfromtimestamp(1505325217)
>>> datetime.datetime(2017, 9, 13, 17, 53, 37)
pst = pytz.timezone('America/Los_Angeles')
d = d.replace(tzinfo=pst)
>>> datetime.datetime(2017, 9, 13, 17, 53, 37, tzinfo=<DstTzInfo 'America/Los_Angeles' LMT-1 day, 16:07:00 STD>)

Two things are very wrong in the code above:

  • datetime.replace replaces the tzinfo: it does not convert the time at all. (utcfromtimestamp gives a datetime in UTC, so changing tzinfo changes the time represented)
  • pytz doesn’t work well with tzinfo. Replacing tzinfo with anything but UTC has some unintended consequences. In the example above, note how the PST timezone setting becomes actually LMT time (Local Mean Time) which is 16:07:00 offset from UTC, NOT 16:00:00.

You end up with a 16h07m offset: bad!

6. datetime to ISO 8601 string

import datetime
import pytz
# naive datetime
d = datetime.datetime.strptime('01/12/2011 16:43:45', '%d/%m/%Y %H:%M:%S')
>>> datetime.datetime(2011, 12, 1, 16, 43, 45)
# add proper timezone
pst = pytz.timezone('America/Los_Angeles')
d = pst.localize(d)
>>> datetime.datetime(2011, 12, 1, 16, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
d.isoformat()
>>> "2011-12-01T16:43:45-08:00"

7. datetime from timestamp

import datetime
import pytz
d = datetime.datetime.utcfromtimestamp(1505325217)
>>> datetime.datetime(2017, 9, 13, 17, 53, 37)
# add timezone info
d = pytz.UTC.localize(d)
>>> datetime.datetime(2017, 9, 13, 17, 53, 37, tzinfo=<UTC>)

8. convert from UTC to another timezone

import datetime
import pytz
d = datetime.datetime.utcfromtimestamp(1505325217)
>>> datetime.datetime(2017, 9, 13, 17, 53, 37)
# add timezone info
d = pytz.UTC.localize(d)
>>> datetime.datetime(2017, 9, 13, 17, 53, 37, tzinfo=<UTC>)
pst = pytz.timezone('America/Los_Angeles')
d.astimezone(pst)
>>> datetime.datetime(2017, 9, 13, 10, 53, 37, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)

9. Add and Subtract time with timedelta

Daylight saving was on Nov 6th in 2016:

import datetime
import pytz
d = datetime.datetime(2016, 11, 5, 16, 43, 45) # naive datetime
utc = pytz.UTC
pst = pytz.timezone('America/Los_Angeles')
d = utc.localize(d) # UTC timezone aware
>>> datetime.datetime(2016, 11, 5, 16, 43, 45, tzinfo=<UTC>)
# add 1 day to UTC date
d = d + datetime.timedelta(days=1)
>>> datetime.datetime(2016, 11, 6, 16, 43, 45, tzinfo=<UTC>)
# now convert to local timezone
d = d.astimezone(pst)
>>> datetime.datetime(2016, 11, 6, 8, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
# daylight saving was respected

DON’T use timedelta with anything but UTC time:

import datetime
import pytz
### DON'T USE THIS CODE, THIS CODE IS WRONG !!!
d = datetime.datetime(2016, 11, 5, 16, 43, 45) # naive datetime
utc = pytz.UTC
pst = pytz.timezone('America/Los_Angeles')
d = utc.localize(d) # UTC timezone aware
>>> datetime.datetime(2016, 11, 5, 16, 43, 45, tzinfo=<UTC>)
# convert d to 'America/Los_Angeles' timezone
d = d.astimezone(pst)
>>> datetime.datetime(2016, 11, 5, 9, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
# add 1 day to PDT date: DON'T DO THAT
d = d + datetime.timedelta(days=1)
>>> datetime.datetime(2016, 11, 6, 9, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
# daylight saving was not respected, still PDT time, not PST as it should be

So, now the long version:

1. Parsing date & time strings

The first thing is: there are so many date and time formats, whether you are in North America or Europe, whether you speak in 12H or 24H time or even military time:

  • “Wednesday September 21st, 06:30PM”
  • “09/21/2016 06:30PM”
  • “2016/09/21 18:30”
  • “21/09/2016 18:30”

Parsing date & time can be a bit of a pain, but that’s not the most complicated thing:

import datetime
format = '%Y-%m-%d %H:%M:%S'
datestring = '2016-09-20 16:43:45'
d = datetime.datetime.strptime(datestring, format)
# Note ISO string format is special. More about that later

What we just obtained is called a naive datetime.

Why naive, you ask? Because it is subjective.


2. Time is relative

If it is 16h43 in London right now, it will only be 16h43 (4:43PM) in San Francisco in another 8 hours from now. The same date string can represent different points in time, at different locations.

With a naive datetime, there is no clear way to know at what actual point-in-time an event occurred, and therefore no clear way to order events.

Comes UTC timestamps


3. UTC Timestamps

Universal Time Coordinated (literal translation of the French TUC (Temps Universal Coordiné)

A UTC timestamp is a number of seconds since epoch, which is January 1st 1970 at 00:00 GMT (Greenwich Mean Time)

epoch is defined at a specific place in the world (the GMT timezone) which makes it possible to represent an actual point-in-time.

Note: you may hear about GMT time, UTC time or even Zulu time. These all represent time at the Greenwich Meridian, that happens to go North to South and crosses around Greenwich, in the U.K.

UTC timestamps are useful to order events (log data) and calculate time difference, in terms of elapsed time.

A naive approach to getting a timestamp in python might go like:

# define epoch, the beginning of times in the UTC timestamp world
epoch = datetime.datetime(1970,1,1,0,0,0)
now = datetime.datetime.utcnow()
timestamp = (now - epoch).total_seconds()
# subtracting datetime objects result in a datetime.timedelta object which can be expressed in seconds.

This works because the utcnow method returns a naive datetime object in UTC time by default, but don’t do this with a timezone aware dates not in UTC.

We’ll see in a little bit how converting a given date to UTC timestamp, takes a little more effort.

While UTC timestamps are a great way to standardize representation of a point in time, the same UTC timestamp converts to different date strings in different places of the world. It is, after all, defined at GMT.

Using a timestamp, we’ve lost the notion of location, and therefore local time.

So, is it one or the other?


4. Time Offsets, ISO-8601 and Local Time

If we want to know about time-of-day of an event sent by a user, or simply need to represent event times in local time as is the case in most user-facing application, you will need to convert from server time (which should really always be UTC) to local time.

A time offset is the timezone offset that needs to be added to UTC time to represent local time.

ISO-8601 is a standard to represent date strings, including time offsets.

It’s really able to represent date string in many different format, but the most commonly used format in computing is:

2017–09–13T20:58:41+07:00

YYYY-MM-DDTHH:mm:ss+/-HH:mm

In python parsing format that becomes

'%Y-%m-%dT%H:%M:%S%z'

to parse an ISO-8601 string in python >3.2, you can use the same function as shown in 1. but if you’re using python 2.7, you’ll need dateutil.parser:

import datetime
import pytz
import dateutil.parser
format = '%Y-%m-%dT%H:%M:%S%z'
datestring = '2016-09-20T16:43:45-07:00'
d = dateutil.parser.parse(datestring) # python 2.7
d = datetime.datetime.strptime(datestring, format) #3.2+

Now, a very important thing to understand:

An ISO-8601 date string provides:

  • local time (the YYYY-MM-DDTHH:mm:ss part)
  • a mean to convert to UTC (the time offset, i.e the +/-HH:mm part),

However the time offset is only valid for that local date/time.

Yes, time offsets are not set in stone: they actually change, because of daylight saving.

2017–07–13T20:58:41+07:00 or 2017–12–13T20:58:41+08:00 is the time for the same place in the Pacific time zone, but one is in the summer (when Daylight Saving Time DST is observed) and the other is in the winter when it is not.

What that means is that you cannot use just the time offset to assume the location, or compute time deltas. 
That will not work well, but you probably won’t notice until there is a DST change, at which point you will probably pull your hair out trying to figure what went wrong.

ISO-8601 string can safely be used to translate local time between time zones, but should not be used as-is to do time manipulations.

5. Timezones

Time offset is not enough. We need timezone information.

What is a timezone?

It’s a little confusing because I’ve talked about UTC time, but UTC is not a timezone. You’ve probably heard about EST, CST, MST or PST if you live in North America, and GMT and CET if you’re in western Europe, and been told these are time zones, but these are not actually timezones, they are naming representing the time offset.

A timezone is usually defined by a continent and a location, like America/Los_Angeles or Europe/Paris or Africa/Tunis

Note there is no notion of Daylight Saving in the name

  • PST: Pacific Standard Time (GMT-08:00), and PDT: Pacific Daylight Time (GMT-07:00), are both time offset applied in the America/Los_Angeles timezone depending on the time of the year.
  • CET: Central European Time (GMT+01:00) and CEST: Central European Summer Time (GMT+02:00) are both time offsets applied in the Europe/Paris timezone
  • CET: Central European Time (GMT+01:00) is also applied in the Africa/Tunis timezone, but daylight saving is not observed, so CET time is used year round.

to deal with timezone, pytz is your friend… most of the time.

We haven’t mentioned it yet, but when we parsed the ISO-8601 earlier, you may have noticed that the datetime object comes out like:

datetime.datetime(2016, 9, 20, 16, 43, 45, tzinfo=tzoffset(None, -25200))

There is a time offset, but not timezone information.

All we can really do with this, is convert back to UTC

import datetime
import pytz
import dateutil.parser
format = '%Y-%m-%dT%H:%M:%S%z'
datestring = '2016-09-20T16:43:45-07:00'
d = dateutil.parser.parse(datestring)
d = d.replace(tzinfo=pytz.UTC) - d.utcoffset()
>>> datetime.datetime(2016, 9, 20, 23, 43, 45, tzinfo=<UTC>)
# now a timezone aware UTC datetime.

Wait, we lost the timezone again.

No we didn’t. We never had the timezone to begin with. We had an offset. As seen above, a timezone may have multiple offsets, so there is no simple way to get a timezone name from the offset.

Timezone name is something you can easily get on the client (browser, in Javascript) and that you should send to your server along with the ISO-8601 string if you need it on the server.

If you know the timezone you want to convert to, then you can do:

import datetime
import pytz
import dateutil.parser
format = '%Y-%m-%dT%H:%M:%S%z'
datestring = '2016-09-20T16:43:45-07:00'
d = dateutil.parser.parse(datestring)
# UTC datetime
d = d.replace(tzinfo=pytz.UTC) - d.utcoffset()
pst = pytz.timezone('America/Los_Angeles')
d.astimezone(pst)
>>> datetime.datetime(2016, 9, 20, 16, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)

but don’t get too excited:

You should not do any datetime operations in localized time. Always use UTC.

Look at this example, where we are a day before Daylight Saving changes, and we want to add one day to the datetime object:

import datetime
import pytz
### DON'T USE THIS CODE, THIS CODE IS WRONG !!!
d = datetime.datetime(2016, 11, 5, 16, 43, 45) # naive datetime
utc = pytz.UTC
pst = pytz.timezone('America/Los_Angeles')
d = utc.localize(d) # UTC timezone aware
>>> datetime.datetime(2016, 11, 5, 16, 43, 45, tzinfo=<UTC>)
# convert d to 'America/Los_Angeles' timezone
d = d.astimezone(pst)
>>> datetime.datetime(2016, 11, 5, 9, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
# add 1 day to PDT date: DON'T DO THAT
d = d + datetime.timedelta(days=1)
>>> datetime.datetime(2016, 11, 6, 9, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
# daylight saving was not respected, still PDT time, not PST as it should be

instead, using the UTC datetime for all operations, and only converting to local time in the end:

import datetime
import pytz
d = datetime.datetime(2016, 11, 5, 16, 43, 45) # naive datetime
utc = pytz.UTC
pst = pytz.timezone('America/Los_Angeles')
d = utc.localize(d) # UTC timezone aware
>>> datetime.datetime(2016, 11, 5, 16, 43, 45, tzinfo=<UTC>)
# add 1 day to UTC date
d = d + datetime.timedelta(days=1)
>>> datetime.datetime(2016, 11, 6, 16, 43, 45, tzinfo=<UTC>)
# now convert to local timezone
d = d.astimezone(pst)
>>> datetime.datetime(2016, 11, 6, 8, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
# daylight saving was respected

7. Converting a naive date to a timestamp

Now, sometimes you need to deal with naive dates, which means you need to know the timezone in order to do anything useful with them, like convert to a timestamp.

Converting a date to a UTC timestamp, as mentioned above, take a little more effort to make sure the date is in UTC to begin with:

# naive datetime 
d = datetime.datetime.strptime('01/12/2011', '%d/%m/%Y')
>>> datetime.datetime(2011, 12, 1, 0, 0)
# add proper timezone for this naive date
pst = pytz.timezone('America/Los_Angeles')
d = pst.localize(d)
>>> datetime.datetime(2011, 12, 1, 0, 0, tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
# convert to UTC timezone 
utc = pytz.UTC
d = d.astimezone(utc)
>>> datetime.datetime(2011, 12, 1, 8, 0, tzinfo=<UTC>)
# epoch is the beginning of time in the UTC timestamp world
epoch = datetime.datetime(1970,1,1,0,0,0,tzinfo=pytz.UTC)
>>> datetime.datetime(1970, 1, 1, 0, 0, tzinfo=<UTC>)
# get the total second difference
ts = (d - epoch).total_seconds()
>>> 1322726400.0

Here we go.

I hope this has been useful.

If you only need to remember a few things, this is what you need to remember:

  • Always use timezone aware datetime in UTC on the server, and for any operation.
  • Use ISO-8601 date strings with offset to transmit date information between client and server.
  • Ship the timezone name from the client, don’t try to guess.

Good luck!