Flipping plots and making dates

This is part 5 of an N part series detailing how I make my animations.

Prev Next

In the previous post I mentioned there were two major problems with the animation produced.

The first one is quite subtle. Notice how the first CO2 concentration value was 316.19 and the height of the output was 500 which means the plot should’ve been well above half way?

Something fishy is going on and not only that I thought the CO2 concentration was rising? If you squint you can make out that it is dropping! That’s a relief I guess.

Well the reason is because the origin is located in the top left corner of the output and that positive Y values go down. Basically the output is flipped upside down. The reason this is so is a legacy from how old computers displayed their output on old cathode ray tube screens. In those a beam of electrons was scanned from left to right across the screen starting from the top left corner and ending at the bottom right. It seemed logical at the time that if you wanted to plot a point on the screen then you just counted how many rows had been scanned down from the top and then plot your point accordingly. This turned into measuring the Y value from the top of the screen.

Unfortunately it’s too late to change computers to be sensible so we have to deal with it everytime we plot points. Simply by subtracting our value from the height of the output. Processing has provided us with a special variable called height to do this. So replace the ellipse command with:

ellipse(frameCount, height - co2, 1, 1);

Running the sketch should look something like this:

Much better.

Now the second problem is to do with the data itself. Recall I mentioned that due to powercuts and hurricanes there are gaps in the data? In fact you may be able to spot discontinuities in the image above.

This is going to be quite a challenge to overcome. One way would be to read the date along with the CO2 value, work out how many days since the last CO2 value and shift the X coordinate accordingly.

This may well work and the gaps will appear correctly in the output but there will be jumps in time creating a chronological discontinuity.

After much experimentation I have settled on an approach that reduces the amount of logic required and hence the amount of code. Basically we have a starting date and a time interval. For the Keeling curve our start date is 29/03/1958 and our time interval is 7 days. For every frame we simply multiply the frameCount by the time interval, add that to the starting date and see if data exists for that point in time. If so plot a point otherwise go to the next frame.

The key part to that algorithm is finding out if the data exists for that point in time. We could remember the next point in the array and check if the date at that point matches the expected one. It’ll work, but there’s a much better way.

Instead of using an array we use something called a dictionary. I personally think this is a stupid name as it instantly invokes images of looking up definitions of words. What’s it got to do with dates and CO2 concentrations?

In computing a dictionary is a storage mechanism that has keys and values. In traditional dictionaries a key would be the word and the value would be the definition. For our purposes the key will be the date and the value would be the CO2 concentration.

We simply check to see if the date exists in the dictionary, plot the point if it does or skip this frame if it doesn’t. It’s slightly more complicated to populate the dictionary but now we can filter out things like the informational text and not have to remember where the data starts in the array.

So change the definition of _data near the top of the code to:

FloatDict _data = new FloatDict();

A FloatDict is a dictionary who’s values are floating point real numbers. Replace the contents of loadData() with:

  String[] lines = loadStrings("weekly_in_situ_co2_mlo.csv");
  
  for (String line : lines) 
  {
    if( line.startsWith("\"") ) continue;
    
    String[] values = split(line, ',');
    String date = values[0];
    float co2 = parseFloat(values[1]);
    _data.set(date, co2);
  }

The code for (String line : lines) says for each line in the array lines do this.

The code if( line.startsWith("\"") ) continue; skips any line that begins with the character ". Unfortunately the very character we are checking to skip is the same character that is used to denote a string in the code! So to get round this we use a special character \ to “escape” the " from the string. This character is called the “escape character”. This’ll probably make more sense in the following example:

"Fred said \"Oh dear, I must escape from here.\" but it was all for nought" 

The code _data.set(date, co2) adds a the CO2 value to the dictionary with the date as the key. This now means we can check to see if the date exists by executing _data.hasKey(<some date>) and getting the value with _data.get(<some date>).

Now we need to be able to manipulate dates. Unfortunately Processing has very poor date handling functionality, this isn’t a problem though, we simply import a date handling library, so at the top of the code add:

import java.time.*;

That’s right, you’ve been coding in Java all this time!

In the same area of the code where FloatDict _data is defined add:

LocalDate _startDate = LocalDate.of(1958, 3, 29);

Now for each frame all we need to do is work out the number of days from the start date, add them to the start date, check to see if data exists for that date and plot it!

To work out the number of days since the start:

int daysFromStart = (frameCount - 1) * 7;

For frame #1 the days from the start will be 0. For frame #2 the days from the start will be 7 etc.

To add that number of days to the start date:

LocalDate frameDate = _startDate.plusDays(daysFromStart);

To check if any data exists for the frameDate :

if(_data.hasKey( frameDate.toString() ))

toString() outputs the date in the format yyyy-mm-dd which is luckily the same format as the data!

To get the data:

float co2 = _data.get(frameDate.toString());

In total:

import java.time.*;
FloatDict _data = new FloatDict();
LocalDate _startDate = LocalDate.of(1958, 3, 29);
void setup()
{
  loadData();
  size(500,500);
}
void draw()
{
  int daysFromStart = (frameCount - 1) * 7;
  LocalDate frameDate = _startDate.plusDays(daysFromStart);
  
  if(_data.hasKey( frameDate.toString() ))
  {
    float co2 = _data.get(frameDate.toString());
    ellipse(frameCount, height - co2, 1, 1);
  }
}
void loadData()
{
  String[] lines = loadStrings("weekly_in_situ_co2_mlo.csv");
  
  for (String line : lines) 
  {
    if( line.startsWith("\"") ) continue;
    
    String[] values = split(line, ',');
    String date = values[0];
    float co2 = parseFloat(values[1]);
    _data.set(date, co2);
  }
}

Which produces:

Hurray! Our gaps are there and the animation doesn’t have chronological discontinuities!

Now let’s make this more compelling…