Does ChatGPT Actually Get Lazier in December?

I put the “winter break” theory to the test.

I saw this viral tweet today showing that GPT-4, when given a date of December in its system prompt, outputs significantly lower content lengths (about 7.5% less) than when given a date in May. This gave rise to a hilarious idea that ChatGPT gets a little lazy around the holidays and, like most humans, likes to take a little winter break.

There has been much dissent around this idea (many people have failed to reproduce this), so I have decided to get to the bottom of it.

The testing setup

I created a quick code test using the API to see how GPT-4 output changes with certain date inputs. Here is the control version of my code without a date injection:

def completion(date):
    response = client.chat.completions.create(
        model="gpt-4-1106-preview",
        temperature=0.8,
        messages=[{"role": "system", "content": f"You are a helpful assistant."},
                  {"role": "user", "content":
                      f"""
                        Explain the concept of quantum entanglement to a 
                        non-specialist audience.
                      """
                   }],
    )
    return response.choices[0].message.content