Looking back at all the tools and techniques that I've developed over the years as a writer, it occurs to me that most of them are, in one way or another, grappling with two critical mental forces: the power (and weakness) of human memory, and the sometimes overwhelming drive to procrastinate.
Let's start with memory, and put off procrastination for the time being. (Appropriate, right?) There are a number of ways that your memory can get in the way of a good writing session when you're in the middle of a project, mostly because you've remembered too much. But when you're just starting out on a project, when you're in that early stage where you're still trying to figure out what you want to write in the first place—at this stage, it's the frailty of memory that causes problems. This is because most good ideas (whether they're ideas for narrative structure, a particular twist in the argument, or a broader topic) come into our minds as hunches: small fragments of a larger idea, hints and intimations. Many of these ideas sit around for months or years before they coalesce into something useful, often by colliding with another hunch. (I wrote a chapter about this phenomenon in my last book, Where Good Ideas Come From.)
The problem with hunches is that it's incredibly easy to forget them, precisely because they're not fully-baked ideas. You're reading an article, and a little spark of an idea pops into your head, but by the time you've finished the article, you're checking your email, or responding to some urgent request from your colleague, and the next thing you know, you've forgotten the hunch for good. And even the ones that you do manage to retain often don't turn out to be useful to you for months or years, which gives you countless opportunities to lose track of them.
This is why for the past eight years or so I've been maintaining a single document where I keep all my hunches: ideas for articles, speeches, software features, startups, ways of framing a chapter I know I'm going to write, even whole books. I now keep it as a Google document so I can update it from wherever I happen to be. There's no organizing principle to it, no taxonomy--just a chronological list of semi-random ideas that I've managed to capture before I forgot them. I call it the spark file.
Now, the spark file itself is not all that unusual: that's why Moleskins or Evernote are so useful to so many people. But the key habit that I've tried to cultivate is this: every three or four months, I go back and re-read the entire spark file. And it's not an inconsequential document: it's almost fifty pages of hunches at this point, the length of several book chapters. But what happens when I re-read the document that I end up seeing new connections that hadn't occurred to me the first (or fifth) time around: the idea I had in 2008 that made almost no sense in 2008, but that turns out to be incredibly useful in 2012, because something has changed in the external world, or because some other idea has supplied the missing piece that turns the hunch into something actionable. Sure, I end up reading over many hunches that never went anywhere, but there are almost always little sparks that I'd forgotten that suddenly seem more promising. And it's always encouraging to see the hunches that turned into fully-realized projects or even entire books.
As an exercise, I just went back and grabbed a random sample of seven entries in my spark file from, all of them written about six years ago. They were, in reverse chronological order:
• a somewhat cryptic op-ed idea about eliminating airplane luggage altogether, a hunch that seems to have been a dead
end for me;
• an idea for a public panel discussion with a friend that I never actually put together, but which would still be a great event, and hopefully one day I'll figure out a way to make it work;
• my first stab at the title of what would become, four years later, Where Good Ideas Come From (my initial version was a question: Where Do Good Ideas Come From?);
• a little theory about the varying emotional effects of good and bad news on entrepreneurs, which I have talked about in meetings a hundred times, but not yet written about;
• a paragraph-long sketch for a book about Darwin, still unwritten, though some elements of this hunch made it into Good Ideas;
• and, finally, a note about a quotation-management tool that would, five years later, turn into the Findings.com service that I helped create.
I'd say that's a representative sample of the spark file: a few hits, a few misses, and a few yet-to-be-determined. But this kind of inventory doesn't quite convey the most interesting part of the experience, which is the feeling of reading through your own words describing new ideas as they are occurring to you for the first time. In a funny way, it feels a bit like you are brainstorming with past versions of yourself. You see your past self groping for an idea that now seems completely obvious five years later. Or, even better, you're reminded of an idea that seems suddenly relevant to a new project you've just started
Needless to say, maintaining and re-reading a spark file is useful for more than just writers, but I think it's a habit that is particularly suited to the special challenges of writing. I often find myself writing out full sentences of an argument or description, instead of just jotting down shorthand summaries, even though I don't yet know where the sentences are ultimately going to appear. The key is to capture as many hunches as possible, and to spend as little time as possible organizing or filtering or prioritizing them. (Keeping a single, chronological file is central to the process, because it forces you to scroll through the whole list each time you want to add something new.) Just get it all down as it comes to you, and make regular visits back to re-acquaint yourself with all your past explorations. You'll be shocked how many useful hunches you've forgotten.