Export Your Pocket Bookmarks to CSV

Introducing pocket2csv

I wrote pocket2csv to export Pocket bookmarks to a CSV file. Why CSV? Because my end goal is to import my Pocket data into Excel and leverage pivot tables to answer many of the basic questions I have about my bookmarks.

Change My Mind. (Originally from https://twitter.com/malanalysis/status/1300542384250458115)

This is two blog posts in one. The first part is a pocket2csv quick start. It’s aimed at people that know what Pocket is and want to use pocket2csv now. The second part explores why I wrote pocket2csv and what I learned from doing it.

pocket2csv Quick Start

pocket2csv is available at https://github.com/knilling/pocket2csv .

Generate a Pocket Consumer Key

pocket2csv needs a Consumer Key to work.

Here’s how you generate one:

1. Visit https://getpocket.com/developer/apps/new .
2. Then:

Generating a new Consumer Key for Pocket.

You’ll see something like this:

My Consumer Key.

After writing this, but before publishing, I deleted that Consumer Key.

Go generate your own Consumer Key. Keep it secret. Keep it safe. Then:

Get & Run pocket2csv

1. Grab a copy of pocket2csv via github or this way:

curl \
  -L \
  -J \
  -o pocket2csv \
  https://bit.ly/3kKEddL

2. Make sure it’s executable:

chmod +x pocket2csv

3. Run it:

Provide your Pocket Consumer Key when prompted for it.

4. Follow the instructions:

The link in step [2] changes every time you run the script

5. Press any key to continue:

When you see [+] Done, you should have your Pocket bookmarks in a file named bookmarks.csv.

My Use Case for pocket2csv

I use Pocket as the final stop in a workflow to collect open source information that is useful to me. Pocket makes bookmarking interesting spots on the web very easy.

In a way, Pocket reminds me of a dumbed down del.icio.us. (del.icio.us, RIP.)

If you want to bookmark something, Pocket’s various web and mobile user interfaces make it simple. Pocket’s mobile user interface and its integration in browsers like Chrome and Firefox are outstanding.

If you want to retrieve a bookmark, or look for patterns in your collection of bookmarks, though, Pocket’s various user interfaces leaves something to be desired. Search is not great. Tags are limited. It’s hard for me to find what I want easily.

I wanted to be able to answer basic questions about my collection of bookmarks, and I couldn’t find obvious ways to do it. For example:

• How many bookmarks do I have?
• How many bookmarks are tagged with the combination of tagX AND tagY?
• How many bookmarks did I add during this particular year?
• …month?
• …day?

And so on.

Fortunately, Pocket offers a REST API that allows you to export your collection of bookmarks. I came across Kevin Barroga’s blog post titled Export Pocket Data. That almost did what I wanted, except for two things:

1. The end result is not a CSV file.
2. Getting a Consumer Key was more involved than I hoped it would be.

Lessons Learned by Developing pocket2csv

Pocket Enabled Me to Bookmark MUCH More Than I Thought I Had

Before I started this project, I had been using Pocket for a few years and on a fairly regular basis. I figured I probably had a few hundred bookmarks. I was wrong.

1582 bookmarks! (Plus one for the header row.)

So it turns out that Pocket is probably a little more addictive than I would have thought. Perhaps this is a reason why the Pocket user interface doesn’t show you your total number of bookmarks.

Insomnia Core: Great for Working with REST APIs

I used the following version of Insomnia Core for some of the early development work:

Version: Insomnia Core 2020.4.1
Release date: 9/22/2020
OS: Linux x64 5.4.72
Electron: 9.1.1
Node: 12.14.1
V8: 8.3.110.13-electron.0
Architecture: x64
node-libcurl: libcurl/7.69.1 OpenSSL/1.1.1g zlib/1.2.11 brotli/1.0.9 libidn2/2.3.0 libssh2/1.9.0 nghttp2/1.41.0

Insomnia Core is a great tool for working with REST APIs. I tend to only interact with REST APIs from time to time, and so the free version of Insomnia Core works great for me.

Insomnia Core is very similar to Postman. Postman was a great tool for working with REST APIs, but I don’t like how you now need to register for an account at postman.com in order to use it. I’d rather not create yet another account for something that I might use twice a year.

I interact with REST APIs infrequently enough that if the folks at Insomnia decided to start charging for Insomnia or if they mandated that I create some kind of account with them, I’d probably look for a replacement — which is exactly what I did for Postman. On the other hand, if I needed to start working with REST APIs on a very regular basis, I would not hesitate to pay for a commercial version of Insomnia. I am not a fan of Insomnia’s subscription-based pricing model, however. I’d rather buy a single license for a tool like this.

Perl Is the Wrong Choice for Handling JSON & Unicode

pocket2csv is not written in Perl. It is a bash script that only uses jq and curl.

In the beginning of this project, however, I wrote an prototype of pocket2csv in Perl that seemed to work reasonably well:

This proof-of-concept takes Pocket bookmark json data in bookmark.json and attempts to convert it to csv. I discovered an issue, though. This script struggles to handle certain Unicode characters. For example, in one instance of one particular bookmark, the script would output:

Should read “Why I’m Learning Perl 6”

Just for the record, I’m not learning Perl 6 (which has been renamed to Raku — a completely new programming language that is not Perl). I’m sticking with Perl 5.32 until Perl 7 comes along.

Anyways, after a bit of digging, I discovered Felipe Gasper’s JSON, Unicode, and Perl … Oh My!. Gasper concludes:

JSON and Perl are odd bedfellows. Perl’s lack of distinct number and string types, for example, can yield JSON that uses the wrong type for one value or the other. Perl’s lack of native booleans produces a similar effect.

…

At the end of the day, Perl’s data model, for all of the conveniences that it affords us, makes communication with many other languages a challenge. The best we can do is to anticipate these problems and deal with them as they arise.

My main take away from this exercise: if you’re dealing with JSON data — particularly JSON data that has any chance of including Unicode characters — Perl is probably not a great choice.

This is a concrete case where Perl 5.32 is truly showing its age. Hopefully Perl 7 fixes it. In the meantime, I think Perl is probably best used as something like an awk++ . If I find myself reaching for something on CPAN (i.e. use JSON;) in the future, and I can’t get a quick script done in the core Perl language, that’s probably a sign to me that I should consider my options.

Could a different language like Python have been a better choice here? Probably. But, in this case, I rediscovered an even better solution.

Working with JSON? Use jq!

If you’re looking to solve issues that involve working with JSON data, your first move should never begin with writing code. This is true for any programming language. Instead, you should begin by experimenting with jq, and then decide if you really need the programming language that you’re considering. I learned this lesson before this project, which is why I say I “rediscovered” it.

From the jq website:

jq is a lightweight and flexible command-line JSON processor.

jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.

jq is written in portable C, and it has zero runtime dependencies. You can download a single binary, scp it to a far away machine of the same type, and expect it to work.

jq can mangle the data format that you have into the one that you want with very little effort, and the program to do so is often shorter and simpler than you’d expect.

jq is a fantastic little utility. It has no dependencies. It runs on everything. It is extremely well documented. It truly is like sed, awk, and grep for JSON.

pocket2csv makes extensive use of jq.

In my experience, it helps to have a commanding knowledge over the JavaScript programming language if you want to expertly wield everything that jq has to offer. That suits me — I’m very comfortable reading and writing JavaScript. However, that kind of knowledge is certainly not required to get many basic tasks done, in the same way that there is no requirement to have a commanding knowledge of ed in order to get stuff done withsed.