Developer Diary — Week 3

TL;DR

1. Finalised BigTable architecture;

2. All exchange data outputting onto a local instance of BigTable;

3. Ready to populate a cloud instance of BigTable; and

4. Query design completed and to be worked on next week.

“I like Bigtable and I cannot lie, you other brothers can’t deny, that when a query walks in with a…”

Moving swiftly on…

Designing RowKeys is sorta like choosing names for your variables i.e. the hardest part of programming.

And we had decided to pull much more data from the exchanges than we initially intended, so there were a lot of RowKeys to design. Which inevitably meant I was staring into space for a good portion of the week.

See: “Development is an iterative process…”

I am rather thankful I have a 49 inch monitor for these sorts of tasks. You get a spreadsheet up on that baby and you are seriously looking at that spreadsheet.

Nevertheless, final design of the database was done, and God help me I would not be touching it again.

So all that was really left was to populate a local instance of Bigtable, to prep for instantiating the cloud copy, which was, quite mercifully, working as intended. And by prep, I mean I had drawn a smiley face on a post-it note and stuck it to my monitor.

As for actually moving data, I might need a few more weeks, a hazmat suit and a priest to exorcise any lingering bugs.

As a consequence of staring into space for hours on end designing RowKeys, query design was mostly done for me, as good RowKeys are designed around the queries you intend to submit to the database, another win for me!

So, next week I’d be looking forward to setting up the webserver and writing the typescript (pls no) packages in a misguided attempt to alleviate the front-end dev’s workload.