How we monitor our time at Tesera
Our company has been using Harvestapp.com for a while now and appreciate the ease of use of the platform. The Harvest mobile apps have evolved nicely over the years and offer a convenient way to track time and expenses, in the office or on the go. As our company's business has grown larger and slightly more complex we’ve had to improve the way we monitor and evaluate our own resource allocation data.
Harvest Structure
In order to gather greater insight we’ve created some consistency within our Harvest structure. In Harvest for product development work we’ve created internal Clients as the largest container. Within clients we have created standardized projects that let us compare our business lines equally. Within the standardized projects, Tasks remain flexible as a tool for the Product Manager to manage operations. External projects are largely managed as before with a Client and a Project created for each.
Simply :
Client:Business Line > Standard Project …N > Flexible Tasks (with budget)
We’ve never been overly impressed with the Harvest reporting. It has always felt challenging to gain a big picture of resource effort even though the views on a singular project tend to be pretty good. Therefor we decided to separate concerns and find a new tool to handle our resourcing picture.
Hunt for Dashboards
With a structure in mind I reviewed several of the leading dashboard tools. Some products like Klipfolio offered sophisticated options but were a little tedious to configure and required learning a complex UI. Others like Geckoboard lacked the dimensionality to really explore the data. Nothing seemed to really get the visibility we required easily without a lot of configuration and perhaps developing an API middleware since Harvest is not often included in the direct integrations (probably due to the Harvest API).
Time data = logs
As we contemplated creating an API middleware to connect Harvest to our dashboard or visualization choice Boris Cosic realized the similarity between time data and log data. As a test he loaded an export of the time data into ElasticSearch which comes pre-packaged with a lightweight Exploration / Visualization tool called Kibana. The testing went quite well and and so we devised some simple ETL tools to move the data from Harvest on a regular basis.
Moving data
Getting data into elasticsearch is relatively simple from CSV data. The Harvest API was a little complex requiring a series of calls to first retrieve Projects, then Time, then looking up Task Names and Client Names. We elected to instead use the bulk export capability. Since there is no programmatic endpoint to this capability at present we applied a little duct tape in the form of using PhantomJS to “click” the button to export all our data for us on a one a day schedule while leveraging Zapier to pickup the e-mail from one of our accounts and place in in S3. From there we trigger Lambda and ECS to load the data into ElasticSearch. While overkill for this application, S3->Lambda->ECS follows an internal pattern we using for working with and processing CSV data. Even though the duct tape method of getting the data isn’t the most elegant we managed to complete our dashboard in a couple days.
Viewing Data
Once we worked out a couple of kinks with UTC times vs. Mountain Standard Times and got used the the syntax of ElasticSearch we were able to accomplish a performant and highly detailed dashboard on our resource allocation in a couple of days. We aggregated using filters by business line and standard project as per below putting the structure we added to Harvest to good use. We also track resources to ensure everyone is maintaining a good balance, to keep an eye on bill billable ratios, and to see which projects people are working on.
Success
Overall this initiative created a lot of value with minimum of development time by adopting and leveraging existing solutions. While we specialize in data systems and analysis the tools we apply to data analysis are also highly effective for improving our own business outcomes.
