StarThinker Open Source Support Ends At Google

Over the past few years the StarThinker open source project has been a tremendous pleasure to work on with stakeholders both internal and external. In the incredible 5 years the project has evolved with Google, gTech, and Google Cloud. On the other hand, as Google Cloud continues to mature, the gap StarThinker covers has successfully shrunk. It’s clear that this scaled infrastructure project has served its purpose, and now it is time for it to be solely a reference and no longer a Google maintained project.

Given it is hard to track how many active implementations are out there for an open source solution like StarThinker, our sunsetting plan will be mainly reactive.

The Clients

External clients who have implemented and are using will can continue to do so. Whenever a maintenance request come in, we will communicate it is no longer supported.

The Solutions

My Customer Solutions Team at Google is creating stand alone versions of each solution, look for alternatives on GitHub in independent repos.

The Code

The GitHub repository will remain available as a reference for existing open source users.

The Retrospective

Aside from the plan, there are some nostalgic milestones I’d love to share with you all:

1. StarThinker was the first open source gTech project for our Customer Solutions Team at Google.
2. Peaked at 48 Forks, 150 Stars, 1,000+ Weekly Views, and 100+ Weekly Clones.
3. An incredible 82 connectors and solutions built and open sourced.
4. Automated tests covering 90% of code.
5. Complete API coverage for 4 Ads products:
   Campaign Manager 360
   Display Video 360
   Search Ads Manager
   Google Analytics
6. The same solution code can be automatically deployed to :
   AppEngine — Full browser based UI with multiple user login.
   Cloud Function — Serverless deployment to Google Cloud Functions.
   AirFlow — Run recipes using cloud composer / airflow framework.
   Colab — For Workshops, deployment using Google Collaboratory.
   Developer — Full test suite in a local development machine.
   Command Line — Quick easy to install utility for running recipes.
   PIP Package — Because sometimes you just want to import.
7. From 2018–2021, 100+ Googlers deployed over 1,000+ data pipelines with the internal UI.
8. Over 550 confirmed unique users downloading solutions from our Ecosystem partners:
9. A total of 15 Googlers contributed to the code with 36 Googlers as reviewers.

Lessons Learned

With all that we’ve accomplished with StarThinker there are some incredible lessons learned:

1. The core framework only accounted for only 3.2% of commits, and 0.6% of LOC maintenance.
2. 47% commits and 70% LOC maintenance was spent on solutions and connectors.
3. 18% of commits and 7% LOC maintenance was spent on documentation and testing.
4. 7% of commits and 16% LOC maintenance was spent on the UI deployment.
5. The rest (< 4%) was deployment tools and scripting.
6. Some tracking of external usage and adoption is possible using Analytics / Google Groups.
7. Authentication is really hard, most agency teams struggle with a full stack implementation.
8. A beautiful public solution gallery is possible using GitHub.
9. Django can be used to build an absolutely incredibly scalable workflow architecture, even Googlers were impressed.

The Future

Within Google, StarThinker is being rolled into both product features and a new global tool being built by gTech. With regards to open source, the solution will remain open to the community as a reference. The PIP package will remain at the current version.

So… fork, develop and prosper.

Thank You

For all the users and contributors who made this project possible, lent a hand at some impossible task, or took the time to guide me on this journey, thank you! It’s been a genuine pleasure working with you!