Since data engineering typically comes with a fairly high maintenance burden, operational creep comes fast and disarms engineers faster than you can hire them. Yes, modern tooling help people be more productive, but arguably that’s only machinery that allows pipeline builders to keep more plates spinning at once.
If there’s a data engineer that is part of the conversation at all, it’s probably to help the data scientists and analysts gathering the data they need. If the data of interest isn’t already available in the structured part of the data warehouse, chances are that the analyst will proceed with a short term solution querying raw data, while the data engineer may help in properly logging and eventually carrying that data into the warehouse. Most likely an answer is required in a timely fashion, and by the time the new dimensions and metrics are backfilled into the warehouse, it’s already old news and everyone has moved on. The analyst will get the glory for the insight, and everyone else may question the need for the slow background process of consolidating this new piece of information in the warehouse.
…is that the photos are necessary to show Amazon shareholders that my business judgment is terrible. I founded Amazon in my garage 24 years ago, and drove all the packages to the post office myself. Today, Amazon employs more than 600,000 people, just finished its most profitable year ever, even while investing heavily in new initiatives, and it’s usually somewhere between the #1 and #5 most valuable company in the world. I will let those results speak for themselves.
Ironically, more investors have reached out (we’re just interested in raising money from our customers for the moment, thanks!), more folks want to contribute to Gumroad, and our shift in focus has brought us closer to our creat…
The future came into focus: I could grow a small team, slowly buy back our investors, and build Gumroad into a meaningful business focused on our creators. We would never become a billion-dollar company, and that started to feel okay. Certainly, the thousands of creators selling on Gumroad wouldn’t mind.
…ompany, raise more money, and do this all again with a new idea. But that didn’t sit right with me. We were responsible to our creators first. That’s what I told every new hire and every investor. I didn’t want to become a serial entrepreneur, and risk disappointing another customer base.