How Fivetran + dbt actually fail: Part II
How much of Snowflake’s revenue is propped up by VC growth engineering and data influencer gimmicks?
We’re going to cover a lot of ground here.
I’ve had several people reach out to me after Part I.
They’ve also been cracking up at the whole thing. It’s VCs and influencers all the way down.
Anyway, let’s get into it.
First, How Fivetran Fails: Part II.
Then, let’s go further up the Mekong of dbt disreality.
How Fivetran Actually Fails: Part II
Fivetran, despite being a major computing partner of ~the cloud data warehouse~ for making so many tables, tables, and more tables, with an acquisition of HVR under their belts now and ideas in their heads that they will now cross-sell Fivetran into these HVR customers, went down for about 2.5 days in July.
Luckily, it was not the beginning of the month when it’s time to close the books. They would have screwed over all of the silly tech and ecommerce companies that:
- Use Fivetran to ingest transactions and orders data.
- Then have analytics engineers reconcile finances in the warehouse in SQL, dbt-ing around.
- Then spit these out to BI tools or Reverse ETL them into Netsuite or wherever this is somehow now deemed correct.
A sub-99% uptime is out of their SLAs and woeful for a company of their size, distribution, and valuation. This wasn’t even affiliated with a cloud region outage.
It is a huge risk for customers that one customer was able to do something stupid and bring Fivetran down for about 2.5 days.
What is interesting here is that I seriously doubt they saw significant churn or negative retention from their outage, even though many of the customers who rely on Fivetran would have faced frozen or out of date data anywhere downstream that relied on Fivetran to handle ingestion jobs.
I have personally stopped recommending Fivetran to customers given the 2.5 day outage as I cannot face that risk to my own business.
Going forward, Fivetran should invest in proper multi-tenancy and limit the exposure of their customers to what other customers are doing on the platform.
It begs the question though:
How important is BI/Analytics/data stuff for a lot of the VC-backed digital and ecommerce TAM that makes up a lot of Fivetran’s customer base?
And do a lot of Fivetran customers even know how to get on the phone and demand money back or credit?
Do they know ‘the nines’ and all that?
I don’t believe most do based my experience, so Fivetran may just take a small hit and not learn their lesson.
They have already lost one account that would have been $30k-40kish a year because I went with a different vendor for a customer, and with another customer I am now using another different vendor and that would have been about another $15k–20kish a year.
The question for Fivetran though is will they continue to build out new connectors and go further into the long tail or move further outside of tech/digital/ecommerce?
So far, they seem to be taking a service desk approach, bottoms-up on request.
With their scale, powder, and headcount this seems like the wrong approach. Instead they should be focused top-down and strategically go after specific applications with high distribution, then focus GTM around these. Connectivity is TAM, period. Being a 1 of 1 as a managed service is TAM, period, and a lot of the market is in play if they go after well distributed applications outside of the tech/ecomm specific world.
The bottoms-up approach has also been historically ignored by the business. Very little ever seems to be updated per the new connector and improvement requests in the Fivetran Support Forums.
There are many ‘holes’ on specific endpoints from major connectors, plus there are a lot of seemingly high TAM/good margin connectors that are just never serviced and this goes back years.
Fivetran has been out to lunch for the past 2 years.
How dbt Actually Fails: Part II
Speaking of out to lunch, let’s get into dbt Part II.
You can’t actually learn the product in depth or use it at scale without inevitably dealing with these VCs and ‘influencers’ that crawl around dbt. The majority of these people are completely useless and have no idea how to even use dbt let alone do much else with data.
Also, I have never seen any product where VCs are so involved, sticking their snouts into what people are doing, trying to figure out what monorails are being built and where, investing in all kinds of monorail insurance and monorail conductor outfits.
Of course, it’s all just selling into the lifestyle brand and audience, influencer after influencer, security breach after security breach.
The One Where Lauren Tried to Learn Analytics Engineering a Year Ago
A year ago, I reached out to dbt Labs to inquire about how to get training on dbt since more and more of my customers were struggling with or trying to use it or wanted to know more.
Perhaps I was missing something.
I was told that there were courses starting through something called the Analytics Engineers Club, which had been started by two people involved with dbt, but as a side business.
I looked into it and was told it was a few thousand dollars for going through the Analytics Engineers Club. Although I figured this was kind of a racket, I’d just write it off and I really wanted to see how the goose was cooked, to do things the right way.
I did the application asking about background and experience, put it through, and on there there was a diversity/underrepresented group question.
Now, I am someone who falls into a gender variant/intersex/transgender bucket. It’s just who I am, I have paid dearly for it at times in my life, but it’s the way it is.
So I filled out the answer to the question.
Now I was rejected(?) from taking the courses because I was considered too senior and experienced.
Okay, whatever, this is just some dumb thing.
I’d figure this dbt stuff out more on my own and get to the bottom of what the heck these dbt tests are and why they are so expensive to run even though the docs tell you to run tests.
Then, I get later contacted back saying there’s another person who falls into some gender variant/intersex/transgender bucket whom they rejected as well.
They felt she was not experienced enough to take the courses.
Alright, so now we’re playing Goldilocks and the Three Bears.
I was asked if I would like to be introduced. Although this is all weird at this point, I agreed. Maybe this other person wanted some help or something?
Then, I get some nutso email after this, which was the actual rejection email from the Analytics Engineers Club to this poor girl, with me copied as some kind of consolation prize.
No analytics engineering for you, also here’s another one of you you can talk to instead. Also we rejected Lauren.
Of course, she also thought this was weird.
See, then I figured it out.
The course is not to teach the technology. These are influencers.
The course is to make more influencers in their downline. It’s an influencer club for entitled people who are then going to go online and talk about how they changed their life by taking a class and changing their job title.
Charge for a class. Don’t let anyone in who isn’t going to be an influencer. Make more influencers. Keep the content mill spinning. Get everyone to talk about how great dbt is. Get a higher # of users in the dbt Slack channel. Get everyone making more compute to pump the valuation.
It’s like the monorail conductor course.
It’s all just monorails.
It’s all influencers and the influencers don’t know anything
There’s actually no point at all to the idea of dbt as a full-time job.
If you’re doing dbt as more than 50% of your job over a period of time, you’ve already lost.
Here’s the playbook.
Step 1: Change your job title to ‘analytics engineer’
Step 2: Make content
Step 3: Say dbt and analytics engineering changed your life
Step 4: Make more content
Step 5: Recruit others
Step 6: A year into your job realize that maybe doing full refreshes three times a day on the 1000 ‘models’ you’ve built is costing you $200k on your Snowflake bill and in reality if you’d been doing incremental updates it would only have been $40k
Step 7: Make content about how you realized this but then don’t change anything
There’s no training or anything, just a set of bottoms-up sold toys to play with, all of which spin compute and are often plugged into Snowflake or Databricks or BigQuery or whatever by people who don’t manage P&Ls or have any clue how a budget or data security work, most of whom work at hugely unprofitable tech companies that can afford all the human bodies and cloud credits this burns.
What is the impact on Snowflake’s revenue collection?
Here’s how the pyramid scheme plays out.
Let’s say in each year the # of dbt users has doubled from prior year.
A dbt user who knows what they are doing uses 1 Snowflake credit, a dbt user who doesn’t know what they are doing uses 2 Snowflake credits.
Only half the people who are brought on each year know what they are doing.
The other half just changed their job title and barely know SQL, let alone how databases work.
In 2020, when dbt got funded with an Andreessen Horowitz crypto fund, they began marketing heavily. So in this year, inefficiencies started adding up and they are passed back to Snowflake as revenue.
Then, in 2021, more people are recruited. More ineffencies start adding up. These are passed back to Snowflake as revenue.
Then, in 2022, more people are recruited, etc. More ineffencies start adding up. These are passed back to Snowflake as revenue. At this point there’s a swamp of tech debt created by all the monorails of SQL join railroads in the sky in the market.
Snowflake sits back and laughs. Their venture arm invests. Their account reps recommend dbt.
Here’s some more back-of-envelope.
If 1000 analytics engineers using ‘dbt+Snowflake’ each make $50,000 of overage inefficiencies in a year, that’s $50mm back to Snowflake in that year.
If 250 additional analytics engineers using ‘dbt+Snowflake’ each make $100,000 of overage inefficiencies in a year, that’s an additional $25mm, so now we’re at $75mm back to Snowflake in that year.
It is honestly this ridiculous at a number of customers I’ve worked with.
Remember: most of these people aren’t trained.
They are set loose on the cloud by earlier analytics engineers who are now analytics engineering managers (or whatever) and most don’t even know basic partitioning or anything about databases and many have removed themselves from both the analysis side and the actual data engineering side. Pure human middleware.
All of this is because of low interest rates. This isn’t ‘good’ revenue for Snowflake, it’s just bubble revenue propped up by vendor and VC gimmicks.
How Snowflake + dbt fails
The main way dbt fails is that all of this pyramid selling is already starting to lead to people ripping out Snowflake or whatever is being used, or at the very least starting to abandon it.
The platform/engine is always first to get blamed — that’s just the way it is, and it’s been like that forever, it’s all one big shell game.
The business sees Snowflake as one big rat nest and wants to know why the data is so out of date and the dashboards are so slow.
Of course, the actual problem is that you hired 6 analytics engineers who changed their job titles and didn’t know anything and they go on online and get their noggins filled with all sorts of bad ideas from data influencers and VCs, so now you’re paying $1.2mm for them, plus $300k to Snowflake for all the dbt inefficiencies they create.
Next year that $300k may be $600k. And the dashboards will be slower or more out of date.
By 2025 Ron DeSantis will be in the White House and we’ll all be drowning in monorails and data thinkfluencers.
I’m not kidding. dbt Labs has tons of powder to spend.
I know some people from Snowflake and Databricks are going to reach out to me all mad, but you all finance this propped up revenue circus directly and indirectly. I would think Google Cloud would also reach out mad as well, but nobody works there anymore, so I am safe.
Anyway, Hightouch is hiring a data influencer.
