How $500,000 Microsoft Azure Sponsorship might kill your startup!

When you just started working on your startup, getting free credits from cloud services such as Azure or AWS might sound like an amazing thing to happen. However, you need to be extra careful with these free offerings since it would be easier to upgrade when you need more powerful servers vs optimizing your code. Your bills might ramp up quickly and it might be too late when you notice it.

Gercek Karakus
6 min readAug 6, 2018

At Raklet, we’re building automated messaging and payments solutions for communities. We got accepted into Microsoft Ventures Accelerator in April 2016 and we got offered $500,000 Azure sponsorship for the 3 years. Yay!

We were really happy to receive the credit as server costs were one of the major costs at that time. We started migrating our infrastructure from a monolithic architecture to a more scalable modular one. We’ve setup local, dev, test, prod resource groups; scaled up our app services and databases. Everything was running much faster, finally!

Then, fast-forward 24 months, I realized that our monthly server costs ramped up above $20,000/month. Unbelievable.

At that point, we only had 12 months left on our sponsorship and we’d run out of credits even before the expiration deadline. So it was time for us to focus on infrastructure optimization instead of growth.

We were able to drop our monthly costs down to ~$4k/month. Continue reading to find out how we did it!

There’s no way to attribute invoice items to actual resources or resource groups

Billing for Microsoft Azure Sponsorship is not on the portal so you have to visit an external website for the usage.

There’s no way on Azure Portal so see sponsorship usage.

Visiting the https://www.microsoftazuresponsorships.com website, you can only generate an excel sheet similar to the one below:

ResourceIDs and names in this sheet do not match to the ones on Azure portal. So we decided to reach out to the Professional Direct Services. After spending 2 months back and forth with them, they simply gave up and we were left with this problem. There’s no way for Microsoft Azure Sponsorship usage to be tied to actual resources on Azure Portal.

I hope this email finds you well. Please note that my team, the Startups Business Desk, has been closely following the congruent support request #118041818024304. As stated in that support request, it is a technical limitation of Azure Sponsorship reporting that we are unable to provide usage by resource group. We understand that this is an important feature and heavily weighs on your continued use of the platform and apologize that we cannot provide a workaround at this time. Please note that engineering is working to implement reporting by usage group in the future. However, we do not have an ETA. — Microsoft Azure Team

We were left with the excel sheet to figure out what was going on with our billing. So, we built pivot tables to figure out what was going on with our account.

We downloaded the excel file and became a pivot table master. With some tricks, we were able to identify break down of each service such as name, type and resource.

Sounds like the ultimate solution but again, ID or names in this sheet do not map to your resources on Azure.

Our billing for March 2018
March 2018 Billing Details — Drilling down into each service would shed some light into the type of the service. It was still not sufficient since we are using similar resources for local, test and prod environments.

We were still not able to identify which resources were being used for different resource groups (local, dev, test, prod) out of our 250+ resources. So we had to drill down into each resource to find our way to optimization.

We were able to drop down to $7.5k/month in May and $4k/month in July. Below you can see the breakdown for July 2018.

Top-level billing view for July 2018
Billing breakdown for July 2018

How did we decreased our costs?

Let me summarize what we have done for each service type and how much it helped:

  1. First thing first: Identify and delete unused resources! We found out that dead code was consuming significant amount of resources.
  2. Make sure you’re using the right scale for your resources. Premium vs Standard resources have a huge difference in pricing.
  3. App Service: If you have multiple apps running on different WebApps, use App Service Plan. Each one of them have 10 slots so you can serve multiple apps and pay only for the App Service Plan.
  4. SQL: Use Elastic Database Pools to contain multiple SQL servers. It helps you share resources and get charged once instead of per server or database.
  5. CosmosDB: make sure you have a single container as you get charged per container. If you need to separate your documents, simply use a single container with PartitionKey
  6. Storage: Identifying total size of each storage account on the portal is impossible. We had to reach out support to give us an excel file with the break-down of each storage and identified that logs from apps were not being deleted automatically.
  7. Cache: Scaled down Redis and fit in the smallest standard plan which is plenty anyway (250MB).
  8. VMs: We are mainly using them for remote development so each developer has an instance. Downgrading and removing premium disks help a ton.

What’s next?

With a little bit educated guess work, we can categorize resources to different groups (again in a sheet) and see that our daily costs are around ~$120/day as of August 1st, 2018.

Our daily cost on August 1st, 2018
  1. Since highest item on the list is VMs, we’ll start development on localhost vs working remote on VMs. We’ll use Docker for Redis and Elastic Search, Azure Storage Emulator for queues, blobs and tables.
  2. We’ll then remove our dependency for .NET 4.5 and upgrade to .NET Core to fit into smaller size app service plans.
  3. Cache heavily, read from ElasticSearch and minimize SQL interactions.

Long story short!

We’re lucky to identify this issue with Azure billing with sufficient time in our hands so that we can prepare accordingly.

We’ve always postponed optimizing our infrastructure in order to focus on growth which is pretty much understandable startups searching for product-market fit. However, years pass by with the blink of an eye and credits expire much faster than anticipated.

Our goal is to drop our costs to $1,000 per month and we’ll see how far we can go…

Software engineer? Interested in challenging Microsoft Azure to drop monthly costs from $20k to $1k? Join us!

Thanks for reading this far and please spread the word 🙏

More where this came from

This story is published in Noteworthy, where thousands come every day to learn about the people & ideas shaping the products we love.

Follow our publication to see more product & design stories featured by the Journal team.

--

--