Expedia Group Technology — Software
The Greater Good of Our Cloud Migration
Improving Expedia’s carbon footprint
I like to find ways to link the very small to the very big. It somehow helps me orientate. For example, recently my wife called me and asked “Is the internet working?”. This question threw me and I considered long and hard before answering, “Do you mean in terms of our societal development or as a greater astral good?”. Her response: “What?, no I mean is the cable in the hole?”.
At Expedia Group™, I work on the Cloud Accelerate Team. A large part of our focus is helping guide software engineering teams out of the data centres and into a public cloud provider, AWS. I wanted to know if our combined efforts have a more altruistic purpose than cost reduction or shiny new toys. So I decided to try to understand if we were reducing carbon emissions. The story goes that cloud computing is one of the great examples of economies of scale in our industry. In theory, this improved economy should lead to less energy consumption which in turn should reduce carbon emissions. The answer I found was that, yes we are doing 69% better and here is the explanation.
A carbon emission model for the cloud
One of the most public figures to discuss this topic is James Hamilton, an AWS distinguished engineer. He has the fascinating job of designing AWS data centres and the infrastructure that links them together. At those huge scales, it is unsurprising that James spends a lot of his time optimising for energy consumption (If you haven’t already, I highly recommend you listen to a talk or two from James.). In his typical eclectic style, James provides a brain dump of, among other things, the science of data centre energy optimisation in a blog post called “Greenpeace, Renewable Energy, and Data Centres”. Jeff Barr, Chief Evangelist for AWS, picked up on this gold nugget and translated it into a working model that I could use for our cloud transformation. Thank you Jeff.
In hindsight, the model is very simple: the amount of servers being used, multiplied by the amount of energy each uses, multiplied by the amount of carbon that energy took to generate will tell you the contribution of carbon emissions. To work this out for our transformation, I had to collect some data points from around the place and make some assumptions. Here goes.
- Cloud server utilisation rate is reported to be 65%, which is astounding when compared to the private data centre average of 12–18%. Glorious is the cloud. It is also commonly assumed that less servers are needed in a cloud environment. Currently, we are in the middle of our transition and it is a challenge to get clear figures but we are down 57% from when we were fully in the data centres. AWS quote 77% as the target for a “full cloud transformation” (ie: on-prem to cloud native).
- The most common industry measure of energy efficiency is something called Power Usage Effectiveness (PUE). James Hamilton himself admits that this is not an exact science but it is very useful for our purposes here. In 2018, an industry report found on average data centres had a PUE of 1.7 and large public clouds were as low as 1.07 (Facebook) or 1.12 (Google). AWS don’t publish their PUE but Jeff mysteriously uses 1.2 in his official blog post on the subject. That is a 29% improvement in PUE. These efficiencies are gained by using the state of the art in facility design and can be considered a “learning effect” in economic terms.
- Finally, AWS reported 50% renewable energy usage in 2018 and were the second largest corporate buyer of solar & wind energy in the US (behind Google). AWS also have “long-term plans” to reach 100% renewable energy and are physically building a green power generation grid. Expedia’s physical data centres in Arizona operate on a 50% renewable energy mix from the Arizona energy provider APS. While currently equal in terms of carbon emissions produced in generation, 35% of the APS mix was nuclear power which is considered sustainable and renewable but assuredly falls in the “less good” category of energy generation.
Using Jeff’s calculation for our specific situation, the more efficient server utilisation and more efficient facilities gives a 69% reduction in carbon emissions (0.43 * 0.71 * 1.0 = 31% of data centre). Jeff calculates that this number is on average 88% and so we are well under that. The main reason for this is the high server utilisation rate the Expedia data centre team were managing after decades of experience at scale. Also our data centres were in Arizona where APS have a power mix comparable to AWS.
And so what?
Well, carbon emission calculations aren’t an exact science (yet) and there are enough assumptions built into the model that it’s not likely 69% is the exact figure. As the engineer who helped me put this together points out, “Ask an engineer [ie: me] what 2+2 is and they will say 4.0. Ask a lawyer and they will say between 3 and 5. Ask an accountant and they will ask you what you would like it to be.”.
However, the exercise did give me some nice insights into how we can do better. The main mechanism we have as a product engineering team is to optimise our compute workload to be maximally efficient in our new cloud environment. Second to that, we as an organisation can choose to seek out more efficient cloud providers. And thirdly, we can lobby our cloud providers to improve energy generation facilities.
An interesting thought experiment is possible where our cloud provider hits 100% renewable energy. At that point, because the environmental cost of the energy is free, none of this calculation is needed because wasted energy has no tangible downsides. The reality of high technology is that this is still far into the future. And even at that point, for us to become truly sustainable as an industry and position ourselves in a more favourable regenerative quadrant, we need to move towards PUE values of less than 1.0 which is the challenge James Hamilton has dedicated himself to.
Finally, AWS is happily publishing data and posting images like the below. While a picture may paint a 1000 words, those words aren’t guaranteed to be true. In fact, AWS don’t connect data centres directly to renewable energy sources. Instead data centres are connected to regional grids and renewable energy is bought on an open market. So despite public statements affirming the fact, the Virginia data centre is not harvesting energy from a 22,000 acre wind farm in Elizabeth City.
