Finding an Optimal TDP

Author: Fritz Kruger, Platform Architect, NUVIA

One of the single most important metrics for warehouse scale data centers is maximizing the perf/TCO ratio — the highest achieved Performance over the lowest Total Cost of Ownership (TCO). TCO can be broken into two categories, Capital expenditure (Capex), which is the cost of acquisition; and Operational expenditure (Opex), which is the cost of ongoing operations. So, the Figure of Merit of a server might look like this:

Perf / (Capex + Opex)

A key consideration when designing a CPU is what TDP to target. To investigate the optimal TDP for a server CPUs, let’s recast the above Figure of Merit as functions of TDP.

Perf(TDP) / (Capex(TDP) + Opex(TDP))

Note that each of these terms increases with increasing TDP, e.g. we get higher performance, but also higher costs, so it’s not immediately clear how to choose the TDP. Let’s consider each of these terms separately , and then combine them to see what it says about the perf/TCO metric.

For this exercise, rough approximations to explain the approach will be used, as well as some very large simplifying assumptions; these will be called out, but please note these do not reflect any actual CPU or semiconductor node. To start with, let’s assume a single socket server, and completely ignore all other power consumption such as DRAM or converter efficiency.

CPU Performance as a Function of TDP

For CPU performance as a function of TDP, there are two major knobs: adding silicon area by adding cores or making the existing cores larger; or increasing frequency by increasing the voltage. For a given TDP, these tradeoffs between frequency and area is a critical space to analyze, but this simplified exploration will explore just the single case of increasing frequency as a direct proxy for performance.

A voltage / frequency curve generally has an inverse exponential shape; this will be highly dependent on the process and physical design of the CPU, and the real world is never so elegant, but let’s use the hyperbolic tangent function as a very simplified approximation.

Frequency ∝ Tanh(voltage)

The curve might look something like this:

Dynamic power in a digital circuit is proportional to Cdyn * f * V^2 (capacitance of all the gates, times frequency, times the voltage squared). Pesky things like leakage for will be ignored in this exploratory case

If power vs frequency is plotted, the equation would look something like this:

TDP ∝ C * f * arctanh(f)²

And the curve would look like this:

We see severely diminishing returns in performance (frequency) at the top end of the graph.

TDP as a Function of Capex

Capex will include the costs of the CPUs, motherboard, DRAM, NIC, drives, etc. We’ll estimate that as $7000. The only parts of Capex in the server that are a function of TDP would be the power supply components, heatsinks, and fans. This generally makes for a small percentage of the total Capex. Let’s make another big assumption and say that it costs roughly $0.25/Watt for those components. But wait, don’t forget the physical data center! A warehouse scale data center requires a huge building, racks, network equipment, electrical substations, generators, busbars, chillers, evaporators and water piping. Every TDP watt planned to deploy has to get connected, its power in, and its heat out. To amortize it over all the servers in the data center, and over the lifetime of the building is a fairly large capex cost, which will be approximated here at $1.00/Watt. Based on this information the calculation looks like:

Capex(TDP) = infrastructure + server power components + server non-power components

Capex(TDP) = $1.00*TDP + $0.25*TDP + $7000

And we see that TDP has only a minimal effect on Capex.

TDP as a Function of Opex

The primary Opex cost is the ongoing cost of electricity and cooling. As much as it would be desired, data centers can rarely run all servers at 100% capacity, so an activity factor is required to estimate how much power is actually going to be needed. Let’s call that one 70%. And since this is a world class organization, with a world class PUE of 1.2, and access to industrial green electricity at a low $0.07/kWh ($0.61/W/yr) the calculations looks like this:

Opex(TDP)/yr = Activity * TDP * PUE * Electricity cost/yr

Opex(TDP)/yr = 70% * TDP * 1.2 * $0.61

Although Opex is strongly impacted by TDP, the absolute dollar value is not that high.

Figure of Merit of a Server

Pulling this all together: When charted, the Figure of Merit which is perf (frequency) over TCO (capex + opex), the result is:

It is clear that increasing TDP beyond a certain point doesn’t help the perf/TCO ratio, and zooming in a bit on that same graph:

It becomes apparent that this server should be run with about a 180W TDP to maximize our perf/TCO point.

Conclusion

Even with all of these simplifying assumptions, there is no simple, closed form solution to determine the optimal TDP point, so we do so numerically. What turns out to be interesting and perhaps non-intuitive is a sensitivity analysis to the inputs to our model. The big knobs end up being:

1) TDP independent Capex — server costs that are fixed regardless of TDP (e.g. DRAM).

2) TDP dependent Capex plus Opex — server costs that are directly proportional to TDP (e.g. power supplies and electricity).

3) The V/F curve — specifically the slope of the curve.

Holding all other inputs constant, the optimal TDP increases with increasing TDP independent Capex.

Holding all other inputs constant, the optimal TDP decreases with increasing TDP dependent Capex (or increasing Opex).

Holding all other inputs constant, the optimal TDP decreases with increasing Voltage/Frequency curve slope.

Choosing the optimal TDP point requires deep understanding and analysis of each of the functions in the Figure of Merit — how do Perf, Capex and Opex vary with TDP? The Capex and Opex portions are largely within the design of any given data center, and out of the CPU designer’s hands. Server CPU designers thus must employ a deep understanding of how data centers are designed, and work very closely with customers to understand their individual Capex and Opex profiles in order to provide the optimal TDP and highest value CPU. Conversely, data center designers and OEMs will understand that given a CPU with a particular TDP will push them to make different design decisions for their buildings and server chassis.

Special thanks to my co-worker Vadim Gektin who pointed out some of these interesting relationships to me.