In Search of a Personal Optimal Diet

exploring the effects of your diet on your health


This essay is about diet advice. Concretely, it describes why your most trusted diet advice for increasing health and decreasing health care costs will soon come from measurements of your personal metabolism, not from diet books or government-promoted dietary guidelines. This prediction is based upon three observations.

1. Your metabolism is more dynamic than books and guidelines.
2. Testing your metabolism is tricky but getting easier. 
3. The costs of not understanding metabolism are increasing.

After some background and details about those three observations, this essay describes some optimal diet calculations, optimal diet motivations, a privacy policy for diet and metabolism data, and a variety of dissenting views. To skip the details, skip to the conclusion.


As a software engineer, I have learned that good testing is associated with successful projects. So before, during and after developing software, I think about testing it. However even though I eat every day, I almost never think about testing my diet. An exception happened in December 2014 when I finally realized that six years of a small and minor scalp inflammation is actually an excellent opportunity for testing my diet. Investigating a claim that inflammation can be reduced by low-carb high-fat diets, I read The Art & Science of Low Carbohydrate Living by Jeff Volek and Stephen Phinney.

In addition to the claim about inflammation, I am also intrigued that a low-carbohydrate high-fat diet can reduce a person’s Respiratory Quotient (RQ), the ratio of carbon dioxide produced to oxygen consumed, by as much as 30%. Coincidentally, about 15 years ago I helped write software that slightly decreased the carbon dioxide produced by a coal fired steam generator. By polling an assortment of sensors, the software calculated the optimal time for steam generator tube cleaning, performed by a large rod called a “soot blower” that occasionally moved around and sprayed water … while the power plant was fired up and fully operating. Cleaning too frequently unnecessarily dampened the fire. Cleaning too infrequently allowed an excessive buildup of insulating carbon on the tubes. In that energy industry project, even a 1% decrease in coal burned and the corresponding 1% decrease in carbon dioxide waste was very significant, which makes a 30% decrease in a body’s production of waste carbon dioxide seem extraordinary even though the comparison is not entirely consistent: the power plant’s decrease in carbon dioxide was due to a process change (i.e., timing of cleaning) and the body’s decrease is due to a fuel change (i.e., fats instead of carbohydrates).

So I started a test of my diet, eating less than 50 grams of carbohydrates per day (including the fiber from a few cups of vegetables) while maintaining my overall calorie count by eating a lot more fat: nuts, olive oil, coconut oil, pasture raised eggs, cheese and heavy cream in the form of homemade high fat Greek yogurt. To my surprise, about half my scalp inflammation disappeared in three months. Although my measurements of diet and inflammation were vague and perhaps not reproducible by others, I was encouraged by the results. In contrast, there were no positive results in the previous six years, which included two examinations by doctors and a $70 shampoo prescription. But now what? To get rid of the remaining inflammation, should I eat even fewer carbohydrates, or a different balance of omega-3 and omega-6 fats, or more probiotics? Consensus on diet advice is rare. In fact, it is easy to find opposing opinions, and sometimes those opposing opinions are backed by opposing results from different diet studies. In terms of the level of contention, diet advice might be right up there with politics.

David Brooks, the New York Times columnist, likes using the phrase “epistemological modesty” when talking about politics and public policy. It is a phrase he borrowed from Edmund Burke, the 18th century Anglo-Irish statesman, political theorist and philosopher. Epistemological modesty is an acceptance that we are very limited in our capacity to completely understand even a single topic. In the words of David Brooks, people who are epistemologically modest tend “to be skeptical of technocratic, rationalist planning and suspicious of schemes to reorganize society from the top down” because

“The political history of the 20th century is the history of social-engineering projects executed by well-intentioned people that began well and ended badly. There were big errors like communism, but also lesser ones, like a Vietnam War designed by the best and the brightest, urban renewal efforts that decimated neighborhoods, welfare policies that had the unintended effect of weakening families and development programs that left a string of white elephant projects across the world.”

There are frequent reminders of epistemological for software developers. With the help of compiler tests, unit tests and system tests, our many mistakes are revealed – one by one – ratcheting up the quality of our initial drafts of software but rarely (i.e., almost never) reaching the point of being proven completely correct. Among professional science writers, epistemological modesty is enforced by testing that is less automated but, in at least some cases, more thorough than the testing that is typical in software development. For example Harvard professor Steven Pinker, the cognitive scientist and prolific author, has setup a process for ratcheting up the quality of his new book manuscripts by putting them through six rounds of external editing. Experts in the field give their feedback to professor Pinker at round two or three. His editing process demonstrates a desire for feedback and a dogged determination to work through mistakes.

In contrast, there is much less epistemological modesty in the realm of diet advice. The reason, from my perspective, appears to be a lack of good testing opportunities. In general, good testing is based upon the objective measurements and careful analysis of inputs and outputs. Concerning inputs, traditional diet surveys are known to be unreliable. Concerning outputs, body weight is an unreliable indicator of health. In other words, diet study data has been very noisy.

Fortunately with today’s inexpensive technologies, there are opportunities to remove some of the noise in diet testing and increase the quality of diet studies. Due to the scale of diet related health problems, the need for improvements in diet testing are immediate. For example, the World Health Organization recently reported that 350 million people worldwide have diabetes and about 90% of the cases are Type II. “Over time, high blood sugars can wreak havoc on every major organ system in the body, causing heart attacks, strokes, kidney failure, blindness, impotence and infections that can lead to amputations. … Diabetes is the direct cause of death for 1.5 million people each year.”

Three Observations

The following three observations prompted the prediction that your most trusted diet advice for increasing health and decreasing health care costs will soon come from measurements of your personal metabolism, not from diet books or government-promoted dietary guidelines.

1. Your metabolism is more dynamic than books and guidelines.

Your body is continuously renewing itself. The New York Times Science article “Your Body Is Younger Than You Think” by Nicholas Wade describes the renewal,

“Although people may think of their body as a fairly permanent structure, most of it is in a state of constant flux as old cells are discarded and new ones generated in their place. Each kind of tissue has its own turnover time, depending in part on the workload endured by its cells.”

Your colon lining gets replaced about every week.

“The epithelium of the human colon turns over at least once per week throughout life. As cells die at the surface, they are replaced by new cell divisions. By age 60, a person has been through at least 3,000 replacement cycles, which means that some cell lineages must pass through many generations. Those renewing lineages would be at high risk for accumulating mutations and progressing to cancer.”

Your outer layer of skin, the epidermis, gets replaced about every two weeks.

“The epidermis, or surface layer of the skin, is recycled every two weeks or so. The reason for the quick replacement is that “This is the body’s saran wrap, and it can be easily damaged by scratching, solvents, wear and tear,” says Elaine Fuchs, an expert on the skin’s stem cells at Rockefeller University.”

Your red blood cells get replaced about every four months.

“The red blood cells, bruised and battered after traveling nearly 1,000 miles through the maze of the body’s circulatory system, last only 120 days or so on average before being dispatched to their graveyard in the spleen.”

Your liver gets replaced about every year.

“As for the liver, the detoxifier of all the natural plant poisons and drugs that pass a person’s lips, its life on the chemical warfare front is quite short. An adult human liver probably has a turnover time of 300 to 500 days, says Markus Grompe, an expert on the liver’s stem cells at the Oregon Health & Science University.”

Your skeleton gets replaced about every 10 years.

“The entire human skeleton is thought to be replaced every 10 years or so in adults, as twin construction crews of bone-dissolving and bone-rebuilding cells combine to remodel it.”

And your body’s amazing continuous renewal is just one task performed by your metabolism, defined by Wikipedia as “the set of life-sustaining chemical transformations within the cells of living organisms. These enzyme-catalyzed reactions allow organisms to grow and reproduce, maintain their structures, and respond to their environments.” The derivation of the word metabolism is from the Greek word metabole meaning “change” and that is probably a good way to think about your body in general. You are dynamic and you are built for change, and so were your ancestors.

Authors of diet books and dietary guidelines have the unenviable task of recommending a good diet for a wide variety of people. If we all had similar metabolisms, those generalizations could be optimal. However, it appears that our metabolisms are quite diverse. Perhaps as diverse as the microbes in our guts and perhaps as diverse as the collective diets of our ancestors: given that our ancestors traveled much less than we travel today, and the food of our ancestors traveled much less, it seems reasonable to assume that the metabolisms of our ancestors subtly adapted to fit their locally available food.

The 2013 journal Nature article “Archaeology: The milk revolution — When a single genetic mutation first let ancient Europeans drink milk, it set the stage for a continental upheaval” by Andrew Curry describes how a single genetic mutation, perhaps originating near the current country of Hungary about 7500 years ago, now allows about 35% of the world’s adult population to digest the lactose in milk. Typically, the ability to digest lactose stops when children are about seven or eight years old.

The article goes on to mention that similar genetic mutations may be responsible for amylase production (an enzyme that helps break down starch, allowing certain subpopulations to efficiently digest grain) and alcohol dehydrogenase production (an enzyme that helps certain subpopulations to efficiently break down alcohol). Of course, genes also affect skin color, which can be interpreted as an ancestral optimization for the skin’s vitamin D production at different geographical latitudes.

If indeed metabolisms vary due to ancestral diets, ancestral locations, and current unique gut microbiomes, dietary guidelines are suspect when they are based upon generalizations over large populations. Statisticians call that type of over-generalization “underfitting”. David Brooks might call it a lack of epistemological modesty.

2. Testing your metabolism is tricky but getting easier.

To test how particular foods affect your particular metabolism, it should not be a surprise that you will need to measure your diet. In a presentation at Ohio State University, Dr. Jeff Volek describes the complexity of measuring diet like this:

“But just to give you a sense of that complexity, even a single meal contains usually all the macronutrients (carbs, protein, fat, maybe throw alcohol in there), and then you’ve got a dozen or so essential micronutrients (vitamins and minerals), and then a whole range of bioactive phytonutrients that have important effects, in many cases, on cellular function and health. And so all these different bioactive nutrients are interacting with our genome, ultimately manifesting in our phenotype and some type of biological response, metabolic profile, that a person has. So this is immensely complex to sort through.”

Adding to that complexity, the U.S. food industry offers 600,000 different food items with a churn rate of 24,000 new food items introduced every year. Adding yet more complexity to the task of testing diet and metabolism, compliance in diet studies is poor. For example, diet study attrition rates range from 10% to 80%. Even if participants do not quit the diet study, controlling and measuring what free-living people eat has been problematic (e.g., people frequently under-report in dietary surveys). However, there are ways to help control the complexity.

Concerning the wide variety of available food, selection becomes much simpler when following one fundamental principle: give your gut a break by eating real foods that easily rot. Consequently, the proposed search for a personal optimal diet promotes foods that are free of preservatives (e.g., freshly prepared fresh food), free of antibiotics (e.g., pasture raised meats and eggs), high in fiber (e.g., lots of fresh vegetables and nuts), and high in probiotics (e.g., yogurt and sauerkraut).

Concerning measuring food intake, for decades, Optical Character Recognition (OCR) has been used by the banking industry to automate the reading of handwriting on checks. Similarly, a smartphone application can automate the reading of grocery receipt text. In fact, reading grocery receipt text is easier than reading handwriting on checks because grocery receipt text is neatly printed in fixed width characters.

3. The costs of not understanding metabolism are increasing.

As described by Robert Lustig, M.D. and Professor of Pediatrics at UCSF Benioff Children’s Hospital, in the documentary The Skinny on Obesity,

“So when you add up the medical costs for those eight diseases [type 2 diabetes, heart disease, lipid problems, hypertension, non-alcoholic fatty liver disease, cancer, polycystic ovarian syndrome, dementia] that is 75% of health care expenditures.”

That is 75% of $3.0 trillion health care expenses per year in the United States. Furthermore, the recent announcement of the “Prevent Diabetes STAT” initiative by the American Medical Association and the Center for Disease Control and Prevention begins,

“With more than 86 million Americans living with prediabetes and nearly 90 percent of them unaware of it, the American Medical Association (AMA) and the Centers for Disease Control and Prevention (CDC) today announced that they have joined forces to take urgent action to Prevent Diabetes STAT and are urging others to join in this critical effort.”

Suggesting an increasing problem, the announcement goes on to say,

“People with prediabetes have higher-than-normal blood glucose levels but not high enough yet to be considered type 2 diabetes. Research shows that 15 percent to 30 percent of overweight people with prediabetes will develop type 2 diabetes within five years unless they lose weight through healthy eating and increased physical activity.”

Calculating a Dietary Guideline

As background before describing some of the calculations in the proposed search for a personal optimal diet, here is a brief overview of some of the definitions and calculations used in the government promoted dietary guidelines.

  • Estimated Average Requirement (EAR): a nutrient intake value that is estimated to meet the requirement of half the healthy individuals in a group.
  • Recommended Dietary Allowance (RDA): the average daily dietary intake level that is sufficient to meet the nutrient requirement of nearly all (97 to 98 percent) healthy individuals in a group.
  • Tolerable Upper Intake Level (UL): the highest level of daily nutrient intake that is likely to pose no risk of adverse health effects to almost all individuals in the general population. As intake increases above the UL, the risk of adverse effects increases.

According to the Institute of Medicine, Food and Nutrition Board,

“The process for setting the RDA depends on being able to set an Estimated Average Requirement (EAR). That is, the RDA is derived from the nutrient requirement so if an EAR cannot be set, no RDA will be set. The EAR is the daily intake value of a nutrient that is estimated to meet the nutrient requirement of half the healthy individuals in a life stage and gender group. Before setting the EAR, a specific criterion of adequacy is selected, based on a careful review of the literature. When selecting the criterion, reduction of disease risk is considered along with many other health parameters. The RDA is set at the EAR plus twice the standard deviation (SD) if known (RDA = EAR + 2 SD); if data about variability in requirements are insufficient to calculate an SD, a coefficient of variation for the EAR of 10 percent is ordinarily assumed (RDA = 1.2 x EAR)”

In other words, to be healthy eat what healthy people eat. That circular logic may have been the rationalization for modeling metabolism as a linear system. Intriguingly, any non-linear system can be modeled by a series of piecewise linear subsystems if their input domains are not too large, and that may explain why the Institute of Medicine divided the nutrition study input domain (i.e., the U.S. population) into 22 subpopulations. According to the published Dietary Reference Intakes Estimated Average Requirements, the Institute of Medicine calculates an Estimated Average Requirement for each of the following groups of “healthy individuals in a life stage and gender”:

  • infants from 0 to 6 months old,
  • infants from 6 to 12 months old,
  • children from 1 to 3 years old,
  • children from 4 to 8 years old,
  • males from 9 to 13 years old,
  • males from 14 to 18 years old,
  • males from 19 to 30 years old,
  • males from 31 to 50 years old,
  • males from 51 to 70 years old,
  • males older than 70 years old,
  • females from 9 to 13 years old,
  • females from 14 to 18 years old,
  • females from 19 to 30 years old,
  • females from 31 to 50 years old,
  • females from 51 to 70 years old,
  • females older than 70 years old,
  • pregnant females from 14 to 18 years old,
  • pregnant females from 19 to 30 years old,
  • pregnant females from 31 to 50 years old,
  • lactating females from 14 to 18 years old,
  • lactating females from 19 to 30 years old, and
  • lactating females from 31 to 50 years old.

That is a total of 22 healthy groups. Specific to each group, the Institute of Medicine seeks to recommend the dose of 27 nutrients, resulting in a matrix of 594 (i.e., 22 x 27) recommendations. It is a monumental task, indicated by the resulting Institute of Medicine’s 1344 page summary publication Dietary Reference Intakes (DRI) : The Essential Guide to Nutrition Requirements. Half of the summary’s 1344 pages are lists of references to research papers.

Yet, within all that published work, there appears to be no analysis of the accuracy of the assumption that the 22 healthy groups are sufficiently small to make a good piecewise linear model of metabolism. Did anyone at the Institute of Medicine consider analyzing the strength of their assumptions? It is easy to think of cases in which the 22 subpopulation piecewise linear metabolism model is not good. For example, given the variations in genetics (e.g., skin color) and environments (e.g., different exposures to sunlight at different latitudes), it is unlikely that a subpopulation’s normal probability distribution of vitamin D nutrient intake would result in a normal probability distribution of vitamin D in their blood, and consequently even less likely to result in a normal probability distribution in health outcomes due to vitamin D, as assumed by the Institute of Medicine.

Acknowledging the limitations of its approach, the Institute of Medicine advises, “Interpreting nutrient intake data in relation to the DRIs can enhance the assessment of an individual’s diet; however, the information obtained must be interpreted cautiously because an individual’s true usual intake and true requirements must be estimated, and assessment of dietary adequacy is only one component of a nutritional status assessment. Ideally, intake data are combined with clinical, biochemical, or anthropometric information to provide a valid assessment of nutritional status.”

In theory, the Institute of Medicine could provide a more valid assessment of nutritional status by significantly increasing the number of healthy groups it studies (for example 22,000 instead of just 22 groups of healthy people). However in practice, there are not enough resources for the Institute of Medicine to study every healthy group associated with a metabolism nonlinearity, including the human genome’s 10 million Single Nucleotide Polymorphisms (SNPs), each a potential nonlinearity. And studying the diets of unhealthy groups would require even more resources.

Calculating an Personal Optimal Diet

In contrast to the Institute of Medicine focusing on 27 individual nutrients in the diets of large groups of healthy people, the proposed search for a personal optimal diet examines metabolic responses to combinations of nutrients in the diets of individuals, in any state of health, to help reveal answers to three basic questions:

  1. What nutrients are in the food you buy? Manually calculating the sum of nutrients purchased in a typical grocery store visit would be tedious. However, instead of manually adding the nutrition fact label data from individual products, it is possible that a smartphone application could quickly augment a grocery receipt picture with nutrition information in a form that is easy to analyze — in essence, nutrition analytics.
  2. What nutrients are in the food you eat? Of course, you might not eat all the food you buy (e.g., sharing groceries within a family). However, it is possible that a smartphone application could estimate the nutrients you eat based upon food receipts and pictures of the meals you are about to eat.
  3. How is your nutrition likely to affect your health? Because your metabolism is nonlinear, your metabolic response to a meal with a variety of nutrients is not necessarily the sum of your metabolic responses to each individual nutrient in the meal. For example, if you ate a meal that was only fat, your blood glucose level would not significantly rise because carbohydrates (that get converted into glucose) were entirely absent from the meal. Alternatively, if you ate a meal that was only carbohydrates, your blood glucose level would rise but, assuming your body produces sufficient insulin and your body is sufficiently sensitive to it, your insulin would trigger the removal of glucose from your blood and your blood glucose level would quickly return to normal. However, because fat reduces insulin sensitivity, somewhere in between a meal of only fat and a meal of only carbohydrates is a meal that would cause your worst case blood glucose level response. That unhealthy meal would have enough carbohydrates to significantly spike your blood glucose level and also enough fat to significantly decrease your ability to remove the glucose from your blood. So, is a fatty diet good or bad for your blood glucose level? It depends.

The proposed search for a personal optimal diet investigates the dependencies in your diet by measuring your diet and testing your metabolism thousands of times. The resulting data is processed by a time extended version of Bayes theorem,

P(E(t+Δt) | C(t)) = P(C(t) | E(t+Δt)) * P(E(t+Δt)) / P(C(t)),

where P denotes ‘the Probability of’, 
E(t) is metabolic Effect (e.g., an increase in blood pressure) at time t, 
C(t) is dietary Cause (e.g., some combination of nutrients) at time t, and
Δt is the time difference between the Cause and the measurement of the Effect.

With enough acquired diet and metabolism data (i.e., C and E) and enough variation in measurement timing (i.e., t and Δt), possible causality signals, like the following figure, can be investigated. For example, C(t) could be a meal with a particular combinations of fats and carbohydrates, and P(E(t+Δt)|C(t)) is the likelihood of a particular spike in your blood glucose level in response to that meal.

In the search for a personal optimal diet, there are just a few simple rules about data acquisition.

  1. Test your metabolism immediately before each meal.
  2. Measure each meal.
  3. Optionally, test your metabolism more often.

As a practical matter, the large amount of metabolism testing implies that the metabolism tests must be virtually free. Here is a list of some free or very inexpensive metabolism tests, and the list is expected to grow as technologies advance.

  • blood pressure — from Walmart, an automatic blood pressure monitor costs $39.88, and each test is virtually free (the 4 AA batteries that operate the pump and electronics occasionally need to be recharged or replaced).
  • urine — from Amazon, 10 parameter urinalysis reagent strips cost about $0.16 each.
  • blood glucose — from Walmart, a glucose monitor costs $16.24, the lancing device costs $13.56, the disposable test strips cost about $0.18 per strip, and the disposable lancets cost about $0.04 per lancet.


Realistically, would most people frequently measure their diet and metabolism? Probably not, at least not without some combination of support and incentives. “Job-to-be-done” is a phrase Clayton Christensen uses to describe a current and known problem that a person actively tries to solve. For example, in the context of eating, satisfying hunger is a job-to-be-done. However, eating a healthy diet when there is not a current health problem is usually not a job-to-be-done because potential health consequences are a future unknown problem, not a job-to-be-done. When there is an acute health problem, people are motivated to eat a healthier diet but diet studies generally do not focus on diets for people with acute health problems. In other words, diet studies are not interesting when people are healthy and are not available when people are sick. Concerning diet studies for sick people, the following section (i.e., A Privacy Policy for Diet and Metabolism Data) describes a possible approach for sick people to voluntarily assemble and conduct their own diet studies. Concerning the general lack of interest in diet studies when people are healthy, employers might be willing and able to help.

About 60% of U.S. companies self-insure the health care expenses of their employees, a percentage that is increasing by about 1% per year, according to Paul Fronstin’s “Self-Insured Health Plans: State Variation and Recent Trends by Firm Size” from the Employee Benefits Research Institute Notes, November 2012 that says, “In 2011, 58.5 percent of workers with health coverage were in self-insured plans, up from 40.9 percent in 1998.” Although companies that self-insure employee health care generally hire an insurance company to administer billing and negotiate health care provider prices, the self-insured company is the one directly paying the employee health care bills, not an insurance company, with the possible exception of catastrophic health care expenses for which the self-insured company may have an insurance policy. As health care costs increase, improving employee health is becoming a financial job-to-be-done for self-insured employers.

One way for an employer to inspire an employee to follow personal optimal diet advice (and hopefully lower health care costs) is to remove disincentives. For example, one disincentive is the typically higher price of foods that are free of preservatives (e.g., freshly prepared fresh food), free of antibiotics (e.g., pasture raised meats and eggs), high in fiber (e.g., lots of fresh vegetables and nuts), and high in probiotics (e.g., yogurt and sauerkraut). To offset that disincentive, the employer could subsidize healthier food at the workplace. The employer could also offer some freshly prepared fresh food in the workplace that is just as convenient as food from vending machines.

In addition to the removal of price and convenience disincentives, the employer could also offer a financial incentive for each achieved metabolism goal. For example, the employer could reward the employee for with a health care savings account deposit. In summary, decreasing the disincentives of inconvenience and high cost while increasing financial incentives might be enough for the employee to join the employer in thinking that a personal optimal diet is a job-to-be-done.

A Privacy Policy for Diet and Metabolism Data

Information about your personal diet and metabolism would benefit a variety of organizations (e.g., food manufacturers, health care providers, insurance companies, employers, governments). However, you and your family would benefit the most. You are the owner of your diet and metabolism data and you should have total control of how it is shared … if it is shared at all. With that as the underlying goal, the proposed personal optimal diet champions the following four fundamental rights.

  • Freedom. Optimal dieters can opt-in, change course, or quit at any time. For example, if an employer offers a personal optimal diet study for three months (i.e., long enough to see health improvements and establish new eating habits), the employees may choose not to participate for several weeks, or not participate at all. In contrast, most diet studies are rigid in duration and very restrictive in the choices available to participants.
  • Efficiency. Efficiency is a right in the sense that inefficiency is a waste of time (i.e., an arbitrary detainment, albeit not necessarily by a government). Efficient testing is crucial because frequent testing is a necessary part of the solution to chronic problems, and testing must be efficient in order to be frequent. For example, testing for fasting blood glucose level while waiting for breakfast to cook can easily fit into a daily routine. In contrast, a typical visit to a doctor’s office for a daily blood test would not be efficient and would be a tedious addition to a daily routine.
  • Privacy. Optimal dieters own their diet and metabolism data. They are the sole proprietors of it. No one else has unapproved access to it. In implementation, diet and metabolism test results are stored encrypted on the optimal dieter’s smartphone. Aggregates of optionally shared diet and metabolism test results are available to other optimal dieters and administrators (e.g., an employer), but that aggregated diet and metabolism data does not include any information that can identify individual optimal dieters. Also, diet and metabolism test results are aggregated into groups with at least N optimal dieters (e.g., N=20, but the number can be larger or smaller, as decided by the optimal dieters) to help protect individuals from being identified by a unique metabolism test result (e.g., waist to hip ratio). Furthermore, an optimal dieter must opt-in to share anonymous diet and metabolism test results to peers in order to see the anonymous aggregated diet and metabolism test results of those peers who also opt-in. This “be seen to see” privacy rule is an option allowing optimal dieters to see how their diet and metabolism test results compare to their peer’s. Peer comparison can be very helpful when interpreting one’s own diet and metabolism test results.
  • Assembly. There is so much to learn about diet and metabolism, the available government funding for diet studies can cover just a small fraction of what there is to learn. Alternatively, optimal dieters are encouraged to self-assemble into special interest groups to efficiently study their particular problem. For example, a group of optimal dieters might be curious about the effects of diet and exercise on chronic high blood pressure. An easy and secure way for a special interest group to collaborate would be valuable to the group as well as to diet researchers. For context, each special interest group should include approximately the same number of people with and without the group’s specific interest. This can be accomplished by coupling the voluntary participation in one group with a mandatory participation in a randomly assigned second group. For example, in order to participate in a special interest group on chronic skin inflammation, I would also need to participate in some randomly chosen special interest group (e.g., a special interest group studying the correlation, if any, between the rate of common colds and the use of antibacterial hand soap).

Dissenting Views

The concept of finding a personal optimal diet contradicts some commonly held views, and the following half dozen dissenting views are established to the point of being cliché.

1. A calorie is a calorie.

“A calorie is a calorie” is a common phrase, even among some diet researchers. For example, in The Weight of the Nation HBO documentary, the co-director of the New York Obesity Research Center says,

“One of the myths that’s out there is that if you eat a calorie of fat it’s different than a calorie of lettuce or a calorie of candy or a calorie of pumpkin pie. For all intents and purposes, a calorie is a calorie.”

Although “a calorie is a calorie” is a chemical property truism, it does not mean that all foods have the same ability to adversely affect your metabolism. Similarly, the physical property truism “yellow is yellow” does not mean that all yellow objects have the same ability to cause a concussion. Only in the narrow context of a food’s ability to generate heat is the co-director’s statement true. However in the broader context, in addition to caloric content there are other chemical properties (e.g., toxicity) that should not be overlooked. For example, one square (10 grams) of Lindt Supreme Dark chocolate with 90% cocoa has 60 Calories, and so does 1.49 teaspoons (5.4 grams) of gasoline. Inconsistent with the claim that a calorie is a calorie, the dark chocolate is nourishing and the gasoline is not.

2. Eat less. Exercise more.

The same co-director of the same New York Obesity Research Center, later describes in The Weight of the Nation HBO documentary, “The laws of physics, as we say, relate to the regulation of body weight: energy in, energy out.” Conservation of energy is another truism. It should not be used to gloss over important questions:

  • What drives the energy imbalance?
  • What role does insulin play in fairly distributing energy to all cells instead of directing too much energy to fat cells?
  • What roles do nutrition deficiencies play in obesity?

As an anecdote, I have been on a high-fat low-carb diet since December 2014 not to lose weight but to experiment with possible ways to control inflammation. However, I have noticed that my hunger signals are more muted than they used to be. Before I started my high-fat low-carb diet, I would get a little light headed and shaky if I delayed a meal by a couple hours. Now I can entirely skip a meal without feeling light headed or shaky.

3. Pay till it hurts.

Some tests are overpriced. As part of her compelling Paying Till It Hurts series in the New York Times health section, Dr. Elizabeth Rosenthal’s The Odd Math of Medical Tests article describes how simple and inexpensive medical tests are often overused and overpriced by hospitals with the primary (and, for hospitals, perhaps necessary) goal of generating large profits from those tests.

Fortunately, there are now a variety of virtually free do-it-yourself metabolism tests (e.g., blood glucose, blood pressure, blood oxygen saturation, pulse rate, basal temperature, urine analysis, waist to hip ratio). The proposed search for a personal optimal diet focuses on just those tests that are virtually free. Of course, all cheap tests are not necessarily useful, but some cheap tests are. For example, higher blood glucose levels appear to increase the risk of dementia. As technologies advance, the number of very inexpensive metabolism tests available for very inexpensive diet studies is likely to increase.

4. Bigger is better … concerning farms.

Foods that are free of preservatives, free of antibiotics, high in fiber, and high in probiotics are typically not mass produced. Therefore, a common claim is that those foods cannot be produced at a large enough scale and at a low enough price for everyone. However, it may be that today’s large factory farms produce less nutrition (but more waste) per acre than today’s small farms. For example, a small farm can use manure as an asset for soil fertility. However, when a large farm has too many animals producing too much manure for the local soil, the manure becomes a liability to be treated in sewage lagoons. In the small and informative book The No-Nonsense Guide to World Food, Wayne Roberts writes,

“Food itself was reinvented by the 1970s cheap food revolution. The reduced cost of meals that came with more processing and convenience was made possible in part by reductions in the amount of the food dollar that went to farmers. US farmers received 37 cents on the food dollar in 1973, but less than 20 cents after 2000. The ability of processors to deliver prefab food without raising prices raises the question: is cheap food cheap despite being processed, or because it is processed? Readers of Michael Pollan’s The Omnivore’s Dilemma will appreciate that processed food is cheap precisely because it uses the multiple personalities of corn to stand in for the functions and tastes otherwise performed by more costly real foods. The more processing, the more corn, the less money spent on actual food, the cheaper the meal — that’s the economic recipe.”

During this recent 30 year decrease in the farmer’s share of the food dollar, from 37 cents to 20 cents, government policy encouraged US farmers to employ economies of scale to survive. The “get big or get out” mantra is described in the Wikipedia article on Earl Butz.

“Butz was Assistant Secretary of Agriculture in Washington, D.C., from 1954 to 1957 under President Dwight Eisenhower. In 1971, President Richard Nixon appointed Butz as Secretary of Agriculture, a position in which he continued to serve after Nixon resigned in 1974 as the result of the Watergate scandal. He was Secretary of Agriculture from 1971 to 1976 under presidents Richard Nixon and Gerald Ford. In his time heading the USDA, Butz drastically changed federal agricultural policy and reengineered many New Deal era farm support programs.
For example, he abolished a program that paid corn farmers to not plant all their land. (See Henry Wallace’s “Ever-Normal Granary”.) This program had attempted to prevent a national oversupply of corn and low corn prices. His mantra to farmers was “get big or get out,” and he urged farmers to plant commodity crops like corn “from fencerow to fencerow.” These policy shifts coincided with the rise of major agribusiness corporations, and the declining financial stability of the small family farm.”

However, the benefits of economies of scale have their limits. As shown in the above Economies of Scale Wikipedia article plot, the Long Run Average Cost (LRAC) eventually increases with increased production. For example in the context of manufacturing, the cost of transporting manufactured goods eventually outweigh the savings associated with a bigger factory producing more goods for a larger area of consumers. In the context of farming, it is a public policy flaw to encourage farmers to exceed their optimal economies of scale by allowing them to pass their diseconomies of scale to others. Again from his book The No-Nonsense Guide to World Food, Wayne Roberts writes,

“The third hidden cost of cheap food comes from repairing damage to the environment after producers ‘externalize’ the costs of proper stewardship thanks to what’s called a ‘regulatory subsidy’. When producers ‘externalize’ a cost, they get rid of a problem they cause — by polluting a waterway with pesticide run-off, for example. When governments allow farmers and companies to cut their costs in this way, it’s a regulatory subsidy. In effect it’s a hidden subsidy, since no formal grants are required, just turning blind eyes.”

5. Bigger is better … concerning food preparation.

Diseconomies of scale can also occur in food preparation. Although there are some economies of scale in preparing larger batches of food, at some point the batches become so large that they cannot be distributed and consumed quickly enough and therefore require preservatives and/or the removal of the naturally occurring fat that tends to go rancid. If preservatives negatively affect your gut microbiome, who pays for that? What about foodborne illnesses? Who pays for those?

The “CDC estimates that each year roughly 48 million people get sick from a foodborne illness, 128,000 are hospitalized, and 3,000 die.” However, sources of foodborne illnesses are currently so difficult to prove in court that foodborne illness liability insurance is relatively inexpensive (e.g., $500 per year) for large restaurants as well as small food trucks. Of course, not requiring any liability insurance at all, freshly prepared foods such as soup, yogurt and sauerkraut are safe and surprisingly easy to make in small batches at home.

6. Bigger is better … concerning diet studies.

When diet studies are based upon averages (e.g., the average metabolism), then bigger diet studies result in more accurate averages but not necessarily better diet advice. Big diet studies typically perform a broad (i.e., many people) but shallow (i.e., few metabolism measurements … perhaps just body weight) search. In contrast, when a diet study is based upon many measurements of your personal diet and metabolism, the search is narrow but deep.

Both types of searches are valuable. The narrow but deep search is personal. The broad but shallow search is communal. Each can inform the other. For example, measurements from many narrow but deep searches can be aggregated to learn more about the community. Conversely, measurements from the community can offer a good starting point for an individual’s narrow and deep searches.


Given that “the National Institutes of Health spends more than than $800 million per year on research into the causes, consequences, treatment and prevention of obesity”, the will to understand obesity is clear. Unfortunately, the way remains elusive. Martin Harwit, in his 1984 book Cosmic Discovery, describes how major advances in the field of astrophysics typically happen at the onset of new technologies, new sensors. They create abrupt leaps in the field, opening entirely new branches of investigation. Today, diet studies appear to be poised for their own abrupt advances, mainly due to technology advances that are decreasing the cost of measuring diet and metabolism. For example, smartphones (a $260 billion global market) are now more widely available than bathroom scales (a $2.2 billion global market).

New technologies are enabling the search for a personal optimal diet as proposed in this essay. The following table lists some of the differences between government-promoted dietary guidelines and a personal optimal diet. In a phrase, the difference is “better data”.

The field of image recognition is another example of a field that quickly advanced due to better data. Professor Fei-Fei Li, Director of Stanford’s Artificial Intelligence Lab and Vision Lab, and her team employed 48,940 Amazon Mechanical Turk temporary workers from 167 countries to clean, sort, and label nearly a billion candidate images, resulting in the 2009 public release of the free ImageNet database consisting of 15 million images in 22 thousand categories. Professor Li’s ImageNet, along with the use of Graphics Processing Units for faster training, enabled the recent and substantial advances in image recognition accuracy by convolutional neural networks. You can watch Professor Li describe the development of ImageNet in her TED Talk.

The need for better diet and metabolism data is widely recognized within the diet research community, as illustrated by the first six “Needs for Future Research” listed in the appendix to the recently published national dietary guidelines:

  1. Expand WWEIA (i.e., the What We Eat In America dietary survey) participation to include more respondents from race/ethnic minorities and non-U.S. born residents.
  2. Include higher proportion of older Americans as respondents in WWEIA.
  3. Increase the number of pregnant women as respondents in WWEIA.
  4. Conduct research on nutrition transitions from childhood to shed light on how and why dietary intake changes so rapidly from early childhood through pre-adolescence and adolescence, and to identify the driving forces behind dietary intake change in these age groups and what programs are most effective at maintaining positive nutrition habits established in very young children.
  5. Evaluate the effects of common variations in dietary patterns in small children on nutrient intakes.
  6. Increase the quantity and quality of food composition databases available for research.

The free and comprehensive set of labeled images called ImageNet significantly advanced the field of image recognition. Similarly, a free and comprehensive set of nutrition facts and labeled images (e.g., images of grocery receipts, restaurant receipts and inexpensive health monitoring devices such as blood glucose meters, blood pressure meters, blood oxygen saturation meters and urine test strips) can advance diet research.

Like what you read? Give Greg Sepesi a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.