My theory slays your devil in the details

On the need for theory and the foolishness of craftsmanship

Many key points in this post were brought into sharp relief through conversations with Professor John Day of Boston University. I am very grateful for his generous contributions. His work on Recursive InterNetwork Architecture (RINA) illustrates many aspects of searching for a good explanation of the problem of networking and using that to develop a solution. All errors here are my own.

Think of the last time you heard, “that’s too theoretical”, or maybe, “I don’t have time for theory right now, I need a practical solution to this”, or even, “that may work in theory, but in practice, things are much more complicated.”

The epitome of this perspective is the old saying, “The devil’s in the details.”

I’m here to convince you that, No, details are not the favorite abode of hiding devils. And more importantly, that not only do you have time for theory, but that without it, you’re very likely doing it wrong.

As Kathy Sierra points out in her book, BADASS: Making Users Awesome, “Experts [consistently] make better choices.”

At the heart of any progress that humans make is a decision between two alternatives that results in a better outcome. Experts are people with deep understanding and knowledge in an area, and they use that to consistently produce better results.

Systematic thinking that continually refines explanations about how the world works is at the core of any field of knowledge.

We will examine what theory is, detail the role that abstraction plays, contrast that with pre-scientific “craftsmanship”, relate these to practices in the field of software development, and point to simple actions you can take immediately to build the habit of using systematic thinking to make better choices.

Why are good explanations important?

David Deutsch, in his talk, A New Way to Explain Explanation, directs us to consider starlight. How can we explain it? He notes that throughout the 100,000 years of the existence of humans, we probably always looked to the night sky and wondered about it.

In the span of 40 years, physicists had discovered the complete explanation for starlight. Not only starlight, but other ancient mysteries and unexpected phenomena were also explained.

Deutsch then asks a compelling question: Why didn’t we figure this out in 100,000 years? It certainly wasn’t for lack of thinking about it or trying to explain it because every culture we know of had myths that dominated their lives. It also wasn’t because those people had different brains than we do.

The difference, according to Deutsch, has everything to do with what makes a good explanation.

What is a theory?

When we try to make sense of the world, there are four related elements:

1. There is some phenomena that we observe, either directly or by means of an instrument like a microscope or telescope.
2. There is a model of the world that is meant to give us leverage on this phenomena. For example, we may want to predict when it will rain so we can carry an umbrella.
3. There is a cause-and-effect relationship. If it rains, and I don’t have an umbrella, I get wet. If I have an umbrella, I don’t get wet.
4. There is the accuracy of the model in describing the cause-and-effect relationships in the phenomena that we care about.

There are three critical properties of a “good” explanation.

1. It must make predictions. “It will rain tomorrow between 2pm and 4pm.”
2. It must be possible to gather evidence that could prove the prediction to be false. “Whatever happens was meant to happen” is too vague to falsify with evidence.
3. It must be hard to vary. If it can be changed to also explain evidence that would have contradicted it, the explanation is useless. (See A New Way to Explain Explanation by David Deutsch.)

When you hear “theory”, substitute the word, “explanation” instead. A theory is nothing more than a good explanation that gives us leverage, the ability to act more effectively. If an explanation is wrong, it has no use.

Theories solve problems

The nature of the universe we inhabit is that work requires energy. Living organisms persist by means of metabolic processes. They take in suitable matter, convert it to energy and structures, and emit waste. There are environments hospitable to these requirements and other environments that are hostile or deadly.

The world is a rich tapestry of opportunities to get what we want, and threats to our well-being that we want to avoid.

We need air, water, food, warmth, shelter, companionship, autonomy, enjoyment, fulfillment. All of our needs and desires, including the things we desire to avoid, represent problems to be solved.

A theory is nothing more than an explanation that can help us get more of what we want or less of what we don’t want. The explanation tells us what causes lead to what effects. If we want more or less of the effects, we have to learn how to influence the causes.

Or more simply, at the center of any theory is the problem we want to solve.

The value of a good theory is that you can apply it to make a decision that will reliably, repeatedly get you the result you want. Experts consistently make better choices.

Two experts applying a good theory cannot disagree on a particular decision unless the decision has no consequence (either choice is equally valid). Good explanations (theories) are a wet blanket on the fire of a disagreement. If good-faith disagreement is possible, the explanation is not good enough.

All models are wrong, but some are useful.
— George E. P. Box

An incorrect or incomplete explanation can “work” right up to the point that it stops working. A classic example of this is Newton’s Laws of Motion and Einstein’s Special Relativity. Newton’s laws worked! And spectacularly so. Right up until Einstein showed that they didn’t.

This is the essence of any theory. It’s a good explanation that works until it doesn’t. Then we refine it and get a better explanation that works better, until it doesn’t. The better a theory, the better an explanation it is. Which just means that it “works” for us in figuring out what to do to get what we want, assuming that what we want is possible.

An optimal solution to a problem is sometimes possible. In the case that it is possible, proving a particular solution optimal does not mean it is the only solution. It means that any other optimal solution will be equivalent to it, and that an equivalent solution will also be optimal.

What is a formal system?

A common confusion is that a theory is the same as some restrictive, inscrutable system of squiggles and marks. That the only way to have a good theory is to express it with calculus or some other math or logic.

This is a misunderstanding of both theory and formal systems.

Explanations are essential. But if we expect to be effective at checking the accuracy of an explanation, we need a tool to help.

One especially powerful tool is called a formal system.

A formal system has four properties:

1. A finite set of symbols for constructing formulas;
2. A grammar that defines how to construct well-formed formulas (wff) out of the symbols;
3. A set of axioms that are themselves wffs.
4. A set of inference rules, essentially, maps from wffs to wffs.

What we’ve just examined is a definition. Anything that has these four properties is a formal system.

The set of inference rules may preserve a property when mapping wff to wff, for example, “truth”.

A formal system gives us a tool to precisely describe an explanation of a problem. It also give us leverage, as any good tool does, on checking the accuracy of the explanation. This is called rigor.

Both precision and rigor are essential for checking the accuracy of an explanation.

Precision is not enough because, for example, we can be absolutely precise about matter traveling faster than the speed of light, but that wouldn’t be true.

Rigor without precision would make applying the explanation difficult. Saying that “light has a maximum speed in any medium” is correct, but the precise speed of light is necessary to carry out a calculation related to light.

A theory (explanation) about the world can only be disproven, it can never be proven. To prove it would require infinite knowledge. But to disprove it only requires a counter example, some piece of evidence that contradicts a prediction the theory makes.

The confusion about “proving” a theory likely results from confusing theories with formal systems. In a formal system, what we call a “proof” is a chain of inference rules, as defined above. If we start from a “true” wff and our inference rules preserve “truth”, then we cannot arrive at a “false” wff.

We prove statements in a formal system to improve our ability for precision and rigor in the theory (good explanation) that we’re concerned with. We disprove elements (statements or predictions) of a theory, forcing us to either discard the theory, or refine it to be more accurate and harder to vary.

In summary, a formal system is not the same as a theory. A theory is an explanation that must be accurate to be useful (otherwise it really doesn’t deserve to be called a theory at all). A formal system is a tool for bringing precision to a theory and checking it for accuracy.

What is an abstraction?

The word “abstract” is often used as a synonym for “theory”, or as an attribute of “theoretical”. Something that is theoretical is considered abstract, in contrast to “concrete”.

When people describe “abstractions”, it’s often in terms of removing the “details” that are considered irrelevant.

Which details of a problem can be removed from consideration? Only those which cannot possibly affect the solution.

There’s a difficulty with this formulation, though. How are we to determine which details cannot possibly affect the solution until we have the solution? This requires that we keep every detail in consideration until the solution is found. But even with a potential solution, we may find a better one, so we still wouldn’t be sure which details to exclude.

Instead of trying to decide which details to consider and which to exclude, we need to shift our focus to which aspects of the problem are invariant and which are variable.

An invariant is a property of a problem, or class of problems. The source of the invariant is the problem, not the solution. There are no abstract solutions.

Abstractions are invariants.

Once we understand which parts of the problem are invariant and which parts vary, we are well on our way to finding a solution, or at least the outline of a solution.

Consider F = ma. This identity represents the invariant in the relationship between force, mass, and acceleration. There are only certain values for each of F, m, and a that make the identity true.

Not only is it true that all problems have invariants, but it’s also true that all living organisms embody some knowledge of those in proportion to how critical the problem is for the organism’s survival.

This should not come as a surprise at all. Imagine some ancestor of humans swinging from branch to branch in a tree. Either it’s the case that any amount of force could be used to generate the swing needed to get from one branch to the next, or a specific amount (within bounds) was needed.

We can extend this simple insight to any number of activities. Work requires the input of energy. If any amount of energy were equivalent for a cat to chase down and capture prey, there would be no competition. We could go along this line endlessly. All of existence is an interplay of the invariants and the variable.

Humans display unlimited potential to solve not only problems that immediately confront our survival, but also an endless number of problems that appear remote and irrelevant (e.g. how big is the universe, or how old is it).

There are often hierarchical relationships between elements of a problem. For example, a building contains rooms which are made up of walls, some of which contain doors and windows. All parts of the structure must be rigid to maintain their shape, but some parts, the load-bearing walls, must support the forces placed on them to keep the structure sound and erect.

The invariant that must be satisfied for the structure to be erect extends though different parts depending on their function. A wall that is not load-bearing may be torn down without affecting the building. Likewise, a new wall can be added, provided that the weight of the materials don’t violate the invariant.

One challenge in understanding a problem is identifying how the invariant holds through various levels of a hierarchy. If the solution varies for parts of the problem that are invariant, it will fail to capture an essential aspect of the problem. Likewise, if a solution is fixed where a problem varies, it will fail as a solution where the problem is different that the fixed solution.

Holding fixed what is invariant and allowing to vary what is variable is at the very core of finding a solution to a problem.

Software itself is never abstract. When people say that a software library is too abstract, what they mean is that it requires them to consider details irrelevant to their problem. That’s not abstraction. Perhaps it’s oramentation, or clutter, or irrelevance. Ultimately, it’s the solution failing to match the problem.

The opposite of “theory” is not “practical”

The word “theoretical” is often used as if it means the opposite of “practical” or “pragmatic”.

When you hear, “that’s too theoretical” there are only two possibilities.

On the one hand, the explanation may be wrong or incomplete or misapplied. If it’s wrong, it may not be possible to improve it, and it should be discarded. If it’s incomplete, it may be possible to improve it. It could be that the theory is right, for example F = ma, but this context isn’t about force, mass, or acceleration, so the theory isn’t relevant.

On the other hand, If the theory isn’t wrong or incomplete or misapplied, then saying it is “too theoretical” is equivalent to saying it is “too explanatory”, which is absurd.

In other words, “that’s too theoretical” isn’t a useful concept. We should be asking about the theory any time we are making a decision. We should be critical of how good the theory is. But we shouldn’t accept making important decisions without at least the awareness that a theory is essential, even if we lack one at that point in time.

The idea that “theory” is the opposite of “practical” is the basis of “the devil’s in the details.” What this is saying is that the theory doesn’t explain the problem well enough to be useful to solve it. But nothing other than a better theory will be useful to solve it. How could it be otherwise? How could we possibly hope to solve a problem without a good understanding of it?

There are two ways that are often proposed. One is making a “data-driven decision”. And the other is something like “craftsmanship”.

Data-driven isn’t a synonym of “theory” or “science”

Data is not the same as theory, and “being data-driven” is not the same as applying a correct explanation to find a solution to a problem.

Data itself can only provide correlations.

Newton didn’t just write up tables of data about where any planet would be at any time and then stop, directing people who wanted to understand motion to a relevant table. He formulated a theory, “laws of motion”, that describe any observed motion and any possible motion.

A table of data on the position of the planets could certainly be evidence to disprove a prediction Newton’s theory made. And the theory could be used to calculate endless amounts of data to fill in the table.

Claiming to make “data-driven” decisions without a good explanation that explains why the data is the way it is and what relationships exist is merely seeking a pseudo-scientific justification for our opinions.

More importantly, without a theory, there is no way to validate the data. All data has the problem of provenance, accuracy, and relevance. Where did it come from and how was it collected? How accurate was the mechanism that collected it? Was it actually data about the thing we’re concerned with?

Correlations may be able to point us in a direction, but they can never tell us when we should ignore the correlation. It is only a correct theory that can tell us with equal validity when to do something and when not to do something else.

When you hear someone claim to be making “data-driven” decisions, but they cannot provide the good explanation (theory) that is the basis for their decisions, recognize that they are acting like any fortune teller, or a broken clock, which will tell you the correct time twice a day, but cannot tell you why it is correct.

“Craftsmanship” is pre-scientific

Craftsmanship predated science, and is replaced by science. Now that we have science, we should not be satisfied with or tolerate craftsmanship.

Theory focuses us on the problem. Craftsmanship focuses us on our solutions.

“Craftsmen” are not incompetent. They often produce exquisite, stunning work. They build ad hoc theories (explanations) of their problem in their minds, but they never systematize it or teach the theory. They teach only application and their “rules-of-thumb”, which are essentially the same thing.

Imagine a mathematician or physicist standing before a group of their peers saying, “Look, based on my 25 years of experience, I’m telling you this is the way it works.” Such a statement would rightly be met with ridicule because truth is not dependent on the person or the personality.

The expertise of a craftsman comes either as “received wisdom”, in other words, having some element of “divine revelation”, or as a human creative act, but fully contained in the “master” and incapable of formalization and transfer to others independent of the craftsman as agent.

Science isn’t the domain of any particular individual.

Again, the issue is not the experience, just like it’s not the skills of the “craftsman”. It’s the refusal of “craftsmanship” to ground what we do in theory, a good explanation of what’s going on and why we make the decisions that we do. It’s quite literally the refusal to think about what we do in a systematic way, resorting instead to reinforcing certain actions and mimicking those.

The very idea of “craftsmanship” represents a horizon of knowledge beyond which humans once could not see. It is a recent concept that knowledge itself can be formalized, expressed, transmitted, learned, and applied absent some special agent, like a god, guru, master, or “craftsman”.

The fact that we can represent the number of apples on a table with a number, like five, is a fundamental aspect of our universe. These are other fundamental aspects of our universe:

1. A number is more than just its representation, like 0b101010, 0x2a, 42, which are the same number in different radixes, and could be dots on a computer screen, marks on a paper, arrangement of an abacus, or magnetic charge on a medium like a computer’s hard disk.
2. We can compute the number that is twice the number of apples on a table.
3. We can as easily represent the number of particles in the universe as we can the number of apples on a table.

A theory is information, and we can study it like any other information. How to describe a problem, how to perform a particular task, and how to effectively teach someone how to perform a task are also all information.

Science replaces “craftsmanship”, and refusing to replace “craftsmanship” with science is to willfully adopt an anti-scientific approach to reality.

Modern software development is pre-scientific

One field with a surprising resistance to theory is modern software development. This is visible in the robust, cultish, and dangerous “software craftsmanship” movement.

It is robust in its deflection of criticism by focusing on adherence to approved practices. It is cultish in the manner that these practices depend on the high esteem of the personalities prescribing them. And it is dangerous because it, at least tacitly and often overtly, dismisses the critical role of theory in making better decisions.

One may hope that it soon takes its rightful place next to other ancient classics like “The Art of Interstellar Travel and Rocketships”, “The Craft of Silicon Wafer Fabrication”, “Heart, Brain, and Lung Surgery for the Novice Craftsman”, and “The Art of Skyscrapers: An Illustrated Practical Guide For Building Without Physics or Calculus” (Chapter 1: Let’s build a pyramid!).

This is a common loop in software development:

1. The “senior” tech people, when confronted with a less experienced colleague, will endlessly admonish them about “YAGNI” (you ain’t gonna need it), “over-engineering”, “too much abstraction”, “ivory tower thinking”.
2. Then the senior people will enforce a process of nitpicking focus on “style guides” and “code reviews” of requests to change the code (PRs in Gitlandia) to ensure that their favorite collection of ad hoc practices are adhered to and their pet peeves are well known.
3. Finally, when something bad happens (a system fails, for instance, improperly disclosing customers’ private data), the senior people will throw their hands up and disavow any responsibility. “It’s all so complex anyway, no one could really know if this way would be better or not. We just need to keep trying things. Fail fast.”

And then around this ridiculous loop we’ll go again, and again.

This leads less experienced people to feel bewildered and frustrated and to doubt themselves. Very soon, though, the less experienced people will start parroting the more senior people, “YAGNI, DRY, too abstract, too theoretical, use this code style, do it this SOLID way…”

Worse, and sadly, that lack-of-responsibility drug is intoxicating, the less experienced people will quickly internalize it. The cycle is primed and repeats.

A common reaction to criticism of these “craftsmanship” or “rules-of-thumb” approaches is that “it depends” and “there’s no one size fits all” approach.

This is nothing more than the “it’s too theoretical” line above in disguise. A failure to focus on the problem, and instead focusing only on the solution, is at the root of these deflections. And in them are mixed up three false, and partially contradictory assertions.

The first is that some undefined, yet indispensable, characteristic of “code quality” is not only correlated with “correctness” (which also lacks a robust definition), but is actually a fundamental cause of correctness. Likewise, any assertion of the value of disposable code is equivalent to a sloppy disregard for correctness.

The second is, approximately, that testing equals correctness. One difficulty here is that if testing did equal correctness, then any code which passed the tests would be correct, but not just any code is “quality code”, and what is “quality code” isn’t in the tests, requiring style guides, manual review, rules-of-thumb, and various “principles” (approximately, mantras or affirmations) about “clean code” or “good design”.

Another difficulty is the scope of the tests. Do they include business outcomes, customer satisfaction and safety, acceptable workload boundaries? Is the property of “correctness” possible to assert in advance? What if the system is comprised of multiple parts with separate responsible parties? Is my code “correct” if it participates in an undesirable outcome because “their code” violated some standard?

The third is that theory can’t help us decide which actions are better than others. Not just that developing a good explanation is hard, but that the very endeavor is wrong-headed. It’s impossible to accomplish. Tilting at windmills.

The “no one size fits all” (NOSFA) or “use the right tool for the job” pseudo-logic has a kernel that’s true but useless, and that is used to reach a conclusion unsupported by it. No useful theory would suggest that a single tool is always the solution to a problem because a useful theory is focused on the problem, not the solution.

The irony of the NOSFA stance is that it’s doing intellectually precisely what it criticizes in a misguided practitioner when it proclaims that “if all you have is a hammer, everything looks like a nail”.

If all you have is NOSFA, everything looks like it needs its own bespoke, practical, pragmatic, non-theoretical solution. There’s no place for applying a good theory here.

Worse, NOSFA both blames the person for not knowing how to think about what they’re trying to do and, at the same time, deprives them of learning how to think (i.e. developing a good theory) about what they’re trying to do by denigrating a theory-based exploration.

Every time you try something and it doesn’t work, there are only two possibilities. Either you tried something that’s impossible and could never work, or to some degree you didn’t know what you were doing.

Even if you were mostly right and it almost worked, the problem is that you don’t know enough. Not about the solution, but about the problem. The explanation (theory) is not good enough.

Do we really think that our software is so much more complicated than fields like math or physics that we cannot build a theory of it?

Opposite that, sometimes the claim is that the software is too simple to justify “theory”, as if using a good explanation to make better decisions is a wasteful luxury, a nice-to-have. “Stop trifling with that theory and get some real work done.”

A variant of the previous objection to theory is that theory must first be grounded in experience. This is absurd. We don’t teach physics to students by first making them go out and build ten bridges and skyscrapers so they have a good “practical” grounding as a basis to understand gravity, statics, or materials.

Both perspectives reduce to a disavowal of the importance, or even the legitimacy, of using systematic thinking in the decisions we make.

If it’s a simple, well-understood domain, then the theory should be highly developed, wide-spread, and relatively easy to teach any novice. If it’s a new or unexplored domain, then the critical importance of careful thought is even higher.

Software is always incomplete

Software expresses algorithms. It describes only how something is done, not what it is doing (although sometimes possible to infer from how it is done, for example, inferring that a program is sorting based on how it is putting the elements in order), and never why it is doing it.

In other words, software code does not tell you what you want to accomplish, nor why you want to accomplish it. It can only tell you how to accomplish it.

Consequently, much of traditional “computer science”, which does have an emphasis on theory, is not applicable to the work of software construction.

Both the work of constructing software, and the use of the software constructed, involves information processing. Our modern world and much of our science has been deeply involved in understanding matter and manufacturing. However, matter and information are profoundly different, and properties, problems, mechanisms, and processes for working with them are profoundly different as well.

The challenge of developing a good explanation of software construction should not be underestimated. However, it should not be considered optional either. It is of vital, urgent importance.

What is progress?

The reason to pursue good explanations (theories) about the world is to improve our well-being, and to progress from less effective means of doing so to more effective ones.

In the introduction to The Beginning of Infinity, David Deutsch writes:

Progress that is both rapid enough to be noticed and stable enough to continue over many generations has been achieved only once in the history of our species. It began at approximately the time of the scientific revolution, and is still underway. It has included improvements not only in scientific understanding, but also in technology, political institutions, moral values, art, and every aspect of human welfare.

Science is a very recent activity that humans have discovered and it is a giant leap forward in how we understand the universe. There is nothing that stands outside our ability to apply science to understand the world.

Yet, the immense hold of the pre-scientific continues to assert a fundamental duality about the world, that science is only about certain parts, and other parts forever stand outside science and cannot be explained by it.

Deutsch continues:

Whenever there has been progress, there have been influential thinkers who denied that it was genuine, that it was desirable, or even that the concept was meaningful. They should have known better. There is indeed an objective difference between a false explanation and a true one, between a chronic failure to solve a problem and solving it, and also between wrong and right, ugly and beautiful, suffering and its alleviation — and thus between stagnation and progress in the fullest sense.

Most of us who inhabit the modern world and use artifacts from that world accept, at least tacitly, that science is responsible. No one seriously suggests that we should study ancient writings to find secrets to accelerate the clock speed of our CPUs or develop new materials for space exploration.

Software, though, that’s somehow beyond the reach of science and instead needs the “apprenticeship” model of medieval Europe pursuing “craftsmanship”, or the approach of the artisans of ancient Chinese civilization.

This view does not withstand scrutiny, however, and should with great haste be relegated to the bins of history with other pre-scientific ideas and practices.

The view that Deutsch presents is breathtaking. He intimately links together everything from the merest mundane facts to humans to the cosmos:

In this book I argue that all progress, both theoretical and practical, has resulted from a single human activity: the quest for what I call good explanations. Though this quest is uniquely human, its effectiveness is also a fundamental fact about reality at the most impersonal, cosmic level — namely that it conforms to universal laws of nature that are indeed good explanations. This simple relationship between the cosmic and the human is a hint of a central role of people in the cosmic scheme of things.

Practical rules for thinking

Numerous times a day we are presented with an opportunity to make a choice.

This is one of those. We can choose to be a person who develops the ability to consistently make better choices, or we can choose not to be.

Your decision rests on whether or not you make theory your intimate, dear friend and constant companion.

1. Re-read this post and be sure the ideas about theory / explanations, formal systems, abstractions, problems, and solutions are clear.
2. Focus on the problem, not your solution.
3. Repeatedly ask, “What do I not understand?” (about the problem, not your solution).
4. Attempt to explain the problem, not your solution.
5. Apply the criteria for good explanations. Make predictions about the problem, identify evidence (in the problem, not your solution!), make the explanation hard to vary.
6. Seek criticism of the explanation, and seek to understand it. Criticism itself is an attempted explanation. It may be useless. Often criticism is nothing more than, “I refute that based on my opinions.” Perfect, stand by while I round-file that. (See Ray Dalio’s 30 minute video about Principles.)

Here’s what you can do right now. Go to something you’ve been working on and look at the problem instead of your solution.

The most important idea in this post is to understand the problem. The invariants (abstractions) are in the problem. The theory is about the problem. Your solution is never more than a rough approximation.

All progress comes from improving the explanation (theory) of the problem. If you get stuck, go back to the problem. If you think the devil’s in the details, you’ve lost sight of the problem, you don’t have a good explanation.

Selected References

Bazerman, M., The Power of Noticing, Simon & Schuster, 2014

Brown, P., Roediger III, H., McDaniel, M., Make it Stick, Harvard University Press, 2014

Day, J., Patterns in Network Architecture, Prentice Hall, 2008

Deutsch, D., The Fabric of Reality, Penquin, 1996

Deutsch, D., The Beginning of Infinity, Viking Penguin, 2011

Fleck, L., Genesis and Development of a Scientific Fact, University of Chicago Press, 1979 (Originally published as Entstehung und Entwicklung einer wissenschaftlichen Tatsache: Einführung in die Lehre vom Denkstil und
Denkkollektiv, Benno Schwabe & Co., Basel, Switzerland, 1935)

Penrose, R., Fashion, Faith, and Fantasy in the New Physics of the Universe, Princeton University Press, 2016

Riel, J., Roger, M., Creating Great Choices, Harvard Business School Publishing, 2017

Smolin, L., The Trouble with Physics, First Mariner Books, 2007