Decision Making, Lawyers, And Lessons Not Learned
Why lawyers persist in making data-light decisions
B y now, the story of Daniel Kahneman and Amos Tversky and their ground-breaking work in behavioral economics is one every lawyer should know. In addition to Kahneman’s Nobel prize in Economic Sciences, we have his book Thinking, Fast and Slow. More recently, we have Michael Lewis’ book, The Undoing Project, about the history of Kahneman’s and Tversky’s work, how they came to collaborate, and the story of their joint development of the ground-breaking ideas.
Cass Sunstein’s (Harvard law professor) and Richard Thaler’s (Nobel laureate in Economic Sciences “for his pioneering work in establishing that people are predictably irrational”) work also should have caused lawyers to take a hard look at how they make decisions. Sunstein’s and Thaler’s book, Nudge, is another accessible and highly informative book in the area.
Still, it is my suspicion that lawyers, who make decisions every day, have ignored the lessons of these ground-breaking thinkers. From what I have seen, lawyers make decisions based on almost no data and gut hunches (usually referred to as “experience”).
Is this simply a case of lawyers acting “predictably irrational”? Or, is there something more fundamental at work? Why is it that lawyers ignore what their clients know — that lawyers’ decision making process is seriously flawed. Lawyers have historically relied on personal experience as the basis for giving advice. How many of us have said or heard, “in my experience,” followed by a prediction about the matter at hand. That prediction may be the likelihood of success in a lawsuit, the chances of reaching a deal, or the reasonableness of a course of action. But the advice has been limited to exactly that — the lawyer’s experience.
Let’s consider for a moment what that might mean. A 60 year old lawyer has practiced for perhaps 35 years. Assume that lawyer handled a similar matter once a month for each of those 35 years (an improbable assumption if we are talking about significant matters, but go with it for a moment). That would mean the lawyer had experience with 420 similar matters. We know that number is highly improbable, but even if it was accurate it would represent a small fraction of the total number of similar matters handled by lawyers across the US (and possibly even within her firm). For example, one lawyer’s experience with sex discrimination charges pales when compared to the number of charges brought in the US (the EEOC reported 25,605 charges based on sex discrimination were filed in 2017).
Instead of seeking out information on those charges (something admittedly difficult to do today), the average lawyer bases his or her advice and decision making on their personal experience. In addition to being an unrepresentative statistical sample, that dataset probably includes many types of bias in the handling and outcome of the charges. In other words, it is totally inadequate for the client’s purposes. Yet, the average lawyer does not look beyond that dataset (occasionally, a lawyer might informally seek input from other lawyers in his or her firm).
Modern legal technology tools are starting to make the broader dataset comparison easier. But even so, most lawyers are unfamiliar with the tools and few use them when they are available. This leaves clients in the awkward position of getting inadequate advice based on small, biased data samples. Certainly not the place where most clients want to be. Clients are aware of the problem and raise concerns about the quality of advice and decision making they receive from their lawyers. Still, lawyers soldier on ignoring data in favor of “in my experience.”
What We Can Do To Improve Decision Making
The fix for this problem involves many parts. Fortunately, none of them are complicated or difficult to do. They all require lawyers (practicing lawyers, judges, professors) to keep open minds. The science of decision making has progressed quite a bit over the past 30 years, but the “science” of legal decision making has hardly budged. It is time we updated the legal model to agree with what we know about decision making and that will require many of us to change how we do things.
First, as with many problems in the legal industry, the best place to start is with legal education. Prospective lawyers should spend time studying how humans make decisions and the many ways we can distort or defeat good decision making. For example, understanding the work of Kahneman and Tversky would be a good start. Follow that with the work of Thaler and Sunstein, and you would have an excellent grounding in modern decision making. To that, we add some basic knowledge of data and statistics. Already, you can see how a course in legal decision making could significantly improve the practice of law.
Second, those lawyers who have graduated from law school should follow a similar path as law students. States that have mandatory continuing legal education could add classes on decision making. Making these classes a mandatory part of the CLE process (as ethics is today) would help. For states without mandatory CLE, law schools could offer one- or two-day workshops on the topic. We won’t get all practicing lawyers to participate, but even getting a few would be an improvement over where we are today.
Third, judges should get exposed to modern decision making ideas. Here the judicial conferences they attend could help. Our judiciary is well-behind the curve when it comes to understanding statistics and how it can (and should) inform judicial decisions. Many studies have shown how judges do not perform as well at decision making as simple algorithms. While we still want people deciding the fate of people, it would help all of us if the judicial decision making process was more consistent and even-handed.
What About Decision Making Algorithms
There is a fourth solution, though one which most lawyers will find both surprising and frightening. The fourth approach is to use simple algorithms to guide lawyers in their decision making. Kahneman and others have shown that using simple algorithms can result in better and more consistent decision making than humans achieve on their own. A lawyer employing such an algorithm could use it to guide his or her decision making. They would not have to slavishly follow the algorithm’s results — there could be circumstances known to the lawyer that the algorithm did not take into account — but they would want to have good reasons for deviating from those results.
We can imagine how this would work. A client goes to a lawyer and asks about its chances in a lawsuit. The lawyer queries the dataset of similar lawsuits to extract basic information, such as the frequency of defendants prevailing in such lawsuits. Then, the lawyer plugs the key variables from her client’s situation into an algorithm that was built using data from the dataset of similar lawsuits. The algorithm gives the odds of the client winning. The lawyer, armed with both pieces of information, advises the client. The lawyer takes into account information not included in the algorithm that is atypical for such lawsuits. But, the client has the benefit of a much deeper dataset than the lawyer’s “in my experience” dataset.
Although lawyers have practiced law for centuries, the profession of lawyering is an immature art and science. Lawyers have ignored aggregate information and what it tells them in favor of small datasets that are meaningless. They have gotten away with this approach, because it is difficult to follow through and hold them accountable for poor decision making. Would the client have won if the case went to trial? Was that provision really “market”? Does the policy accurately reflect what others are doing? These and other similar questions have been difficult to answer, because the data has been difficult to gather.
Legal data still is difficult to gather, to cleanse, and to put in useable form. But that is changing rapidly. What passes today as “knowledge management” in most firms just scratches the surface of what could be done. Combine that data with the growing body of public datasets, and the world of decision making gets very interesting. Each firm could develop a competitive edge — how it handles Type X lawsuits and its success rate compared to the general population. Firms don’t seem to grasp that data has already become the differentiator. No lawyer holds a monopoly on the way to handle a matter, but data can give every lawyer an advantage. Decision making informed by data could make that advantage hard to beat.
Ken is an author on innovation, leadership, and on the future of people, processes, and technology in the legal industry. He also is an adjunct professor and Research Fellow at Michigan State University’s College of Law. You can follow him on Twitter, connect with him on LinkedIn, and follow him on Facebook.
