Chat GTP-4 Could Pass the Bar Exam
How Our Technology Evolves FAST
The U.S. requires all lawyers to take what is known as the bar exam to become licensed to the bar of a U.S. state. Researchers recently put Chat GPT 3.5 through this rigorous test to see how well it performed.
Human applicants need to complete at least seven years of post-secondary education and three years at an accredited law school before taking the bar exam. It takes weeks and even months for an applicant to prepare for this test, and one in five tend to fail on the first try.
Chat GPT and the Bar Exam
Researchers at Chicago Kent College of Law, Bucerius Law School Hamburg, and the Stanford Center for Legal Informatics (CodeX) examined the Open AI GPT 3.5 model to see if it was capable of passing this difficult test. The researchers used the multistate multiple choice section of the Bar Exam.
GPT 3.5 ended up trailing humans by about 17%. In the Criminal law category, it was up to 36%. In the Torts and Evidence categories, however, Chat GPT 3.5 got a passing grade. It exceeded the baseline chance of 50% with random guessing. Without fine-tuning, according to research, it passed two bar categories. It managed to reach parity with humans on one. It exceeded the researcher's expectations.
