Reflections on AI and Prosecution Conference
I recently had the opportunity to attend and speak at the AI and Prosecution conference hosted by Prosecutor Leaders of Now (PLN) in Dallas, TX. The event brought together representatives from prosecutor offices, public defenders, community members affected by and working within the criminal justice system, technologists, and academics to discuss current and soon to be uses of AI in the criminal justice system and concerns and opportunities related to its use. The workshop culminated in a design workshop day with diverse sets of attendees working together to identify issues and design prototype solutions. Overall, the workshop was, from my perspective, a massive success in
- Educating prosecutors about the basics of AI and different machine learning techniques;
- Educating prosecutors about where these tools are being used in software that office’s already use;
- Laying out clear concerns and opportunities where AI is or will be affecting prosecution practice;
- Including groups affected by prosecution in the discussion so that prosecutors have greater awareness of a diversity of perspectives; and
- Providing a helpful framework — design and systems thinking — for practitioners to approach prototyping uses of AI.
Day 1
Divided between a morning of lectures on AI and afternoon panel discussions, I felt that the audience received plenty of reinforced knowledge regarding how machine learning works and how it’s currently being used. The presentations and panels were supplemented before the workshop by a series of suggested readings from PLN that, if read, helped to provide background for the day. My favorite reading was this paper by Brandon Garrett and Cynthia Rudin that I hope to write a future blog post about — I really appreciate the thoughtful approach the paper provides in giving consistent language that lawyers can use to understand and talk about AI. I also agree with the broad thrust of the paper to emphasize the use of “glass box” models in the criminal justice system.
I presented this slide deck as an expansion on a prior blog post to try and provide prosecutors without a foundation in AI/machine learning with a framework for identifying when software or tools might be using models and language to communicate with technologists about AI and models in software. The presentation before mine by Lucas Meyer did a great job laying out the history of AI and how it has developed up to the LLMs of today. LexisNexis presented on their platforms AI features, which was fascinating to see the many different ways they have fit generative AI into their platform. I have concerns about the extent to which the tool has been tested or validated for how it might harm people in the criminal justice space, but impressed at the prospect for attorneys in saving time on formatting and formulaic legal language generation. Finally, Brandon Garrett presented his prior work and legal challenges with the use of AI, which I found wonderfully practical and laid out that there is a lot of work to do in thinking about all the different ways AI and machine learning is affecting legal practice.
After the presentations, there were three panel discussions — one on ethics and AI, one on defense attorneys’ use of AI, and one featuring voices of persons impacted by the criminal justice system. I moderated the ethics discussion and will be writing a future post about the most interesting points that arose from that discussion.
The panel featuring defense attorneys and Justice Text surprised me. The general refrain I have become accustomed to is that many prosecutor offices are 10–20 years behind in technology and resource constrained, but that defense counsel aren’t even using technology and have next to no resources. This panel demonstrated though that at least some public defender offices are utilizing advanced technologies to cope with resource constraints. It also demonstrated that there are companies tailoring AI technologies for defense work in a way that I have not found for prosecutors — while there are general tools like automated transcription, I don’t know of technology companies outside of CMS providers that are specializing in building advanced AI tools for prosecutors. The unifying theme though from this panel that I believe all public agencies can benefit from is that to institute new technologies effectively requires:
- Leadership in adoption from both those at the top of the organization and from actual staff responsible for using the technology;
- That technology should be bought after consideration and feedback from staff on what the need is and how the technology will address the issue; and
- Where offices adopt values around organizational improvement and structures around identifying and addressing needs, change is most likely to take root and be successful.
The last panel of the day consisted of individuals (EJUSA, Kid C.A.T., Alliance for Safety and Justice, and CROP organization) directly impacted by the criminal justice system including those that had been convicted of crimes and victims of crime. While this panel did not cover the use of AI in prosecution, it framed how prosecution directly affects people and the perspectives that prosecutors should consider in their work generally as well as in adopting new technologies. I really appreciate such panels because I find in my work that the most gaping hole in prosecutor knowledge is having conversations with the people that are directly impacted by their work. Hopefully incorporating those voices does help to shape how we change and improve our criminal justice system moving forward.
Day 2
The second day consisted of short lectures on systems and design thinking followed by actual design activities with mixed groups of people — prosecutors, defense attorneys, technologists, and funders. For most, this was the first time working on problem identification and then developing a prototype solution and I hope was helpful for providing a framework to create new solutions. The group I was a part of followed a wonderfully circuitous path to a proposal for a Bureau of Prisons policy change to allow smartphones in meetings between arrested persons and defense counsel to use translating applications to facilitate early and better communication with non-English speaking people.
Landing on this proposal surprised me as the group started with trying to address the problem that prosecutors are so inundated with cases that early and rigorous review of cases to weed out legally insufficient cases is often not possible. With that framing, we initially proposed ideas for things such as using AI to review evidence and provide summarization for attorneys, setting policies for timely review of cases, or using legal tools like deferred prosecution agreements to facilitate conditional release. Then a group member proposed narrowing our project to trying to address the needs of arrested individuals that do not speak English and how these individuals are especially vulnerable to delays in legal proceedings because of communication barriers. This then led to a discussion about how such individuals can be held in jail through multiple arraignments as defense counsel await communicating with their client and being able to provide critical information to prosecutors and judges. Ultimately, this led to discussion of the policies in prisons that prevent the use of already available technology for such communication challenges and the proposal to change policy and the development of a simple application that builds on existing software to make the system more just.
Conclusion
I really enjoyed the workshop and hope that practicing attorneys and funders came away with ideas for new projects and a framework for evaluating and adopting technologies to improve the criminal justice system. At Justice Innovation Lab, we work with prosecutor offices interested in improving through deliberate problem identification, iterative design and rigorous evaluation so the theme of the workshop fit well with our organizational goals. We also work at the intersection of technology and policy in this space and find that while creating the tools is certainly challenging the time, creativity, adoption, and ethical policy change required for these projects are the most significant barriers. I hope that offices that attended the sessions were inspired by some of the ideas and that they might be partners in the future.
By: Rory Pulvino, Justice Innovation Lab Director of Analytics. Admin for a Prosecutor Analytics discussion group.
For more information about Justice Innovation Lab, visit www.JusticeInnovationLab.org.
