Key Takeaways
- A study found that the use of AI assistants increased annual case resolutions by over 6%.
- The custom AI tool, named JudgeGPT, was deployed in 118 trial courts across Pakistan.
- Targeted training for judges improved their adoption and usage of the AI tool.
A large-scale field trial has shown that integrating artificial intelligence (AI) into Pakistan’s justice system can significantly boost case resolution. According to a research paper titled 'Courts of Tomorrow: Evidence from a Nationwide Rollout of Generative AI', published on Tuesday, the use of an AI assistant resulted in an additional 1,848 cases being resolved per year, marking a 6.3% increase over the average.
The study, conducted among roughly half of Pakistan’s trial judges and 80% of district courts, involved 1,559 judges who were randomly divided into three groups: one with access to JudgeGPT and targeted training, another with only generic technology and law training, and a control group receiving no AI tool or training.
The custom AI assistant, named JudgeGPT, is based on OpenAI’s GPT-4 family of models and was tailored for the Pakistani context. It underwent intensive beta testing at the Federal Judicial Academy before deployment in 118 courts across the country. The research paper highlights that targeted training played a crucial role in enhancing judges' adoption and usage of the tool.
The study found that judges who received AI access alongside targeted training were more likely to adopt it, use it more intensively, and continue using it over time. Their attitudes toward AI also shifted positively; they expected the tool and the targeted training to increase their productivity. The findings suggest that pairing AI with appropriate training can lead to better integration and utilization.
The researchers noted that the AI tool was most effectively implemented when paired with targeted training for judges, particularly in tasks where language models were likely to be more useful, such as text improvement. This approach helped shift usage away from more open-ended legal queries, which are costly to verify. The trial also examined whether assigning judges to the AI tool increased local case resolution and found that it did.
To measure the impact of JudgeGPT, the study utilized various methods including baseline surveys on judge’s attitudes towards generative AI, post-treatment perception surveys, and analysis of judicial opinions before and after the trial. The records from the JudgeGPT platform were also accessed to determine the types of tasks judges assigned to the tool.
The research paper concluded that although there was an increase in text classified as AI-generated in post-treatment opinions, this did not appear to come at the expense of reduced quality. The study’s authors observed that targeted training shifted its use towards more productive tasks and away from areas where responses are harder to verify.
