Menu
If you have ever had to look up the details of a law, you have probably encountered complex legal texts that are difficult to understand. The sentences are long, complex and often written so long ago that the language is outdated.
In haar masterscriptie werkte Aisja Thijssen aan een oplossing om wetten gemakkelijker te begrijpen. In dit onderzoek gebruikt ze een op regels gebaseerd algoritme dat gebruik maakt van de grammaticale relaties tussen woorden in een zin en woordenlijsten om stroomdiagrammen te maken van de juridische tekst. Bovendien gebruikte ze grote taalmodellen (LLMs) om de formulering en volgorde van de verschillende gebeurtenissen in deze stroomdiagrammen te verbeteren.
Aisja has chosen to think of laws as process descriptions. A process is best described in a process model. A flowchart is a very simple representation of a process model. It shows what happens first and what happens later. Previous research says that making flowcharts of legal texts should make it easier for readers to understand the process of the law.
After evaluating multiple options, Aisja chose to use a process model extraction algorithm developed for the English language and transform it into a rule-based algorithm that performs well for the Dutch language. This rule-based algorithm uses the grammatical relationships between words in a sentence to identify events consisting of actors, actions and objects. In developing this rule-based method, she evaluated the events identified by the algorithm with experts from PNA. She then used ChatGPT to improve the flowchart created by the rule-based algorithm.
ChatGPT is a Conversational AI tool and LLM that can generate text. Through its ability to understand text and gain insights from text, it can improve the descriptions of events obtained and better identify the sequence of different events. Aisja defined three different tasks for ChatGPT and used prompt engineering to create prompts that tell ChatGPT what these tasks are. Prompt engineering is the process of creating prompts or instructions for an LLM that specify what role it should take on and what task it should perform.
After creating both a rule-based flowchart and an LLM-enriched flowchart, Aisja evaluated the usefulness of these flowcharts in understanding legal texts with experts from PNA. This evaluation measured whether the flowcharts made the text easier to understand and whether the people who used the flowcharts felt that the flowcharts made the law easier to understand. The results of these evaluations showed that the flowcharts did not make the laws easier to understand. People were not more accurate or faster in answering questions about the legal text. However, for the longest piece of legislation, which was perceived as difficult to read, people did find the LLM-enhanced flowchart useful, at least more useful than the rule-based flowchart. The study found that although flowcharts do not help people understand the law better, readers have the experience that reading and understanding the law is easier when they have a flowchart.
This research has several potential applications within PNA that will be further explored and developed in the future.
Ben je geïnteresseerd in het schrijven van je scriptie bij ons? Ben je enthousiast over de mogelijkheid om deel uit te maken van het Data Science & AI team van PNA? Lees meer over onze toekomstige projecten!