Smart LEOS – Which new functionalities should be implemented next? And what can be learnt from corrigenda?

You probably know about LEOS – Legislation Editing Open Software - and hopefully you, as we here in the Commission, have come to love it. LEOS already does a pretty good job – it takes care of unproductive work (it keeps references right, ensures that the structure is correct, …) and allows one to concentrate on the essence of one’s job, i.e., drafting legislation and developing policies. It also enables an easy and intuitive way of close cooperating in a team – and moreover all is configurable. The good news does not stop here, there is more to come, and you can help us to set priorities.

But first a few of our views. We strongly believe that technology, especially IT, should be at the service of the user. Also, as resources are scarce, we consider the use of proven though innovative technology, i.e., what is feasible now or soon. Strictly speaking this is not a limitation, state of the art technology can do a lot, and the fact that we fully embrace an open-source approach helps. So much functionality is readily available as open-source and working together allows us to progress faster. Yes, we are strongly committed to cooperation – co-design, co-development and co-deployment. More on how to do this in concrete terms will come in a next blog.

On 16 June 2022 we published the final report of a study on ‘Drafting legislation in the era of Artificial Intelligence and Digitisation’ see here. One of the main recommendations in the report is to ‘develop an IT eco system with a smart LEOS at its core’. On smart LEOS, the report identified the following new functionalities:

• context-aware legal verification – accurate usage of citations (their validity and relevance), references exist and are current, consistency of definitions, correct use of specific lexicon, acronyms and other abbreviations;
• fine granular change tracking - comparison of documents, modifications, change tracking, revision features, fork and merge of versions;
• linguistics support - use correct linguistic formulations within the structure of the document, correct formulation in accordance with English Style Guides, detect divergences between different linguistic translations, suggest linguistic formulations in provisions;
• ‘automatic’ legal drafting - e.g. for drafting transitional measures, on regulatory reporting requirements or construct the consolidation text applying amendments;
• legal assistance – within the act and between acts (e.g., detect and avoid structures that could create unintended ambiguities, correlation between recitals and the enacting terms, linkages between preceding acts and the one being drafted, identification of incompatibilities in temporal parameters, detection of explicit or implied obligations, highlight rights, permissions or penalties);
• input on policy dimensions - e.g. estimate the impact of a legislative act, measure digital-readiness or contribution to gender-equality strategy;
• advanced visualisation - IT allows the smart visualisation of legislative content to help the comprehension, position and standing of a legal act. It also allows to represent the context of legal acts by clustering acts or visualisation of the framework in which an act operates and of its consolidation; and
• discovery of legal drafting practices – mainly using AI, data analytics algorithm to detect patterns, good practices, and e.g. common errors.

To illustrate the power of modern technology in the context of the latter see the use case of corrigenda [1]. The use case, which is part of the above referred study, examined a large dataset of corrigenda of regulations to detect patterns that could be avoided in legal drafting. In fact, 2.513 documents, 3.478 pairs of modifier and modified text were examined.  This concerned 87.906 words in the original text and 100.416 words in the corrigenda. On average each correcting document contained 1,81 modifications, but corrigenda with up to 77 instructions of modifications were found. The methodology involved the following main steps:  

• Preliminary light-taxonomy of Corrigenda,
• Conversion in Akoma Ntoso,
• Classification of Corrigenda,
• Clustering of Corrigenda,
• Distance of text calculation,
• Data Analytics,
• Evaluation, and
• Legal interpretation.

The results are that

• corrigenda contain too much text which is a source of new errors, and which makes it difficult to detect the changes
•  the use case detected intense periods of modifications between 2004 and 2009;  
•  the work underscored the limitations of unsupervised Machine Learning (ML); and
•  confirmed the hypothesis that a supervised hybrid architecture helps in arriving at explainable AI improving transparency.

The learnings in the use case are useful to avoid corrigenda in the future, and should, corrigenda occur, ways to reduce the time, complexity and costs of publishing corrigenda. A next blog will report on the other three use cases piloted in the study.

In the meantime, your comments on the identified smart functionalities will be most welcome. You can comment below or contact us at DIGIT-LEOS-FEEDBACK@ec.europa.eu. Looking forward to hearing from you.

[1] Corrigenda are a special modification to correct an error that occurred in the official publication process. Since under theory of law this is a material, though not substantial, error, corrigenda have immediate effect on the legislative act. Corrigenda involves directives, regulations and decisions. The aim of the use case is to automatically isolate the portion of the text involved at a ‘fine granularity’ level, to understand the legal impact of the modification (e.g., temporal modification or other) and to evaluate ‘why which type of corrigenda’ are frequent.