Generative AI Scorecard

Why we act

Progress in the development of so-called generative AI with applications such as ChatGPT has reached a point where there no longer is any doubt that artificial intelligence (AI) will become an integral part of legal work. AI will be used as an assistance system, or as an autonomous solution for answering simple questions, in law firms and legal departments as well as in courts, for administrative authorities, and probably also in the context of lawmaking.

For lawyers working with AI, it will be essential to know the strengths and weaknesses of the systems to be able to assess the opportunities and risks associated with its use correctly. An objective and standardized assessment requires a uniform catalog of criteria and a uniform scale of values applicable to the different systems. The results generated by the system will be assessed, and the goal of the assessment is to cover different abilities of the system (including descriptive, design, systematic, and analytical abilities).

Jens Wagner

What do we propose

The Liquid Legal Institute e.V. develops a scorecard that allows for an objective and standardized assessment of the strengths and weaknesses as well as the opportunities and risks of ChatGPT and other forms of generative AI for use in the field of law.

The scorecard should perform the following functions:

  1. objective assessment of existing AI systems;
  2. creation of awareness regarding the capabilities of the AI systems used;
  3. benchmark in the further development and training of AI systems for the field of law.

Liquid Legal Summit 2024

Current issues such as decreasing enrollments, a lack of essential skills, inefficient processes, and digital disparities in justice indicate a deep-rooted crisis. Accelerated by AI developments, these problems particularly impact copyright and regulatory aspects of the legal sector. These topics will be discussed at the Liquid Legal Summit on July 4 at the Kameha Grand Hotel in Bonn, focusing on joint efforts to tackle these challenges.