Project image: © Ilya Pavlov on Unsplash

License-Aware Web Crawling for Open Search AI

Why we do it

The LAW4OSAI (License-Aware Web Crawling for Open Search AI) project is driven by the need to bridge the gap between legal expertise and technological advancements in web search. With the proliferation of digital content, it’s crucial to ensure that web crawling and search technologies respect the rights of content creators. By enabling license-aware crawling of web content, we aim to promote a more transparent and legally compliant approach to web search. This project seeks to empower both content creators and users by providing tools and insights that facilitate informed decision-making regarding the use and dissemination of digital content.

The project is a collaboration between the University of Twente, the Liquid Legal Institute, and fingolex. LAW4OSAI is part of the OpenWebSearch.EU community.

University of Twente


The project has received funding from the European Union’s Horizon research and innovation programme under grant agreement No 101070014 OpenWebSearch.EU project within its Cascading Funding.

Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union, granting authority. Neither the European Union nor the granting authority can be held responsible for them.

How we do it

To achieve our objectives, we foster interdisciplinary collaboration between legal experts specializing in copyright and technology law, and technical professionals in the open web search community. Through this collaboration, we develop innovative tools and methodologies for automatically identifying and retrieving content licenses during web crawling processes.

By integrating legal insights into web crawling algorithms, we aim to enhance the accuracy and efficiency of license detection mechanisms. Additionally, we provide guidance and resources to facilitate compliance with European laws and support the rights of content creators in the digital landscape.


Daniel Braun

Daniel Braun


Bernhard Waltl


Evgeny Ioffe


Baltasar Cevc

Isabelle Creutzburg

Isabelle Creutzburg

Liquid Legal Summit 2024

Current issues such as decreasing enrollments, a lack of essential skills, inefficient processes, and digital disparities in justice indicate a deep-rooted crisis. Accelerated by AI developments, these problems particularly impact copyright and regulatory aspects of the legal sector. These topics will be discussed at the Liquid Legal Summit on July 4 at the Kameha Grand Hotel in Bonn, focusing on joint efforts to tackle these challenges.