An open search engine for Europe
Open Search Foundation works on an open navigation through the web
with the help of LRZ, data centres and enterprises. Photo: Unsplash
These numbers make you think: about 77 million people in Germany search for information on the Internet every day at Google. The search engine has a market share of more than 90 percent in Germany. This is similar all over Europe and worldwide. Its closest competitor, Microsoft's Bing, reaches under 10 percent of internet users in Germany and just under 3 percent worldwide. Consequently, a single company provides orientation on the net. With its secret algorithms, it decides which information offers are listed where in the ranking of results. "The algorithms and data pools of major search engines are opaque boxes, we don't know which filters lead to the results we are presented with, and we also don't know what the companies do with our data," states Stefan Voigt, PhD in geography and board member of the Open Search Foundation (OSF), and demands: "The internet must be mapped openly."
Doing sustainable maths together
The non-profit organisation, which started in 2017 and is now supported by universities, research institutions, companies and users in Germany and Europe, is working on an open search index for the internet. The Leibniz Supercomputing Centre (LRZ) is also involved. "An important partner for the Open Search Foundation", as Christine Plote, founding member of the OSF, reports: "The computer resources of LRZ are already working on the open index for the internet. The best solution against the dominance of existing search engines is diversity."
This is to be created with the new search index. It makes transparent which information from the web it takes in and how result lists are created. With this map for the internet, alternatives to Google & Co, innovative services and business models could be developed. "The Open Search Foundation opened an open door for us with this idea," says Prof. Dr. Dieter Kranzlmüller, Director of the LRZ. "We are committed to open science and prefer to base our services on open source software, so Open Search fits perfectly into our concept. The internet was built as an open communication infrastructure, this openness must be maintained for core tasks like search." Like Kranzlmüller, some LRZ employees are personally involved in OSF and tinker with concepts or technologies.
Work is already underway on the technology for the Internet's open index. The first search agents or crawlers have been developed, and plans are in place for infrastructures to manage archives or for storage. Instead of huge server farms like those maintained by search engine operators, which consume a lot of electricity, the OSF trusts in commitment, cooperation and decentralisation. The LRZ as well as other data centres, companies and organisations make temporarily unused computer capacities available: "We work distributed and together in European data centres with an opensource code," explains Voigt.
Developing interdisciplinary search techniques
This commitment is catching on: OSF is still a grassroots movement, but one that is making itself heard internationally. Thanks to the contacts of fellow campaigners, it is spreading far and wide, especially in science and research, in Europe and beyond. Conferences and workshops on issues related to computer-assisted knowledge discovery are spreading the idea of open search and at the same time tasks on technology. The first international Open Search Symposium took place at LRZ in 2019, and the third will be held digitally at CERN in Switzerland from 11 to 13 October 2021. Here they will discuss new search technologies, speak about new plans and delegate work to volunteers and acivists.
The cooperation between OSF and LRZ is now bearing even more fruit, bringing the topic into European research programmes: A broad, interdisciplinary project is already planned at Horizon 2020 to promote the development of digital search technologies according to social and ethical criteria and to provide the building blocks for an open search in Europe. (vs)