Leveraging Europe's data resources

Extreme Data: Tools and technics for the transfer of data and for different data formats.
Photo: M.Spiske/Unsplash

Data is what drives science and academic research: EXA4MIND, Extreme Analytics for Mining Data Spaces, is the name of a new European research project from the Horizon Europe programme that is building a platform for the use, analysis and sharing of Big, or rather, Extreme Data by the end of 2025. The Leibniz-Rechenzentrum (LRZ) is involved in EXA4MIND. The project is coordinated by its long-term research partner, the National Supercomputing Centre IT4Innovations in the Czech Republic. "EXA4MIND is preparing a number of flagship applications," explains Dr Stephan Hachinger, who heads the research data management team at the LRZ. "The project aims to harmonise and provide access to big data from heterogeneous storage, database and management systems. It enables the processing of data on European supercomputers, on systems for artificial intelligence processes and through the use of cloud services."

Although the processing of mass or extreme data is widely promoted in science and business, it poses many technical problems for companies and researchers: For example, a driver assistance system must be able to process information from various sources, such as traffic services, sensors or cameras on the car, weather, as well as cartographicand other services. However, this information is available in a variety of formats and database systems and therefore needs to be standardised and processed for supercomputers or artificial intelligence systems. EXA4MIND is developing software, tools, workflows and interfaces for this purpose. In addition, the data sets and the results generated with them should be available, accessible and reusable in accordance with the FAIR principles and, if possible, they should flow into existing European data repositories and platforms such as the European Data Spaces, EUDAT or the European Open Science Cloud (EOSC). The LRZ is responsible for the management of data in distributed storage and databases at EXA4MIND.

In developing the technologies, EXA4MIND is focusing on four economic areas: autonomous driving, digital agriculture, molecular dynamics simulations and the processing of publicly available social data. It plans to collaborate with companies that collect and process such data. "We are bringing together databases, supercomputing, European Open Research Data Management, big data and artificial intelligence to advance data processing," says Hachinger. EXA4MIND also complements the EU project LEXIS, which ended last year with the successful establishment of a data and workflow platform. LEXIS now provides the infrastructure to process data sets from research and industry with the most suitable high-performance system in Europe and to control the necessary data flows automatically. EXA4MIND focuses on huge data volumes and efficient analysis technologies, combining modern database systems with supercomputing. The LRZ is building on its experience with LEXIS and other data integration projects such as the National Research Data Infrastructure (NFDI). (vs)

Researchers and developers: The EXA4MIND team works on new tools for the work with
research data. Photo: EXA4MIND