Knowledge extraction from multimedia – Multimodal Data Fusion and Analytics Group

TEXTAILES

TEXTAILES TEXTile digitisAtIon tooLs and mEthodS for cultural heritage The TEXTaiLES project aims to redefine the digitisation of Cultural Heritage (CH) textile objects and the development of AI-based processing tools, establishing a standardised protocol seamlessly integrated into the European Collaborative Cloud for Cultural Heritage (ECCCH). By studying past and present practices in textile digitisation, TEXTaiLES methodically explores their strengths and limitations. The project will develop advanced tools that address the entire digitisation lifecycle of textile objects, focusing on non-destructive methods. Using robotic, cost-effective multi-sensor solutions, TEXTaiLES ensures the preservation of artefacts during digitisation, complemented by reliable data analysis workflows. The project integrates state-of-the-art AI technologies to illuminate the intricate details of textiles. From revealing their weaving patterns and microstructures to identifying degradation factors, the AI-driven toolkit offers in-depth insights. It…

MELIORA

MELIORA Multimodal Engagement and sustainable Lifestyle Interventions Optimizing breast cancer Risk reduction supported by Artificial intelligence MELIORA Consortium will jointly work with local actors at different levels for the realisation of eight (8) in total tailor-made lifestyle intervention and behavioural modification studies, including Artificial intelligence and digital tools, to promote sustainable behavioural changes among three target populations: i) healthy women at risk of developing breast cancer, ii) breast cancer patients and iii) breast cancer survivors. The studies will take place across 4 different European countries (Greece, Sweden, Spain and Lithuania) and 6 piloting centres targeting 2000 women in urban and rural areas including participants throughout the socioeconomic spectrum. Primary health care centres, hospitals or patients’/survivors’ organisations will be used as entry points to the community. The goal of the MELIORA…

CONNECTOR

CONNECTOR CustOms exteNded iNteroperablE Common informaTiOn shaRing environment CONNECTOR aims to support the European Integrated Border Management (EIBM) and the EU Customs Action Plan by emphasising the importance of collaborative efforts among Customs, Border and Coast Guard Authorities in managing borders, especially under current and future challenging circumstances. This objective can be achieved through Customs participation in the Common Information Sharing Environment (CISE) network and Enhanced Common Information Sharing Environment (e-CISE) via the proposed Customs Extended Common Information Sharing Environment (CE-CISE). CONNECTOR proposes an integrated, common, and shared risk assessment approach for all Border Management Authorities. This approach considers pan-EU common risk indicators for each end user group, including Customs, Border and Coast Guards Authorities, as well as FRONTEX. Its aim is to safeguard the external EU border, protect EU…

PRECRISIS

PRECRISIS PRotECting public spaces thRough Integrated Smarter Innovative Security PRECRISIS aims to strengthen the protection of public spaces by preventing terrorist attacks and violent crimes and mitigating their impacts. It will promote integrated smarter innovative security through the empowerment of law enforcement authorities, first responders, local managers of public spaces and related security stakeholders, enhancing public-private cooperation. PRECRISIS will develop, test, and roll out innovative, sustainable, and effective AI-based digital solutions and practical toolkits, in full compliance with ethical standards and related legislation on the protection of personal data, by integrating existing expertise, best practices and privacy-by-design approaches to make public spaces more secure. Technological innovation delivered by the PRECRISIS Platform will provide identification of potential vulnerabilities connected to public spaces and a real-time simulation of plausible scenarios before, during…

ALLIES

ALLIES AI-based framework for supporting micro (and small) HSPs on the report and removaL of onLIne tErroriSt content The ALLIES project aims at supporting micro (and small) HSPs in achieving compliance with the new requirements and obligations under the TCO Regulation. To achieve this, ALLIES adopts four main approaches, each targeting skill development and innovation. These four pillars will be respectively focused on (1) learning and awareness raising, (2) technical development and adaptation, (3) training and education, and (4) experience sharing and reporting mechanisms. Both the learning and awareness raising initiatives will further expand the capacity of the TCO Regulation by designing a taxonomy of online terrorist related behaviours. The second pillar will be focused on the development of a specific set of AI-boosted user-friendly tools that will enhance small…

CESAGRAM

CESAGRAM Towards a Comprehensive European Strategy Against tech-facilitated GRooming And Missing Child sexual abuse online is a growing phenomenon. The British National Society for the Prevention of Cruelty to Children (NSPCC) reported a 70% increase in the number of sexual communications with child offences in the last three years. The hotlines for missing children have reported concerns that grooming is an increasing factor in disappearances, particularly in cases of children who run away. In 2020, the second most common reason for children to contact Child Helplines was for reasons of violence of which 3,2% was online sexual abuse. Disappointingly, the COVID pandemic has exacerbated the problem. The COVID-19 crisis pushed 1.2 billion children out of schools, leading to children spending many more hours online. As Europol emphasised in its 2021…

FaRADAI

FaRADAI Frugal and Robust AI for Defence Advanced Intelligence An important crosscutting need for Artificial Intelligence is to create technologies for trustworthy autonomous and frugal learning, i.e. the ability of a system to adapt and learn from its environment, including from user supervision, for a reasonable cost and without intervention from expert developers nor regression. Such technologies can be highly disruptive and have high impacts for many capabilities, especially when the information to manage is highly variable or unpredictable and high adaptability is needed. These technologies can also alleviate the current need to provide data to the system developers to get improvements depending on such data, which can be critical when the data is confidential, and is thus critical for defence. They can more generally enhance technological independence. Selected actions…

SUN

SUN Social and hUman ceNtered XR The SUN Horizon Europe project aims at investigating and developing extended reality (XR) solutions that integrate the physical and the virtual world in a convincing way, from a human and social perspective. The virtual world will be a means to augment the physical world with new opportunities for social and human interaction. Relevant limitations that will be addressed are: • Lack of solutions to develop scalable and cost-effective new XR applications. • Lack of convincing solutions for mixing of the virtual and physical environment. • Lack of plausible and convincing human interaction interfaces in XR. • Barriers due to resource limitations of end-user devices. The SUN project develops solutions to surpass the aforementioned limitations, including: • Scalable solutions to obtain plausible and convincing virtual…

CTC

CTC Cut The Cord With the nature of terror incidents undergoing a technology-aided transformation (e.g. use of cryptocurrencies to enhance anonymity), the monitoring and analysis of financial transactions play a pivotal role in the effort to timely prevent actions and funding of global terrorism. Financial crime exploits every aspect of digital transformation and identifies non-traditional products for financing and supporting terrorist organisations, such as social media, cryptocurrencies, and other Financial Technologies. The prevention and disruption of terrorism financing requires international cooperation and advanced investigative techniques based on new technologies. The CTC project (Cut The Cord) aims to enhance the public-private cooperation through the establishment of a wide stakeholder community, with the vision to sustain it beyond the end of the project. Joint training exercises (theoretical and hands-on) will be organised…

Nestor

NESTOR NESTOR – aN Enhanced pre-frontier intelligence picture to Safeguard The EurOpean boRders NESTOR aims to demonstrate a fully functional next generation holistic border surveillance system providing pre-frontier situational awareness beyond maritime and land border following the concept of the European Integrated Border Management. NESTOR long-range, wide area surveillance capabilities for detection, recognition classification and tracking of moving targets (e.g. persons, vehicles, drones etc.) is based on optical, thermal imagine and Radio Frequency (RF) spectrum analysis fed by an interoperable long range IP-based network of sensors including stationary installations and mobile manned or unmanned aerial, water surface, underwater and ground vehicles, capable of functioning both as standalone, tethered and in swarms (UAV, USV, UUV and UGV) platforms. NESTOR Border Command Control and Coordination intelligence (BC3i) system using cutting-edge Artificial Intelligence…

STARLIGHT

STARLIGHT STARLIGHT – Sustainable Autonomy and Resilience for LEAs using AI against High priority Threats STARLIGHT aims to empower LEAs with automated, operational, and cyber-resilient investigation, intelligence, surveillance and control capabilities to tackle traditional and emergent criminal activities, terrorism, cybercrime, and cyber-attacks, within and at the borders of the EU. STARLIGHT also aims at contributing to the establishment of a strong EU AI-based security ecosystem, thus enhancing the EU’s strategic autonomy in the field of AI for LEAs. The ecosystem will be built around an AI framework for reliable, accountable, responsible and transparent LEA AI solutions that will enable the involvement and transfer of knowledge between LEAs, Researchers, Industry, SMEs, and Policy makers. Overall, STARLIGHT will ensure that European LEAs lead the way in AI innovation, autonomy and resilience, addressing…

ODYSSEUS

ODYSSEUS Preventing, Countering and Investigation Terrorist Attacks Through Prognostic, Detection and Forensic Mechanisms for Explosive Precursors ODYSSEUS aims to increase the knowledge on homemade explosives (HMEs) and explosive precursors, including precursors not previously studied, and develop effective and efficient prognostic, detection, and forensic tools to improve the capabilities of Law Enforcement Agencies (LEAs) towards the prevention, countering, and investigation of terrorist incidents involving HMEs. ODYSSEUS will build upon relevant previous projects mainly HOMER, through the involvement of HOMER’s core partners in this consortium, and will thus continue the work already performed in HOMER on some precursors and also further extend it to not previously studied precursors. To discover potentially hitherto unknown information, online HMEs recipes will be collected and their content will be analysed in order to extract knowledge about…

CHROMATA

CHROMATA Dances, Songs, Myths and Customs for the Development of Technologies for Intangible Cultural Heritage The preservation of the intangible cultural heritage (e.g., traditional dances, songs, stories, techniques etc.) is a challenging task, since it is more difficult than maintaining tangible objects. The main purpose of the national project CHROMATA is to design and implement an interactive virtual reality platform that will gather elements of the intangible cultural heritage from available sources in areas of cultural interest and will constitute the starting point for the creation of indoor and outdoor environments of virtual and augmented reality that can (a) revive and represent the elements (e.g., traditional dances, stories, etc.) of the intangible heritage and b) create new spatial and virtual transcripts, thus promoting new uses for it. For the implementation…

xR4DRAMA

xR4DRAMA Extended Reality For DisasteR management And Media plAnning The EU-funded xR4DRAMA project aims to create a solution that will improve the situational awareness of those user groups who are responsible for handling disasters, man-made crises or public events. The groups range from first responders, local authorities and security forces to media companies and event planners. Situational awareness is crucial for them to execute tasks efficiently, plan things and take the appropriate decisions. The proof of concept of this project will focus on first responders and media companies before extending the use cases to other domains. xR4DRAMA's methodology is based on X Reality (XR) technologies, and the tool can be used remotely and on site for an augmented and shared perception of a given situation. In particular, innovative technologies will…

WQeMS

WQeMS Copernicus Assisted Lake Water Quality Emergency Monitoring Service Monitoring the extent of the effluents in the lake provides a warning about the risk of water contamination and contributes to impact mitigation strategies, both for the drinking water production and the environmental sustainability. WQeMS focuses its activities on the monitoring of lakes with the aim to provide an operational Water Quality Emergency Monitoring Service, which enhances our ability to deliver drinking water in all EU member states. The project exploits Sentinel satellite data for quality monitoring at a finer spatial resolution level, following validated processes with in situ data. WQeMS provides extensive information about water quality by building a multi-temporal and high spatial resolution monitoring framework. The goal is the optimization of the use of resources by gaining access to…

PathoCERT

PathoCERT Pathogen Contamination Emergency Response Technologies Pathogens are a determining factor in emergency response due to their life-threatening nature, both for the public as well as for the safety of first responders. In many cases, pathogen contaminations are difficult to detect, and require specialized technologies, tools and procedures to handle them. Waterborne pathogen contamination events can occur anywhere, and may be caused by various natural events or they can be the result of human activity, either accidental or malicious. During these emergencies, first responders may need to operate within a certain pre-defined incident area, and are likely to be exposed to contaminated water originating from various sources, such as surface water, wastewater or drinking water. This can pose a significant risk of illness, disease or even death, through skin contact,…

V4Design

V4Design Visual and textual content re-purposing FOR(4) architecture, Design and video virtual reality games V4Design promotes the re-use of visual and textual content, such as documentaries, paintings and their critiques so as to be repurposed and be used for the inspiration of architecture and video game (VR) designing. Visual content analysis is performed through 3D reconstruction and modelling, localization of objects in visual data, automated extraction of aesthetics/style information and language understanding from visual and textual data. 3D objects are also enhanced with semantics and explanatory text descriptions. V4Design will develop a platform that provides architects, video game creators and designers of any expertise with innovative tools necessary to enhance and simplify the creative phase of the designing process. The main idea behind V4Design is to reuse (i) visual: movies,…

iV&L Net

iV&L Net The European Network on Integrating Vision and Language Combining Computer Vision and Language Processing For Advanced Search, Retrieval, Annotation and Description of Visual Data, COST Action IC1307, is designed to bring together two previously unconnected research communities, Computer Vision (CV) and Natural Language Processing (NLP). It is set up to allow researchers to meet, exchange ideas, expertise and technology, and form new partnerships. Building on the successful EPSRC Network on Vision and Language (V&L Net), the aim of the COST Action is to create a lasting interdisciplinary research community situated at the nexus between language processing and computer vision. This is hugely important because of the explosive growth of visual and textual data (both on the World Wide Web and held in private repositories by diverse institutions and…

TENSOR

TENSOR reTriEval and aNalysis of heterogeneouS online content for TerrOrist activity Recognition Law Enforcement Agencies (LEAs) across Europe face today important challenges in how they identify, gather and interpret terrorist generated content online. The Dark Web presents additional challenges due to its inaccessibility and the fact that undetected material can contribute to the advancement of terrorist violence and radicalisation. LEAs also face the challenge of extracting and summarising meaningful and relevant content hidden in huge amounts of online data to inform their resource deployment and investigations. In this context, the main objective of the TENSOR project is to provide a powerful terrorism intelligence platform offering LEAs fast and reliable planning and prevention functionalities for the early detection of terrorist organised activities, radicalisation and recruitment. The platform integrates a set of…

EOPEN

EOPEN Open interoperable platform for unified access and analysis of Earth Observation data Earth Observation (EO) data access through the Copernicus data distributor systems has paved the way to monitor changes on Earth, using Sentinel data. One of the main objectives of EOPEN is to fuse Sentinel data with multiple, heterogeneous and big data sources, to improve the monitoring capabilities of the future EO downstream sector. Additionally, the involvement of mature ICT solutions in the Earth Observation sector shall address major challenges in effectively handling and disseminating Copernicus-related information to the wider user community, beyond the EU borders. EOPEN will fuse Copernicus big data content with observations from non-EO data, such as weather, environmental and social media information, aiming at interactive, real-time and user-friendly visualisations and decisions from early warning…

MULTISENSOR

MULTISENSOR Mining and Understanding of multilinguaL contenT for Intelligent Sentiment Enriched coNtext and Social Oriented inteRpretation The consumption of large amounts of multilingual and multimedia content regardless of its reliability and cross-validation can have important consequences on the society. An indicative example is the current crisis of the financial markets in Europe, which has created an extremely unstable ground for economic transactions and caused insecurity in the population. The fact that the national mass media provide exaggerated and contradictory information and the inaptness to understand local contexts from different countries has a considerable share in the aggravation of the crisis. To break this spiral, we need multilingual technologies with sentiment, social and spatiotemporal competence that are able to interpret, relate and summarise economic information and news created from various local…

beAWARE

beAWARE Enhancing decision support and management services in extreme weather climate events In every disaster and crisis, incident time is the enemy, and getting accurate information about the scope, extent, and impact of the disaster is critical to creating and orchestrating an effective disaster response and recovery effort. The main goal of beAWARE is to provide support in all the phases of an emergency incident. More specifically, we propose an integrated solution to support forecasting, early warnings, transmission and routing of the emergency data, aggregated analysis of multimodal data and management the coordination between the first responders and the authorities. Our intention is to rely on platforms, theories and methodologies that are already used for disaster forecasting and management and add the elements that are necessary to make them working…

MULTIFORESEE

MULTIFORESEE MULTI-modal Imaging of FOREnsic SciEnce Evidence The main objective of the COST Action CA16101, entitled MULTI-modal Imaging of FOREnsic SciEnce Evidence (MULTI-FORESEE)- tools for Forensic Science, is to promote innovative, multi-informative, operationally deployable and commercially exploitable imaging solutions/technology to analyse forensic evidence. Forensic evidence includes, but not limited to, fingermarks, hair, paint, biofluids, digital evidence, fibers, documents and living individuals. Imaging technologies include optical, mass spectrometric, spectroscopic, chemical, physical and digital forensic techniques complemented by expertise in IT solutions and computational modelling. Imaging technologies enable multiple physical and chemical information to be captured in one analysis, from one specimen, with information being more easily conveyed and understood for a more rapid exploitation. The enhanced value of the evidence gathered will be conducive to much more informed investigations and judicial…

MindSpaces

MindSpaces Art-driven adaptive outdoors and indoors design MindSpaces aims to create a novel approach to urban and architectural design by generating 3D-VR immersive and emotion-adaptive ‘neuro-environments’ that will help in designing emotionally-relevant urban spaces. The emotional aspects of an environment will be captured through the use of mobile EEG (Elecroencephalography) headsets, wearable bracelets/watches, and other physiological sensors that will be embedded with a VR-headset, so as to allow capturing the neuro-feedback of a VR-experience. The neuro-feedback will allow the virtual space to be adapted accordingly, resulting in an emotion-adaptive space. The ‘neuro-environments’ will be used at two levels of granularity. At the first level, artists and creatives will experience the ‘neuro-environments’ with the intention to improve the emotional-relevance of the urban space through the neuro-feedback of individuals, that have very-well…

aqua3S

aqua3S Enhancing Standardisation strategies to integrate innovative technologies for Safety and Security in existing water networks Exposure of citizens to potential disasters has led to vulnerable societies that require risk reduction measures. Drinking water is one of the main risk sources when its safety and security are not ensured. aqua3S combines novel technologies in water safety and security, aiming to standardize existing sensor technologies complemented by state-of-the-art detection mechanisms. Hence, it could propose innovative solutions to water facilities and responsible authorities in order to detect and tackle water-related crises in a timely manner. On the one hand, sensor networks are deployed in water supply networks and sources, supported by complex sensors for enhanced detection; on the other hand, sensor measurements are supported by videos from Unmanned Aerial Vehicles (UAVs), satellite…

VECTOR

VECTOR Virtual Evidence Capture Tool for Ordnance Recovery The identification, classification and risk assessment of explosive devices such as unexploded ordinances (UXOs) and improvised explosive devices (IEDs), is of critical importance to NATO and its partners' strategic interests and tactical operations. However, there are considerable challenges in doing so without specialised expertise present on the ground. VECTOR is going to provide an innovative solution that will enable real-time communication and collaboration between first responders and off-site experts via a multimedia capture application and a command and control dashboard. By synthesising cutting-edge 3D photogrammetry, image recognition, data visualization, and augmented / virtual reality technology, VECTOR aims to go beyond the state-of-the-art to enhance explosive ordnance disposal (EOD) operations against a wide range of threats.