Multimodal analytics based on AI – Multimodal Data Fusion and Analytics Group

CESAGRAM

CESAGRAM Towards a Comprehensive European Strategy Against tech-facilitated GRooming And Missing Child sexual abuse online is a growing phenomenon. The British National Society for the Prevention of Cruelty to Children (NSPCC) reported a 70% increase in the number of sexual communications with child offences in the last three years. The hotlines for missing children have reported concerns that grooming is an increasing factor in disappearances, particularly in cases of children who run away. In 2020, the second most common reason for children to contact Child Helplines was for reasons of violence of which 3,2% was online sexual abuse. Disappointingly, the COVID pandemic has exacerbated the problem. The COVID-19 crisis pushed 1.2 billion children out of schools, leading to children spending many more hours online. As Europol emphasised in its 2021…

PERIVALLON

PERIVALLON Protecting the EuRopean terrItory from organised enVironmentAl crime through inteLLigent threat detectiON tools PERIVALLON aims to provide an improved and comprehensive intelligence picture of organised environmental crime and develop effective and efficient tools and solutions for detecting and preventing such types of criminal activities and for assessing their environmental impact based on geospatial intelligence, remote sensing, scanning, online monitoring, analysis, correlation, risk assessment, and predictive analytics technologies, by leveraging the latest advancements in Artificial Intelligence (AI) in the fields of computer vision and multimodal analytics. As a result, enhanced investigation processes and methodologies will be derived through the capabilities provided by the developed tools and solutions, and the insights obtained though the proposed Environmental Crime Observatory. The capacity of end users (including Police Authorities and Border Guards) will also…

TeChBioT

TeChBioT Surveillance and Reconnaissance Techniques for Chemical & Biological Threats Threats associated with chemical, biological, radiological and nuclear (CBRN) hazards include natural disasters, accidental incidents during the installation or transportation of hazardous materials, as well as intentional incidents, such as terrorist attacks or state-sponsored events. Over the past decade, CBRN threats that originate deliberately have persisted or even increased. However, the COVID-19 pandemic is the event that has generated the most attention regarding the social, political, and economic consequences of CBRN crises in general, as well as biological/ health emergencies and civil protection mechanisms. Important gaps and weaknesses remain in rapid detection, identification, and monitoring of low-volatile chemical warfare agents (CWAs) and non-volatile biological warfare agents (BWAs), mainly in complex natural environments. Gas chromatography combined with mass spectrometry (GC-MS) or…

DYNAMO

DYNAMO Prevention, detection, response and mitigation of combined physical and cyber threats to critical infrastructure in Europe The scope of DYNAMO is to combine the two fields of business continuity management (BCM) and cyber threat intelligence (CTI) to generate a situational awareness picture for decision support across all stages of the resilience cycle (prepare, prevent, protect, response, recover). Professionals of different backgrounds will work together with end-users to develop, refine and combine selected tools into a single platform. In alignment to end-user needs, human factors, high ethical standards and societal impacts, DYNAMO includes the following goals: Resilience assessment as basis for BCM - An assessment with different levels of detail offers with varying existent data a fast or detailed evaluation of the investigated sector and helps to identify critical processes.…

FaRADAI

FaRADAI Frugal and Robust AI for Defence Advanced Intelligence An important crosscutting need for Artificial Intelligence is to create technologies for trustworthy autonomous and frugal learning, i.e. the ability of a system to adapt and learn from its environment, including from user supervision, for a reasonable cost and without intervention from expert developers nor regression. Such technologies can be highly disruptive and have high impacts for many capabilities, especially when the information to manage is highly variable or unpredictable and high adaptability is needed. These technologies can also alleviate the current need to provide data to the system developers to get improvements depending on such data, which can be critical when the data is confidential, and is thus critical for defence. They can more generally enhance technological independence. Selected actions…

DiMAT

DiMAT Digital Modelling and Simulation for Design, Processing and Manufacturing of Advanced Materials The DiMAT project addresses the sustainability and resilience of processes and systems within the materials manufacturing industry. An important takeaway from crises such as climate change and the pandemic was that sustainable action and resilient processes need to be the standard, not the exception. The digital transformation in the manufacturing industry (Industry 4.0, smart factory, etc.) is weakly coupled with materials science and engineering, a fact most prominent when considering that the share of the materials production cost is about 30-55 %, but the energy cost to produce a component is about 2-8 %. Large companies benefit from the close collaboration of the scientific community and the industry for the development of Integrated Computational Materials Engineering (ICME),…

SUN

SUN Social and hUman ceNtered XR The SUN Horizon Europe project aims at investigating and developing extended reality (XR) solutions that integrate the physical and the virtual world in a convincing way, from a human and social perspective. The virtual world will be a means to augment the physical world with new opportunities for social and human interaction. Relevant limitations that will be addressed are: • Lack of solutions to develop scalable and cost-effective new XR applications. • Lack of convincing solutions for mixing of the virtual and physical environment. • Lack of plausible and convincing human interaction interfaces in XR. • Barriers due to resource limitations of end-user devices. The SUN project develops solutions to surpass the aforementioned limitations, including: • Scalable solutions to obtain plausible and convincing virtual…

WATERVERSE

WATERVERSE Water Data Management Ecosystem for Water Data Spaces The WATERVERSE mission is to develop a Water Data Management Ecosystem (WDME) for making data management practices and resources in the water sector accessible, affordable, secure, fair, and easy to use, improving usability of data and the interoperability of data-intensive processes, thus lowering the entry barrier to data spaces, enhancing the resilience of water utilities and boosting the perceived value of data and therefore the market opportunities behind it. WATERVERSE takes a holistic, interdisciplinary approach in the water domain, blending together complementary competencies of 17 partners located in 10 EU countries, representing the water domain with Research organizations (including social sciences experts), water utilities, water domain technology providers and innovation companies, as well as the technical community that is driving the…

TALON

TALON Autonomous and Self-organized Artificial Intelligent Orchestrator for a Greener Industry 4.0 Next-generation industrial systems promise to deliver unprecedented excellence not only in terms of performance, but also explainability, trustworthiness and transparency. To achieve this new objectives, state-of-the-art concepts of artificial intelligence (AI), edge-to-cloud (E2C) computing, blockchain, and visualisation need to be de-risked and applied. Motivated by this, TALON aims at sculpturing the road towards the next Industrial revolution by developing a fully-automated AI architecture capable of bringing intelligence near the edge in a flexible, adaptable, explainable, energy and data efficient manner. M4D, the participating team from CERTH, contributes in the definition of TALON’s user and system requirements for the various use cases and the overall system, as well as in the definition of the KPIs, requirements, and the technology…

ReSilence

ReSilence Retune the Soundscape of future cities through art and science collaboration The main objective of the ReSilence project is to support the development of art-driven technologies for designing the soundscape of future cities through the collaboration of artists, architects, urban designers, scientists, engineers and researchers, and targeting AI and XR tools addressing primarily challenges from the Urbanism/Mobility sector while exploring the borders between music and noise, and user experience measurements assessing a soundspace. The project ReSilence builds on the novel and prima facie paradoxical observation that elements of city life tend to become more silent, allowing us to question which sounds are necessary for safety and environmental awareness, and which sounds can be made more pleasant. Nowadays city planners, architects, engineers, and other scientific experts are urgently studying the…

RAISE

RAISE Research Analysis Identifier SystEm The EU-funded RAISE project aims to address the challenge of an open, fair and reliable Research Community, where every researcher will be accredited for its work and all research data will be equally accessible for processing without violating data protection regulations. RAISE vision is to provide the infrastructure for a distributed crowdsourced data processing system, moving from open data to open access data for processing. RAISE will produce the mechanism for sending the algorithm to the dataset instead of sending the data to the algorithm. The success of such an application will provide to users a transparent way of processing and sharing data, enabling the research community to publish their outcomes with evidence-based authenticity of the data-analysis performed, ensuring at the same time the accreditation…

XRECO

XRECO XR mEdia eCOsystem While media organisations increasingly support non-linear experiences for the consumer, those are still limited to single channels and media domains. Although several media organisations have recently succeeded in breaking data silos, data sharing is mostly limited to the organisation. Thus, there are challenges for producing content feeding multiple channels with different granularities and structures, mainly related to data discovery, management and (re-)use. XRECO will create a new data-driven ecosystem for the media industry, focusing on facilitating data sharing, search and discovery and supporting creation of news and entertainment content, in particular, the creation and (re-)use of location-related 2D and 3D assets and the creation of XR experiences. The ecosystem core, represented by a Neural Media Repository (NMR), will foster inter-organisation content sharing and provide increased access…

LEAGUE

LEAGUE Limiting online sexual Exploitation and Abuse Gender based on Underaged boys by Educating experts The Limiting online sexual Exploitation and Abuse Gender based on Underaged boys by Educating experts (LEAGUE) project addresses the further need of prevention and increased reporting of online child sexual abuse among boys aged 10–18. The actions proposed for this project will help to the development of the competencies of the organizations and authorities responsible for assisting victims of such crimes. LEAGUE aims to prevent and report online child sexual abuse of boys aged 10-18, to facilitate early detection, protection and support to underaged boys and young people who are victims or potential victims of violence, with particular emphasis to situations emerging from the COVID-19 pandemic and to enhance multidisciplinary cooperation and coordination among relevant…

CTC

CTC Cut The Cord With the nature of terror incidents undergoing a technology-aided transformation (e.g. use of cryptocurrencies to enhance anonymity), the monitoring and analysis of financial transactions play a pivotal role in the effort to timely prevent actions and funding of global terrorism. Financial crime exploits every aspect of digital transformation and identifies non-traditional products for financing and supporting terrorist organisations, such as social media, cryptocurrencies, and other Financial Technologies. The prevention and disruption of terrorism financing requires international cooperation and advanced investigative techniques based on new technologies. The CTC project (Cut The Cord) aims to enhance the public-private cooperation through the establishment of a wide stakeholder community, with the vision to sustain it beyond the end of the project. Joint training exercises (theoretical and hands-on) will be organised…

STARLIGHT

STARLIGHT STARLIGHT – Sustainable Autonomy and Resilience for LEAs using AI against High priority Threats STARLIGHT aims to empower LEAs with automated, operational, and cyber-resilient investigation, intelligence, surveillance and control capabilities to tackle traditional and emergent criminal activities, terrorism, cybercrime, and cyber-attacks, within and at the borders of the EU. STARLIGHT also aims at contributing to the establishment of a strong EU AI-based security ecosystem, thus enhancing the EU’s strategic autonomy in the field of AI for LEAs. The ecosystem will be built around an AI framework for reliable, accountable, responsible and transparent LEA AI solutions that will enable the involvement and transfer of knowledge between LEAs, Researchers, Industry, SMEs, and Policy makers. Overall, STARLIGHT will ensure that European LEAs lead the way in AI innovation, autonomy and resilience, addressing…

ODYSSEUS

ODYSSEUS Preventing, Countering and Investigation Terrorist Attacks Through Prognostic, Detection and Forensic Mechanisms for Explosive Precursors ODYSSEUS aims to increase the knowledge on homemade explosives (HMEs) and explosive precursors, including precursors not previously studied, and develop effective and efficient prognostic, detection, and forensic tools to improve the capabilities of Law Enforcement Agencies (LEAs) towards the prevention, countering, and investigation of terrorist incidents involving HMEs. ODYSSEUS will build upon relevant previous projects mainly HOMER, through the involvement of HOMER’s core partners in this consortium, and will thus continue the work already performed in HOMER on some precursors and also further extend it to not previously studied precursors. To discover potentially hitherto unknown information, online HMEs recipes will be collected and their content will be analysed in order to extract knowledge about…

SECANT

SECANT SECurity And privacy protectioN in Internet of Things devices SECANT aims to contribute decisively towards improving the readiness and resilience of organisations against the crippling modern cyber-threats, by increasing the privacy, data protection, and accountability across the entire interconnected ICT ecosystem, and by reducing the costs for security training in the European market. To this end, the SECANT platform will enhance the capabilities of organisations’ stakeholders, by implementing (a) collaborative threat intelligence collection, analysis, and sharing; (b) innovative risk analysis specifically designed for interconnected nodes of an industrial ecosystem; (c) cutting-edge trust and accountability mechanisms for data protection, and (d) security awareness training for more informed security choices. The effectiveness and versatility of the proposed solution will be validated in four realistic pilot use case scenarios applied in the…

Nightingale

Nightingale Novel InteGrated toolkit for enhanced pre-Hospital life support and Triage IN challenGing And Large Emergencies Major emergencies, crisis and disasters have become more frequent the last decades. However, nowadays, emergency medical services and non-medical civil protection practitioners in a mass casualty incident scene are often striving to save lives but they are relying on complicated or outdated procedures and technology of the past. Additionally, disasters might differ in nature, geographic extent and other parameters, leading, in such a way, in a multitude of challenges. Therefore, it is uppermost affected people to feel safe and to get top-level healthcare that modern technology and current civil protection systems can offer. Nightingale is aimed at upgrading pre-hospital life support and Triage by delivering enhanced Operational Capacity to respond to mass casualty incidents…

VITALISE

VITALISE VIrtual healTh And weLlbeing Living Lab InftraStructurE The EU-funded VITALISE aims to establish new Living Labs in order to support the research community in the Health and Wellbeing domain. It also invests in learning about methodologies of user engagement and co-creation, such as design thinking, focus groups and ideation. VITALISE will design and develop ICT tools for shared access of similar devices and applications used across Living Labs, as well as for collecting, storing and sharing datasets. It will enhance closer interaction between multidisciplinary researchers among and beyond the consortium partners through the Joint Research Activities, Transnational Physical and Virtual Access. Physical access refers to access in the infrastructure by physical presence (17 Living Lab research infrastructures across Europe) while Virtual refers to remote access using digital tools. VITALISE…

ALAMEDA

ALAMEDA Bridging the Early Diagnosis and Treatment Gab of Brain Diseases via Smart, Connected, Proactive and Evidence-based Technological Interventions The EU-funded ALAMEDA project aims to provide new opportunities for improved personalized healthcare and prevention that have been enabled by the recent advances in the design and development of Innovative health risk prediction and intervention tools. ALAMEDA’s vision is to research and prototype the next generation of personalized AI healthcare support systems for people with brain diseases and disorders, specifically focusing on the needs of patients with Parkinson’s, Multiple Sclerosis and Stroke (PMSS). The project’s innovations will take advantage of new machine learning models, built upon lifestyle retrospective data as well as new streams of patient data that involve the monitoring of everyday activities, such as sleep behaviour and emotional status.…

AIDA

AIDA Artificial Intelligence and advanced Data Analytics for Law Enforcement Agencies The goal of AIDA is to provide Law Enforcement Agencies (LEAs) in the EU Member States (MS) with innovative, dedicated and sustainable digital technologies to facilitate the entire Big Data analytics cycle as part of their investigative and proactive prevention capabilities focused on cybercrime and counterterrorism. More specifically, AIDA will provide law enforcement across the EU with a modular capacity to acquire, (pre-)process, enrich, analyse and visualise multi-source Big Data in near real-time. The latter will shift the capacity of EU Member States to fight cybercrime and terrorism, while implementing governance procedures to maintain full compliance with fundamental rights and applicable legislation, including privacy and protection of personal data and ensure chain of custody. The AIDA framework will adapt…

TENSOR

TENSOR reTriEval and aNalysis of heterogeneouS online content for TerrOrist activity Recognition Law Enforcement Agencies (LEAs) across Europe face today important challenges in how they identify, gather and interpret terrorist generated content online. The Dark Web presents additional challenges due to its inaccessibility and the fact that undetected material can contribute to the advancement of terrorist violence and radicalisation. LEAs also face the challenge of extracting and summarising meaningful and relevant content hidden in huge amounts of online data to inform their resource deployment and investigations. In this context, the main objective of the TENSOR project is to provide a powerful terrorism intelligence platform offering LEAs fast and reliable planning and prevention functionalities for the early detection of terrorist organised activities, radicalisation and recruitment. The platform integrates a set of…

EOPEN

EOPEN Open interoperable platform for unified access and analysis of Earth Observation data Earth Observation (EO) data access through the Copernicus data distributor systems has paved the way to monitor changes on Earth, using Sentinel data. One of the main objectives of EOPEN is to fuse Sentinel data with multiple, heterogeneous and big data sources, to improve the monitoring capabilities of the future EO downstream sector. Additionally, the involvement of mature ICT solutions in the Earth Observation sector shall address major challenges in effectively handling and disseminating Copernicus-related information to the wider user community, beyond the EU borders. EOPEN will fuse Copernicus big data content with observations from non-EO data, such as weather, environmental and social media information, aiming at interactive, real-time and user-friendly visualisations and decisions from early warning…

CONNEXIONs

CONNEXIONs InterCONnected NEXt-Generation Immersive IoT Platform of Crime and Terrorism DetectiON, PredictiON, InvestigatiON, and PreventiON Services CONNEXIONs aims to develop and demonstrate next-generation detection, prediction, prevention, and investigation services. These services will be based on multidimensional integration and correlation of heterogeneous multimodal data, and delivery of pertinent information to various stakeholders in an interactive manner tailored to their needs, through augmented and virtual reality environments. The CONNEXIONs solution encompasses the entire lifecycle of law enforcement operations including: pre-occurrence crime prediction and prevention during-occurrence LEA operations post-occurrence investigation, and crime-scene simulation and 3D reconstruction. CONNEXIONs will meaningfully enhance operational and (near) real-time situational awareness, through automated identification, interpretation, fusion and correlation of multiple heterogeneous big data sources, as well as their delivery via immersive solutions. Such multimodal data include Surface/Deep/Dark Web…

MULTISENSOR

MULTISENSOR Mining and Understanding of multilinguaL contenT for Intelligent Sentiment Enriched coNtext and Social Oriented inteRpretation The consumption of large amounts of multilingual and multimedia content regardless of its reliability and cross-validation can have important consequences on the society. An indicative example is the current crisis of the financial markets in Europe, which has created an extremely unstable ground for economic transactions and caused insecurity in the population. The fact that the national mass media provide exaggerated and contradictory information and the inaptness to understand local contexts from different countries has a considerable share in the aggravation of the crisis. To break this spiral, we need multilingual technologies with sentiment, social and spatiotemporal competence that are able to interpret, relate and summarise economic information and news created from various local…

ECHO

ECHO European network of Cybersecurity centres and competence Hub for innovation and Operations ECHO delivers an organized and coordinated approach to improve proactive cyber defence of the European Union, through effective and efficient multi-sector collaboration. The Partners will execute on a 48-month work plan to develop, model and demonstrate a network of cyber research and competence centres, with a central competence at the hub. The Central Competence Hub serves as the focal point for the ECHO Multi-sector Assessment Framework enabling multi-sector dependencies management, provision of an ECHO Early Warning System, an ECHO Federation of Cyber Ranges and management of an expanding collection of Partner Engagements. The ECHO Multi-sector Assessment Framework refers to the analysis of challenges and opportunities derived from sector specific use cases, transversal cybersecurity needs analysis and development…

CREST

CREST Fighting Crime and TerroRism with an IoT-enabled Autonomous Platform based on an Ecosystem of Advanced IntelligEnce, Operations, and InveStigation Technologies CREST aims to equip LEAs with an advanced prediction, prevention, operation, and investigation platform by leveraging the IoT ecosystem, autonomous systems, and targeted technologies and building upon the concept of multidimensional integration and correlation of heterogeneous multimodal data streams (ranging from online content to IoT-enabled sensors) for a) threat detection and assessment, b) dynamic mission planning and adaptive navigation for improved surveillance based on autonomous systems, c) distributed command and control of law enforcement missions, d) sharing of information and exchange of digital evidence based on blockchain, and e) delivery of pertinent information to different stakeholders in an interactive manner tailored to their needs. CREST will also provide chain-of-custody,…

INFINITY

INFINITY IMMERSE. INTERACT. INVESTIGATE Infinity’s ambition is to become a flagship project against society’s most pressing cybercriminal, terrorist and hybrid threats. Synthesising the latest innovations in virtual and augmented reality, artificial intelligence and machine learning with big data and visual analytics, Infinity will deliver an integrated solution that aims to revolutionise data driven investigations. Bringing together a strong representation from national and supranational agencies with an end user-driven design, it will directly address the core needs of contemporary law enforcement. Specifically, it will equip investigators and analysts with cutting-edge tools to acquire, process, visualise and act upon the enormous quantities of data they are faced with every day. Bolstered by cognitive research, automated systems and instinctive interfaces and controls, Infinity will be designed and developed to maximise the potential of…

VECTOR

VECTOR Virtual Evidence Capture Tool for Ordnance Recovery The identification, classification and risk assessment of explosive devices such as unexploded ordinances (UXOs) and improvised explosive devices (IEDs), is of critical importance to NATO and its partners' strategic interests and tactical operations. However, there are considerable challenges in doing so without specialised expertise present on the ground. VECTOR is going to provide an innovative solution that will enable real-time communication and collaboration between first responders and off-site experts via a multimedia capture application and a command and control dashboard. By synthesising cutting-edge 3D photogrammetry, image recognition, data visualization, and augmented / virtual reality technology, VECTOR aims to go beyond the state-of-the-art to enhance explosive ordnance disposal (EOD) operations against a wide range of threats.