Multimodal Data fusion – Multimodal Data Fusion and Analytics Group

BeOpen

BeOpen BeOpen an Open framework for boosting EU High ValueDatasets from Public Sector In the last decade, open data production related to Public Sector is quickly increased due to different factors, in particular: EU Open Data Directive fostered the openness of specific typology of datasets from the Public Sector (i.e. High Value Datasets); the improvement of the smart city paradigms (e.g. IoT deployments, digital twins) has dramatically improved the production of city-related data accessible by citizens; private companies are more and more contributing to public sector/utilities data availability used by citizens in their everyday life. Anyway, actual data use by third-party services still remains a challenge; a lot of data produced by public authorities, even “officially open”, is still not discoverable or is accessible only by humans, in a heterogeneous…

DYNAMO

DYNAMO Prevention, detection, response and mitigation of combined physical and cyber threats to critical infrastructure in Europe The scope of DYNAMO is to combine the two fields of business continuity management (BCM) and cyber threat intelligence (CTI) to generate a situational awareness picture for decision support across all stages of the resilience cycle (prepare, prevent, protect, response, recover). Professionals of different backgrounds will work together with end-users to develop, refine and combine selected tools into a single platform. In alignment to end-user needs, human factors, high ethical standards and societal impacts, DYNAMO includes the following goals: Resilience assessment as basis for BCM - An assessment with different levels of detail offers with varying existent data a fast or detailed evaluation of the investigated sector and helps to identify critical processes.…

MultiRATE

MultiRATE Holistic framework for the MatUrity evaLuaTIon of ReAdiness level for security TEchnologies In recent years, there are plenty of metrics that try to evaluate the maturity of products/systems/processes when identifying their deployment readiness. Even though a significant effort has been made to integrate such widely used frameworks, methodologies and indicators in products/systems/processes, their usage by EU funded security R&I projects has been limited. As a result, a robust scaling framework is more than of utmost importance. The proposed MultiRATE scaling framework will rely on existing established readiness level approaches and will formally combine them into one acceptable and reliable framework. MultiRATE will be able to evaluate and apply the readiness of all aspects of products/systems/processes. It will make a thorough research on the existing methodologies and will develop a…

FaRADAI

FaRADAI Frugal and Robust AI for Defence Advanced Intelligence An important crosscutting need for Artificial Intelligence is to create technologies for trustworthy autonomous and frugal learning, i.e. the ability of a system to adapt and learn from its environment, including from user supervision, for a reasonable cost and without intervention from expert developers nor regression. Such technologies can be highly disruptive and have high impacts for many capabilities, especially when the information to manage is highly variable or unpredictable and high adaptability is needed. These technologies can also alleviate the current need to provide data to the system developers to get improvements depending on such data, which can be critical when the data is confidential, and is thus critical for defence. They can more generally enhance technological independence. Selected actions…

SUN

SUN Social and hUman ceNtered XR The SUN Horizon Europe project aims at investigating and developing extended reality (XR) solutions that integrate the physical and the virtual world in a convincing way, from a human and social perspective. The virtual world will be a means to augment the physical world with new opportunities for social and human interaction. Relevant limitations that will be addressed are: • Lack of solutions to develop scalable and cost-effective new XR applications. • Lack of convincing solutions for mixing of the virtual and physical environment. • Lack of plausible and convincing human interaction interfaces in XR. • Barriers due to resource limitations of end-user devices. The SUN project develops solutions to surpass the aforementioned limitations, including: • Scalable solutions to obtain plausible and convincing virtual…

WATERVERSE

WATERVERSE Water Data Management Ecosystem for Water Data Spaces The WATERVERSE mission is to develop a Water Data Management Ecosystem (WDME) for making data management practices and resources in the water sector accessible, affordable, secure, fair, and easy to use, improving usability of data and the interoperability of data-intensive processes, thus lowering the entry barrier to data spaces, enhancing the resilience of water utilities and boosting the perceived value of data and therefore the market opportunities behind it. WATERVERSE takes a holistic, interdisciplinary approach in the water domain, blending together complementary competencies of 17 partners located in 10 EU countries, representing the water domain with Research organizations (including social sciences experts), water utilities, water domain technology providers and innovation companies, as well as the technical community that is driving the…

TALON

TALON Autonomous and Self-organized Artificial Intelligent Orchestrator for a Greener Industry 4.0 Next-generation industrial systems promise to deliver unprecedented excellence not only in terms of performance, but also explainability, trustworthiness and transparency. To achieve this new objectives, state-of-the-art concepts of artificial intelligence (AI), edge-to-cloud (E2C) computing, blockchain, and visualisation need to be de-risked and applied. Motivated by this, TALON aims at sculpturing the road towards the next Industrial revolution by developing a fully-automated AI architecture capable of bringing intelligence near the edge in a flexible, adaptable, explainable, energy and data efficient manner. M4D, the participating team from CERTH, contributes in the definition of TALON’s user and system requirements for the various use cases and the overall system, as well as in the definition of the KPIs, requirements, and the technology…

XRECO

XRECO XR mEdia eCOsystem While media organisations increasingly support non-linear experiences for the consumer, those are still limited to single channels and media domains. Although several media organisations have recently succeeded in breaking data silos, data sharing is mostly limited to the organisation. Thus, there are challenges for producing content feeding multiple channels with different granularities and structures, mainly related to data discovery, management and (re-)use. XRECO will create a new data-driven ecosystem for the media industry, focusing on facilitating data sharing, search and discovery and supporting creation of news and entertainment content, in particular, the creation and (re-)use of location-related 2D and 3D assets and the creation of XR experiences. The ecosystem core, represented by a Neural Media Repository (NMR), will foster inter-organisation content sharing and provide increased access…

CTC

CTC Cut The Cord With the nature of terror incidents undergoing a technology-aided transformation (e.g. use of cryptocurrencies to enhance anonymity), the monitoring and analysis of financial transactions play a pivotal role in the effort to timely prevent actions and funding of global terrorism. Financial crime exploits every aspect of digital transformation and identifies non-traditional products for financing and supporting terrorist organisations, such as social media, cryptocurrencies, and other Financial Technologies. The prevention and disruption of terrorism financing requires international cooperation and advanced investigative techniques based on new technologies. The CTC project (Cut The Cord) aims to enhance the public-private cooperation through the establishment of a wide stakeholder community, with the vision to sustain it beyond the end of the project. Joint training exercises (theoretical and hands-on) will be organised…

Nestor

NESTOR NESTOR – aN Enhanced pre-frontier intelligence picture to Safeguard The EurOpean boRders NESTOR aims to demonstrate a fully functional next generation holistic border surveillance system providing pre-frontier situational awareness beyond maritime and land border following the concept of the European Integrated Border Management. NESTOR long-range, wide area surveillance capabilities for detection, recognition classification and tracking of moving targets (e.g. persons, vehicles, drones etc.) is based on optical, thermal imagine and Radio Frequency (RF) spectrum analysis fed by an interoperable long range IP-based network of sensors including stationary installations and mobile manned or unmanned aerial, water surface, underwater and ground vehicles, capable of functioning both as standalone, tethered and in swarms (UAV, USV, UUV and UGV) platforms. NESTOR Border Command Control and Coordination intelligence (BC3i) system using cutting-edge Artificial Intelligence…

STARLIGHT

STARLIGHT STARLIGHT – Sustainable Autonomy and Resilience for LEAs using AI against High priority Threats STARLIGHT aims to empower LEAs with automated, operational, and cyber-resilient investigation, intelligence, surveillance and control capabilities to tackle traditional and emergent criminal activities, terrorism, cybercrime, and cyber-attacks, within and at the borders of the EU. STARLIGHT also aims at contributing to the establishment of a strong EU AI-based security ecosystem, thus enhancing the EU’s strategic autonomy in the field of AI for LEAs. The ecosystem will be built around an AI framework for reliable, accountable, responsible and transparent LEA AI solutions that will enable the involvement and transfer of knowledge between LEAs, Researchers, Industry, SMEs, and Policy makers. Overall, STARLIGHT will ensure that European LEAs lead the way in AI innovation, autonomy and resilience, addressing…

Nightingale

Nightingale Novel InteGrated toolkit for enhanced pre-Hospital life support and Triage IN challenGing And Large Emergencies Major emergencies, crisis and disasters have become more frequent the last decades. However, nowadays, emergency medical services and non-medical civil protection practitioners in a mass casualty incident scene are often striving to save lives but they are relying on complicated or outdated procedures and technology of the past. Additionally, disasters might differ in nature, geographic extent and other parameters, leading, in such a way, in a multitude of challenges. Therefore, it is uppermost affected people to feel safe and to get top-level healthcare that modern technology and current civil protection systems can offer. Nightingale is aimed at upgrading pre-hospital life support and Triage by delivering enhanced Operational Capacity to respond to mass casualty incidents…

VITALISE

VITALISE VIrtual healTh And weLlbeing Living Lab InftraStructurE The EU-funded VITALISE aims to establish new Living Labs in order to support the research community in the Health and Wellbeing domain. It also invests in learning about methodologies of user engagement and co-creation, such as design thinking, focus groups and ideation. VITALISE will design and develop ICT tools for shared access of similar devices and applications used across Living Labs, as well as for collecting, storing and sharing datasets. It will enhance closer interaction between multidisciplinary researchers among and beyond the consortium partners through the Joint Research Activities, Transnational Physical and Virtual Access. Physical access refers to access in the infrastructure by physical presence (17 Living Lab research infrastructures across Europe) while Virtual refers to remote access using digital tools. VITALISE…

ALAMEDA

ALAMEDA Bridging the Early Diagnosis and Treatment Gab of Brain Diseases via Smart, Connected, Proactive and Evidence-based Technological Interventions The EU-funded ALAMEDA project aims to provide new opportunities for improved personalized healthcare and prevention that have been enabled by the recent advances in the design and development of Innovative health risk prediction and intervention tools. ALAMEDA’s vision is to research and prototype the next generation of personalized AI healthcare support systems for people with brain diseases and disorders, specifically focusing on the needs of patients with Parkinson’s, Multiple Sclerosis and Stroke (PMSS). The project’s innovations will take advantage of new machine learning models, built upon lifestyle retrospective data as well as new streams of patient data that involve the monitoring of everyday activities, such as sleep behaviour and emotional status.…

CHROMATA

CHROMATA Dances, Songs, Myths and Customs for the Development of Technologies for Intangible Cultural Heritage The preservation of the intangible cultural heritage (e.g., traditional dances, songs, stories, techniques etc.) is a challenging task, since it is more difficult than maintaining tangible objects. The main purpose of the national project CHROMATA is to design and implement an interactive virtual reality platform that will gather elements of the intangible cultural heritage from available sources in areas of cultural interest and will constitute the starting point for the creation of indoor and outdoor environments of virtual and augmented reality that can (a) revive and represent the elements (e.g., traditional dances, stories, etc.) of the intangible heritage and b) create new spatial and virtual transcripts, thus promoting new uses for it. For the implementation…

xR4DRAMA

xR4DRAMA Extended Reality For DisasteR management And Media plAnning The EU-funded xR4DRAMA project aims to create a solution that will improve the situational awareness of those user groups who are responsible for handling disasters, man-made crises or public events. The groups range from first responders, local authorities and security forces to media companies and event planners. Situational awareness is crucial for them to execute tasks efficiently, plan things and take the appropriate decisions. The proof of concept of this project will focus on first responders and media companies before extending the use cases to other domains. xR4DRAMA's methodology is based on X Reality (XR) technologies, and the tool can be used remotely and on site for an augmented and shared perception of a given situation. In particular, innovative technologies will…

i4Q

i4Q Industrial Data Services for Quality Control in Smart Manufacturing i4Q Project will create a framework consisting of sustainable IoT-based Reliable Industrial Data Services (RIDS), that will provide solutions for factories in handling large amounts of data. The management and analysis of data derived from the work flows of factories will assist in the prediction and optimization of manufacturing processes that subsequently leads to improved process and product quality. Cost-effective, smart and small size interconnected factory devices will be used for online monitoring and control of manufacturing and will extract huge amounts of relevant data. i4Q Solutions will employ state-of-the-art communication protocols, ensuring data accuracy, precision and integrity through distributed ledger technology. The main objectives of i4q project are: 1) to develop, design and build the i4q framework, 2) to…

ASHVIN

ASHVIN aims at enabling the European construction industry to significantly improve its productivity, while reducing cost and ensuring absolutely safe work conditions. Towards this objective, ASHVIN will provide a European wide digital twin standard, an open source digital twin platform integrating IoT and image technologies, and a set of tools and demonstrated procedures to apply the platform and the standard proven to guarantee productivity, cost, and safety improvements. The envisioned platform will provide a digital representation of the construction product at hand and will allow to collect real-time data before, during, and after production of the product to continuously monitor changes in the environment and within the production process. Based on the platform, ASHVIN will develop and demonstrate applications that use the digital twin data. These applications will allow it…

GATEKEEPER

GATEKEEPER Smart Living Homes - Whole Interventions Demonstrator for People at Health and Social Risks The main objective of the project is to create a GATEKEEPER, that connects healthcare providers, businesses, entrepreneurs, elderly citizens and the communities they live in, in order to originate an open, trust-based arena for matching ideas, technologies, user needs and processes, aimed at ensuring healthier independent lives for the ageing populations. By 2022, GATEKEEPER will be embodied in an open source, European, standard-based, interoperable and secure framework available to all developers, for creating combined digital solutions for personalised early detection and interventions that (i) harness the next generation of healthcare and wellness innovations; (ii) cover the whole care continuum for elderly citizens, including primary, secondary and tertiary prevention, chronic diseases and co-morbidities; (iii) straightforwardly fit…

ISOLA

ISOLA A system platform that enhances the situational awareness and support Ship’s Security Stakeholders to the execution of their duties referring to the Ship's Security Plan Maritime transport enables trade and contacts between all EU nations and has always been a catalyst for economic development and prosperity of the Union. One of the major concerns of EU policy is to protect the citizens and economies from the consequences of unlawful intentional acts against shipping and port operations. ISOLA will develop, integrate, test, deploy, demonstrate and validate a systematic and fully automated security approach by incorporating innovative technologies for sensing, monitoring, data fusion, alarming and reporting real-time during illegal incidents. This will ensure high level of security among all passengers of the ship and augmentation of the Ship Security Plan. The…

SAFERS

SAFERS Enhancing our society's resilience against wildfires Forest fires are exacerbated by extreme weather conditions, which are increasing both in frequency and in magnitude due to climate change effects. This points to the need for improving the effectiveness of emergency management solutions aimed to help society in becoming more resilient to emergencies arising before, during and after forest fire events. Therefore, it is proposed to realize a comprehensive Emergency Management System (EMS) that we name SAFERS: Structured Approaches for Forest fire Emergencies in Resilient Societies. SAFERS will act along the key phases of the emergency management cycle, coupling information from EO data and services offered by Copernicus and GEOSS, crowdsourced data from social media and from specific applications that can be used by both citizens as well as from in-field…

ROBORDER

ROBORDER Autonomous swarm of heterogenous robots for border surveillance Border authorities and Law Enforcement Agencies (LEAs) across Europe face important challenges in how they patrol and protect the borders. Their work becomes more problematic considering the heterogeneity of threats, the wideness of the surveyed area, the adverse weather conditions and the wide range of terrains. Towards delivering a solution including all such factors, the main goal of ROBORDER is to develop and demonstrate a fully-functional autonomous border surveillance system with unmanned mobile robots including aerial, water surface, underwater and ground vehicles which will incorporate multimodal sensors as part of an interoperable network. Our intention is to implement a heterogenous robot system and enhance it with detection capabilities for early identification of criminal activities at border and coastal areas along with…

EOPEN

EOPEN Open interoperable platform for unified access and analysis of Earth Observation data Earth Observation (EO) data access through the Copernicus data distributor systems has paved the way to monitor changes on Earth, using Sentinel data. One of the main objectives of EOPEN is to fuse Sentinel data with multiple, heterogeneous and big data sources, to improve the monitoring capabilities of the future EO downstream sector. Additionally, the involvement of mature ICT solutions in the Earth Observation sector shall address major challenges in effectively handling and disseminating Copernicus-related information to the wider user community, beyond the EU borders. EOPEN will fuse Copernicus big data content with observations from non-EO data, such as weather, environmental and social media information, aiming at interactive, real-time and user-friendly visualisations and decisions from early warning…

REA

REA According to the World Health Organization (WHO), there are currently more than 1 billion people worldwide (about one in seven) who have some form of disability. It is estimated that over 150 million adults worldwide face serious movement issues and are not functionally independent. Care and rehabilitation are important factors in maintaining and improving the quality of life of people with physical or mental injuries. At the same time, they have important short-term and long-term economic implications for both individuals and their families, as well as for the health system and society. With the aim of upgrading the monitoring and support technologies currently used in clinical and non-clinical rehabilitation and patient care environments, the goal of the project is to develop an innovative interconnected virtual agent with enhanced interactive…

CONNEXIONs

CONNEXIONs InterCONnected NEXt-Generation Immersive IoT Platform of Crime and Terrorism DetectiON, PredictiON, InvestigatiON, and PreventiON Services CONNEXIONs aims to develop and demonstrate next-generation detection, prediction, prevention, and investigation services. These services will be based on multidimensional integration and correlation of heterogeneous multimodal data, and delivery of pertinent information to various stakeholders in an interactive manner tailored to their needs, through augmented and virtual reality environments. The CONNEXIONs solution encompasses the entire lifecycle of law enforcement operations including: pre-occurrence crime prediction and prevention during-occurrence LEA operations post-occurrence investigation, and crime-scene simulation and 3D reconstruction. CONNEXIONs will meaningfully enhance operational and (near) real-time situational awareness, through automated identification, interpretation, fusion and correlation of multiple heterogeneous big data sources, as well as their delivery via immersive solutions. Such multimodal data include Surface/Deep/Dark Web…

MULTISENSOR

MULTISENSOR Mining and Understanding of multilinguaL contenT for Intelligent Sentiment Enriched coNtext and Social Oriented inteRpretation The consumption of large amounts of multilingual and multimedia content regardless of its reliability and cross-validation can have important consequences on the society. An indicative example is the current crisis of the financial markets in Europe, which has created an extremely unstable ground for economic transactions and caused insecurity in the population. The fact that the national mass media provide exaggerated and contradictory information and the inaptness to understand local contexts from different countries has a considerable share in the aggravation of the crisis. To break this spiral, we need multilingual technologies with sentiment, social and spatiotemporal competence that are able to interpret, relate and summarise economic information and news created from various local…

beAWARE

beAWARE Enhancing decision support and management services in extreme weather climate events In every disaster and crisis, incident time is the enemy, and getting accurate information about the scope, extent, and impact of the disaster is critical to creating and orchestrating an effective disaster response and recovery effort. The main goal of beAWARE is to provide support in all the phases of an emergency incident. More specifically, we propose an integrated solution to support forecasting, early warnings, transmission and routing of the emergency data, aggregated analysis of multimodal data and management the coordination between the first responders and the authorities. Our intention is to rely on platforms, theories and methodologies that are already used for disaster forecasting and management and add the elements that are necessary to make them working…

DESMOS

DESMOS Smart Interconnected Infrastructures for Security and Protection The goal of DESMOS project is to combine technology and human resources to provide a protection and security environment for citizens, emphasizing on tourists, by creating the appropriate infrastructure to address a set of ‘urgent’ situations. Given that the use of mobile and portable devices is now widespread, the project intends to use existing technology and customization for the benefit of citizens. Emergencies that may arise may be of a medical nature (allergies, medical conditions), dangerous incidents (e.g. thefts, vandalism) or situations encountered in crowded places (e.g. child disappearance). Within the framework of the project, a platform for participatory cooperation between people and infrastructures will be created and through the reporting of incidents and emergencies using ICT, assistance will be provided by…

DIH^2

DIH^2 A Pan-European Network of Robotics DIHs for Agile Production DIH^2 is a network of 26 DIHs, with a target to reach over 170 DIHs. The sole aim of the network is to spark incremental (cut 50% cost of advance robotics solutions, double the growth of robotics market) and disruptive (maximum productivity & optimum agility) innovations in over 300,000 Manufacturing SMEs and Mid-Caps. It will support SMEs in their Agile Production challenge (50% increase in productivity) and unleash their digitalization potential by enabling robot solutions that are more cost effective at lower lot sizes. DIH^2 relies on: A Common Open Platform Reference Architecture for Agile Production (COPRA-AP) -based on Industrial Data Space Reference Architecture Model and FIWARE technologies- to serve the needs of SMEs by means of a continuously growing…

ARESIBO

ARESIBO Augmented Reality Enriched Situation Awareness for Border Security ARESIBO aims at improving the efficiency of the border surveillance systems by providing the operational teams and the tactical command and control level with an accurate and comprehensive information. The pillars of research in ARESIBO are three-fold: (i). Set-up a complete configuration at tactical and execution level to optimize the collaboration between human and sensors (fixed and mobile), (ii) Improve situation awareness by enhancing the understanding of the situation through adapted processing of sensor data, correlation between heterogeneous data and information and creation of knowledge through deep learning techniques and (iii) Create a situation awareness capability at C2 level that will combine reports on previous missions, real time situation understanding and threat analysis for future actions. This capability will be used…

aqua3S

aqua3S Enhancing Standardisation strategies to integrate innovative technologies for Safety and Security in existing water networks Exposure of citizens to potential disasters has led to vulnerable societies that require risk reduction measures. Drinking water is one of the main risk sources when its safety and security are not ensured. aqua3S combines novel technologies in water safety and security, aiming to standardize existing sensor technologies complemented by state-of-the-art detection mechanisms. Hence, it could propose innovative solutions to water facilities and responsible authorities in order to detect and tackle water-related crises in a timely manner. On the one hand, sensor networks are deployed in water supply networks and sources, supported by complex sensors for enhanced detection; on the other hand, sensor measurements are supported by videos from Unmanned Aerial Vehicles (UAVs), satellite…

CREST

CREST Fighting Crime and TerroRism with an IoT-enabled Autonomous Platform based on an Ecosystem of Advanced IntelligEnce, Operations, and InveStigation Technologies CREST aims to equip LEAs with an advanced prediction, prevention, operation, and investigation platform by leveraging the IoT ecosystem, autonomous systems, and targeted technologies and building upon the concept of multidimensional integration and correlation of heterogeneous multimodal data streams (ranging from online content to IoT-enabled sensors) for a) threat detection and assessment, b) dynamic mission planning and adaptive navigation for improved surveillance based on autonomous systems, c) distributed command and control of law enforcement missions, d) sharing of information and exchange of digital evidence based on blockchain, and e) delivery of pertinent information to different stakeholders in an interactive manner tailored to their needs. CREST will also provide chain-of-custody,…