Computer vision – Multimodal Data Fusion and Analytics Group

PERIVALLON

PERIVALLON Protecting the EuRopean terrItory from organised enVironmentAl crime through inteLLigent threat detectiON tools PERIVALLON aims to provide an improved and comprehensive intelligence picture of organised environmental crime and develop effective and efficient tools and solutions for detecting and preventing such types of criminal activities and for assessing their environmental impact based on geospatial intelligence, remote sensing, scanning, online monitoring, analysis, correlation, risk assessment, and predictive analytics technologies, by leveraging the latest advancements in Artificial Intelligence (AI) in the fields of computer vision and multimodal analytics. As a result, enhanced investigation processes and methodologies will be derived through the capabilities provided by the developed tools and solutions, and the insights obtained though the proposed Environmental Crime Observatory. The capacity of end users (including Police Authorities and Border Guards) will also…

FaRADAI

FaRADAI Frugal and Robust AI for Defence Advanced Intelligence An important crosscutting need for Artificial Intelligence is to create technologies for trustworthy autonomous and frugal learning, i.e. the ability of a system to adapt and learn from its environment, including from user supervision, for a reasonable cost and without intervention from expert developers nor regression. Such technologies can be highly disruptive and have high impacts for many capabilities, especially when the information to manage is highly variable or unpredictable and high adaptability is needed. These technologies can also alleviate the current need to provide data to the system developers to get improvements depending on such data, which can be critical when the data is confidential, and is thus critical for defence. They can more generally enhance technological independence. Selected actions…

SUN

SUN Social and hUman ceNtered XR The SUN Horizon Europe project aims at investigating and developing extended reality (XR) solutions that integrate the physical and the virtual world in a convincing way, from a human and social perspective. The virtual world will be a means to augment the physical world with new opportunities for social and human interaction. Relevant limitations that will be addressed are: • Lack of solutions to develop scalable and cost-effective new XR applications. • Lack of convincing solutions for mixing of the virtual and physical environment. • Lack of plausible and convincing human interaction interfaces in XR. • Barriers due to resource limitations of end-user devices. The SUN project develops solutions to surpass the aforementioned limitations, including: • Scalable solutions to obtain plausible and convincing virtual…

LAGO

LAGO LESSEN DATA ACCESS AND GOVERNANCE OBSTACLES LAGO will deliver the foundation for a trusted EU FTC Research Data Ecosystem (RDE) to address the so-called “Data Issue” in the FCT research landscape, i.e., the lack of domain-specific data in sufficient quality and quantity to enable appropriate training and testing of the developed methods, platforms and tools. LAGO will be instrumental in identifying common barriers and subsequently providing the structural, governance and technical foundations to foster and innovate data-oriented research collaboration among LEAs, security practitioners, relevant EU agencies, academic and industry researchers, policy makers and regulators. For this purpose,LAGO will develop an evidence-based and validated multi-actor Reference Architecture for the FCT RDE for these actors to deposit, share and co-create data and tools for FCT research purposes based on common rules,…

WATERVERSE

WATERVERSE Water Data Management Ecosystem for Water Data Spaces The WATERVERSE mission is to develop a Water Data Management Ecosystem (WDME) for making data management practices and resources in the water sector accessible, affordable, secure, fair, and easy to use, improving usability of data and the interoperability of data-intensive processes, thus lowering the entry barrier to data spaces, enhancing the resilience of water utilities and boosting the perceived value of data and therefore the market opportunities behind it. WATERVERSE takes a holistic, interdisciplinary approach in the water domain, blending together complementary competencies of 17 partners located in 10 EU countries, representing the water domain with Research organizations (including social sciences experts), water utilities, water domain technology providers and innovation companies, as well as the technical community that is driving the…

TALON

TALON Autonomous and Self-organized Artificial Intelligent Orchestrator for a Greener Industry 4.0 Next-generation industrial systems promise to deliver unprecedented excellence not only in terms of performance, but also explainability, trustworthiness and transparency. To achieve this new objectives, state-of-the-art concepts of artificial intelligence (AI), edge-to-cloud (E2C) computing, blockchain, and visualisation need to be de-risked and applied. Motivated by this, TALON aims at sculpturing the road towards the next Industrial revolution by developing a fully-automated AI architecture capable of bringing intelligence near the edge in a flexible, adaptable, explainable, energy and data efficient manner. M4D, the participating team from CERTH, contributes in the definition of TALON’s user and system requirements for the various use cases and the overall system, as well as in the definition of the KPIs, requirements, and the technology…

AIDEAS

AIDEAS AI Driven industrial Equipment product life cycle boosting Agility, Sustainability and resilience AIDEAS will develop AI technologies for supporting the entire lifecycle (design, manufacturing, use, and repair/reuse/recycle) of industrial equipment as a strategic instrument to improve sustainability, agility and resilience of the European machinery manufacturing companies. AIDEAS will deploy 4 integrated Suites: 1) Design: AI technologies, integrated with CAD/CAM/CAE systems, for optimising the design of industrial equipment structural components, mechanisms and control components; 2) Manufacturing: AI technologies for industrial equipment purchased components selection and procurement, manufactured parts processes optimisation, operations sequencing, quality control and customisation; 3) Use: AI technologies with added value for the industrial equipment user, providing enhanced support for installation and initial calibration, production, quality assurance and predictive maintenance for working on optimal conditions; 4) Repair-Reuse-Recycle: AI…

ReSilence

ReSilence Retune the Soundscape of future cities through art and science collaboration The main objective of the ReSilence project is to support the development of art-driven technologies for designing the soundscape of future cities through the collaboration of artists, architects, urban designers, scientists, engineers and researchers, and targeting AI and XR tools addressing primarily challenges from the Urbanism/Mobility sector while exploring the borders between music and noise, and user experience measurements assessing a soundspace. The project ReSilence builds on the novel and prima facie paradoxical observation that elements of city life tend to become more silent, allowing us to question which sounds are necessary for safety and environmental awareness, and which sounds can be made more pleasant. Nowadays city planners, architects, engineers, and other scientific experts are urgently studying the…

Nestor

NESTOR NESTOR – aN Enhanced pre-frontier intelligence picture to Safeguard The EurOpean boRders NESTOR aims to demonstrate a fully functional next generation holistic border surveillance system providing pre-frontier situational awareness beyond maritime and land border following the concept of the European Integrated Border Management. NESTOR long-range, wide area surveillance capabilities for detection, recognition classification and tracking of moving targets (e.g. persons, vehicles, drones etc.) is based on optical, thermal imagine and Radio Frequency (RF) spectrum analysis fed by an interoperable long range IP-based network of sensors including stationary installations and mobile manned or unmanned aerial, water surface, underwater and ground vehicles, capable of functioning both as standalone, tethered and in swarms (UAV, USV, UUV and UGV) platforms. NESTOR Border Command Control and Coordination intelligence (BC3i) system using cutting-edge Artificial Intelligence…

LAW-GAME

LAW-GAME An Interactive, Collaborative Digital Gamification Approach to Effective Experiential Training and Prediction of Criminal Actions Criminal incidents play an important role as a disturbance of European Unions’ welfare. Police forces face daily acts of crime, such as intentional homicide and assault, drug trafficking, human and illegal firearm trafficking, as well as other forms of illegal activities, associated with the presence of organized crime groups(OCGs) in the EU. It is very crucial to have trained security personnel with an orientation in detail and critical thinking. Skillful law enforcement officers are demanded to tackle effectively this type of incidents, who in most cases are not easily available. The improvement of law enforcement officers will increase both LEAs capabilities and effectiveness in the fight against crime and terrorism and protection of European…

STRONG

STRONG advanced firST RespONders training First responders play a crucial and integral role in today’s civil society. Among the plethora of services they provide, one may find law enforcement, firefighting and emergency medical services (EMS). In all three of the aforementioned services, time management could play an important role in order to assist first responders to reach their maximum potential and effectively and efficiently deal with the crisis. For instance, crime rates could significantly decrease if a police officer has the complete situational awareness picture to hinder criminals; fire fighters could be able to save more lives in less time from burning buildings and EMS could act in a proper and quick manner in order to treat medical conditions (such as cardiac arrests) that could result in death. The same…

CHROMATA

CHROMATA Dances, Songs, Myths and Customs for the Development of Technologies for Intangible Cultural Heritage The preservation of the intangible cultural heritage (e.g., traditional dances, songs, stories, techniques etc.) is a challenging task, since it is more difficult than maintaining tangible objects. The main purpose of the national project CHROMATA is to design and implement an interactive virtual reality platform that will gather elements of the intangible cultural heritage from available sources in areas of cultural interest and will constitute the starting point for the creation of indoor and outdoor environments of virtual and augmented reality that can (a) revive and represent the elements (e.g., traditional dances, stories, etc.) of the intangible heritage and b) create new spatial and virtual transcripts, thus promoting new uses for it. For the implementation…

xR4DRAMA

xR4DRAMA Extended Reality For DisasteR management And Media plAnning The EU-funded xR4DRAMA project aims to create a solution that will improve the situational awareness of those user groups who are responsible for handling disasters, man-made crises or public events. The groups range from first responders, local authorities and security forces to media companies and event planners. Situational awareness is crucial for them to execute tasks efficiently, plan things and take the appropriate decisions. The proof of concept of this project will focus on first responders and media companies before extending the use cases to other domains. xR4DRAMA's methodology is based on X Reality (XR) technologies, and the tool can be used remotely and on site for an augmented and shared perception of a given situation. In particular, innovative technologies will…

ASHVIN

ASHVIN aims at enabling the European construction industry to significantly improve its productivity, while reducing cost and ensuring absolutely safe work conditions. Towards this objective, ASHVIN will provide a European wide digital twin standard, an open source digital twin platform integrating IoT and image technologies, and a set of tools and demonstrated procedures to apply the platform and the standard proven to guarantee productivity, cost, and safety improvements. The envisioned platform will provide a digital representation of the construction product at hand and will allow to collect real-time data before, during, and after production of the product to continuously monitor changes in the environment and within the production process. Based on the platform, ASHVIN will develop and demonstrate applications that use the digital twin data. These applications will allow it…

ISOLA

ISOLA A system platform that enhances the situational awareness and support Ship’s Security Stakeholders to the execution of their duties referring to the Ship's Security Plan Maritime transport enables trade and contacts between all EU nations and has always been a catalyst for economic development and prosperity of the Union. One of the major concerns of EU policy is to protect the citizens and economies from the consequences of unlawful intentional acts against shipping and port operations. ISOLA will develop, integrate, test, deploy, demonstrate and validate a systematic and fully automated security approach by incorporating innovative technologies for sensing, monitoring, data fusion, alarming and reporting real-time during illegal incidents. This will ensure high level of security among all passengers of the ship and augmentation of the Ship Security Plan. The…

7SHIELD

7SHIELD Prevention, detection, response and mitigation of combined physical and cyber threats to critical infrastructure in Europe The 7SHIELD project addresses the security and the resilience of EU Ground Segments of Space Systems, meeting the crosscutting and the sectorial criteria of the EU critical infrastructures (2008/14). The Copernicus era has created a new market with the massive amounts of satellite data that the ground segments of space systems receive serve to the market and governmental bodies. A physical/cyber-attack to their installations or communication networks, respectively, would cause debilitating impact on public safety and security of EU citizens and public authorities. A physical attack on a space ground segment makes the distribution of satellite data problematic and, on the other hand, a cyber-attack in its data storage, access and exchange affects…

iV&L Net

iV&L Net The European Network on Integrating Vision and Language Combining Computer Vision and Language Processing For Advanced Search, Retrieval, Annotation and Description of Visual Data, COST Action IC1307, is designed to bring together two previously unconnected research communities, Computer Vision (CV) and Natural Language Processing (NLP). It is set up to allow researchers to meet, exchange ideas, expertise and technology, and form new partnerships. Building on the successful EPSRC Network on Vision and Language (V&L Net), the aim of the COST Action is to create a lasting interdisciplinary research community situated at the nexus between language processing and computer vision. This is hugely important because of the explosive growth of visual and textual data (both on the World Wide Web and held in private repositories by diverse institutions and…

ROBORDER

ROBORDER Autonomous swarm of heterogenous robots for border surveillance Border authorities and Law Enforcement Agencies (LEAs) across Europe face important challenges in how they patrol and protect the borders. Their work becomes more problematic considering the heterogeneity of threats, the wideness of the surveyed area, the adverse weather conditions and the wide range of terrains. Towards delivering a solution including all such factors, the main goal of ROBORDER is to develop and demonstrate a fully-functional autonomous border surveillance system with unmanned mobile robots including aerial, water surface, underwater and ground vehicles which will incorporate multimodal sensors as part of an interoperable network. Our intention is to implement a heterogenous robot system and enhance it with detection capabilities for early identification of criminal activities at border and coastal areas along with…

REA

REA According to the World Health Organization (WHO), there are currently more than 1 billion people worldwide (about one in seven) who have some form of disability. It is estimated that over 150 million adults worldwide face serious movement issues and are not functionally independent. Care and rehabilitation are important factors in maintaining and improving the quality of life of people with physical or mental injuries. At the same time, they have important short-term and long-term economic implications for both individuals and their families, as well as for the health system and society. With the aim of upgrading the monitoring and support technologies currently used in clinical and non-clinical rehabilitation and patient care environments, the goal of the project is to develop an innovative interconnected virtual agent with enhanced interactive…

CONNEXIONs

CONNEXIONs InterCONnected NEXt-Generation Immersive IoT Platform of Crime and Terrorism DetectiON, PredictiON, InvestigatiON, and PreventiON Services CONNEXIONs aims to develop and demonstrate next-generation detection, prediction, prevention, and investigation services. These services will be based on multidimensional integration and correlation of heterogeneous multimodal data, and delivery of pertinent information to various stakeholders in an interactive manner tailored to their needs, through augmented and virtual reality environments. The CONNEXIONs solution encompasses the entire lifecycle of law enforcement operations including: pre-occurrence crime prediction and prevention during-occurrence LEA operations post-occurrence investigation, and crime-scene simulation and 3D reconstruction. CONNEXIONs will meaningfully enhance operational and (near) real-time situational awareness, through automated identification, interpretation, fusion and correlation of multiple heterogeneous big data sources, as well as their delivery via immersive solutions. Such multimodal data include Surface/Deep/Dark Web…

beAWARE

beAWARE Enhancing decision support and management services in extreme weather climate events In every disaster and crisis, incident time is the enemy, and getting accurate information about the scope, extent, and impact of the disaster is critical to creating and orchestrating an effective disaster response and recovery effort. The main goal of beAWARE is to provide support in all the phases of an emergency incident. More specifically, we propose an integrated solution to support forecasting, early warnings, transmission and routing of the emergency data, aggregated analysis of multimodal data and management the coordination between the first responders and the authorities. Our intention is to rely on platforms, theories and methodologies that are already used for disaster forecasting and management and add the elements that are necessary to make them working…

MULTIFORESEE

MULTIFORESEE MULTI-modal Imaging of FOREnsic SciEnce Evidence The main objective of the COST Action CA16101, entitled MULTI-modal Imaging of FOREnsic SciEnce Evidence (MULTI-FORESEE)- tools for Forensic Science, is to promote innovative, multi-informative, operationally deployable and commercially exploitable imaging solutions/technology to analyse forensic evidence. Forensic evidence includes, but not limited to, fingermarks, hair, paint, biofluids, digital evidence, fibers, documents and living individuals. Imaging technologies include optical, mass spectrometric, spectroscopic, chemical, physical and digital forensic techniques complemented by expertise in IT solutions and computational modelling. Imaging technologies enable multiple physical and chemical information to be captured in one analysis, from one specimen, with information being more easily conveyed and understood for a more rapid exploitation. The enhanced value of the evidence gathered will be conducive to much more informed investigations and judicial…

CoFly

CoFly The main aim of CoFly project is to develop an operational framework for micro air vehicles (MAVs) that would be affordable and widely distributed in the marketplace. In addition, the main objective is to provide the entrepreneurs the ability to complete high precision manual labour operations with lesser cost and time-consuming way, meaning without the need of a MAV operator expert. Although a large variety of affordable MAV platforms is available, their majority offer limited control functionalities. Moreover, a wide spectrum of navigational sensors and computing systems, capable to be attached on a MAV, is also available. Thus, a MAV platform with advanced functionalities comprises the ultimate objective of the project, including an interactive navigation system as well as conceptual and autonomous decision-making abilities. The final product will increase…

DIH^2

DIH^2 A Pan-European Network of Robotics DIHs for Agile Production DIH^2 is a network of 26 DIHs, with a target to reach over 170 DIHs. The sole aim of the network is to spark incremental (cut 50% cost of advance robotics solutions, double the growth of robotics market) and disruptive (maximum productivity & optimum agility) innovations in over 300,000 Manufacturing SMEs and Mid-Caps. It will support SMEs in their Agile Production challenge (50% increase in productivity) and unleash their digitalization potential by enabling robot solutions that are more cost effective at lower lot sizes. DIH^2 relies on: A Common Open Platform Reference Architecture for Agile Production (COPRA-AP) -based on Industrial Data Space Reference Architecture Model and FIWARE technologies- to serve the needs of SMEs by means of a continuously growing…

ARESIBO

ARESIBO Augmented Reality Enriched Situation Awareness for Border Security ARESIBO aims at improving the efficiency of the border surveillance systems by providing the operational teams and the tactical command and control level with an accurate and comprehensive information. The pillars of research in ARESIBO are three-fold: (i). Set-up a complete configuration at tactical and execution level to optimize the collaboration between human and sensors (fixed and mobile), (ii) Improve situation awareness by enhancing the understanding of the situation through adapted processing of sensor data, correlation between heterogeneous data and information and creation of knowledge through deep learning techniques and (iii) Create a situation awareness capability at C2 level that will combine reports on previous missions, real time situation understanding and threat analysis for future actions. This capability will be used…

aqua3S

aqua3S Enhancing Standardisation strategies to integrate innovative technologies for Safety and Security in existing water networks Exposure of citizens to potential disasters has led to vulnerable societies that require risk reduction measures. Drinking water is one of the main risk sources when its safety and security are not ensured. aqua3S combines novel technologies in water safety and security, aiming to standardize existing sensor technologies complemented by state-of-the-art detection mechanisms. Hence, it could propose innovative solutions to water facilities and responsible authorities in order to detect and tackle water-related crises in a timely manner. On the one hand, sensor networks are deployed in water supply networks and sources, supported by complex sensors for enhanced detection; on the other hand, sensor measurements are supported by videos from Unmanned Aerial Vehicles (UAVs), satellite…

CREST

CREST Fighting Crime and TerroRism with an IoT-enabled Autonomous Platform based on an Ecosystem of Advanced IntelligEnce, Operations, and InveStigation Technologies CREST aims to equip LEAs with an advanced prediction, prevention, operation, and investigation platform by leveraging the IoT ecosystem, autonomous systems, and targeted technologies and building upon the concept of multidimensional integration and correlation of heterogeneous multimodal data streams (ranging from online content to IoT-enabled sensors) for a) threat detection and assessment, b) dynamic mission planning and adaptive navigation for improved surveillance based on autonomous systems, c) distributed command and control of law enforcement missions, d) sharing of information and exchange of digital evidence based on blockchain, and e) delivery of pertinent information to different stakeholders in an interactive manner tailored to their needs. CREST will also provide chain-of-custody,…

VECTOR

VECTOR Virtual Evidence Capture Tool for Ordnance Recovery The identification, classification and risk assessment of explosive devices such as unexploded ordinances (UXOs) and improvised explosive devices (IEDs), is of critical importance to NATO and its partners' strategic interests and tactical operations. However, there are considerable challenges in doing so without specialised expertise present on the ground. VECTOR is going to provide an innovative solution that will enable real-time communication and collaboration between first responders and off-site experts via a multimedia capture application and a command and control dashboard. By synthesising cutting-edge 3D photogrammetry, image recognition, data visualization, and augmented / virtual reality technology, VECTOR aims to go beyond the state-of-the-art to enhance explosive ordnance disposal (EOD) operations against a wide range of threats.