Multimedia retrieval – Multimodal Data Fusion and Analytics Group

XRECO

XRECO XR mEdia eCOsystem While media organisations increasingly support non-linear experiences for the consumer, those are still limited to single channels and media domains. Although several media organisations have recently succeeded in breaking data silos, data sharing is mostly limited to the organisation. Thus, there are challenges for producing content feeding multiple channels with different granularities and structures, mainly related to data discovery, management and (re-)use. XRECO will create a new data-driven ecosystem for the media industry, focusing on facilitating data sharing, search and discovery and supporting creation of news and entertainment content, in particular, the creation and (re-)use of location-related 2D and 3D assets and the creation of XR experiences. The ecosystem core, represented by a Neural Media Repository (NMR), will foster inter-organisation content sharing and provide increased access…

PrismArch

PrismArch Virtual reality aided design blending cross-disciplinary aspects of architecture in a multi-simulation environment The recent past of the AEC industry is characterized by digital breakthroughs that have dramatically shaped the design process. The powerful programs that are currently available offer unlimited possibilities, allowing complex and creative designs to be modelled accurately with ease while enabling collaboration and Building Information Modeling. The realization of structurally complex constructions is an inter-disciplinary process that passes through the close collaboration of architects and engineers. By employing advanced VR applications, designers can immersively perceive and interact with the current status of their creations and realize the consequences of their decisions. Unfortunately, these solutions focus on the aesthetic aspect of architecture while ignoring the driving cause hidden behind each architectural creation, functionality, that can only…

CALLISTO

CALLISTO Copernicus Artificial Intelligence Services and data fusion with other distributed data sources and processing at the edge to support DIAS and HPC infrastructures Artificial Intelligence (AI) is already part of our lives and is extensively entering the space sector to offer value-added Earth Observation (EO) products and services. CALLISTO project integrates Copernicus data, already indexed in DIAS platforms such as ONDA-DIAS, utilising High Performance Computing infrastructures for enhanced scalability when needed. Complementary distributed data sources involve Galileo positioning data, visual content from UAVs, Web and social media data linking them with open geospatial data, in-situ sensor data. On top of these data sources, AI methods are applied to extract meaningful knowledge such as concepts, changes, activities, events, 3D-models, videos and animations of the user community. AI methods are also…

ISOLA

ISOLA A system platform that enhances the situational awareness and support Ship’s Security Stakeholders to the execution of their duties referring to the Ship's Security Plan Maritime transport enables trade and contacts between all EU nations and has always been a catalyst for economic development and prosperity of the Union. One of the major concerns of EU policy is to protect the citizens and economies from the consequences of unlawful intentional acts against shipping and port operations. ISOLA will develop, integrate, test, deploy, demonstrate and validate a systematic and fully automated security approach by incorporating innovative technologies for sensing, monitoring, data fusion, alarming and reporting real-time during illegal incidents. This will ensure high level of security among all passengers of the ship and augmentation of the Ship Security Plan. The…

KRISTINA

KRISTINA A Knowledge-Based Information Agent with Social Competence and Human Interaction Capabilities KRISTINA’s overall objective is to research and develop technologies for a human-like socially competent and communicative agent that is run on mobile communication devices and that serves for migrants with language and cultural barriers in the host country as a trusted information provision party and mediator in questions related to basic care and healthcare. To develop such an agent, KRISTINA will advance the state of the art in dialogue management, multimodal (vocal, facial and gestural) communication analysis and multimodal communication. The technologies will be validated in two use cases, in which prolonged trials will be carried out for each prototype that marks the termination of a SW development cycle, with a representative number of migrants recruited as users…

V4Design

V4Design Visual and textual content re-purposing FOR(4) architecture, Design and video virtual reality games V4Design promotes the re-use of visual and textual content, such as documentaries, paintings and their critiques so as to be repurposed and be used for the inspiration of architecture and video game (VR) designing. Visual content analysis is performed through 3D reconstruction and modelling, localization of objects in visual data, automated extraction of aesthetics/style information and language understanding from visual and textual data. 3D objects are also enhanced with semantics and explanatory text descriptions. V4Design will develop a platform that provides architects, video game creators and designers of any expertise with innovative tools necessary to enhance and simplify the creative phase of the designing process. The main idea behind V4Design is to reuse (i) visual: movies,…

iV&L Net

iV&L Net The European Network on Integrating Vision and Language Combining Computer Vision and Language Processing For Advanced Search, Retrieval, Annotation and Description of Visual Data, COST Action IC1307, is designed to bring together two previously unconnected research communities, Computer Vision (CV) and Natural Language Processing (NLP). It is set up to allow researchers to meet, exchange ideas, expertise and technology, and form new partnerships. Building on the successful EPSRC Network on Vision and Language (V&L Net), the aim of the COST Action is to create a lasting interdisciplinary research community situated at the nexus between language processing and computer vision. This is hugely important because of the explosive growth of visual and textual data (both on the World Wide Web and held in private repositories by diverse institutions and…

TENSOR

TENSOR reTriEval and aNalysis of heterogeneouS online content for TerrOrist activity Recognition Law Enforcement Agencies (LEAs) across Europe face today important challenges in how they identify, gather and interpret terrorist generated content online. The Dark Web presents additional challenges due to its inaccessibility and the fact that undetected material can contribute to the advancement of terrorist violence and radicalisation. LEAs also face the challenge of extracting and summarising meaningful and relevant content hidden in huge amounts of online data to inform their resource deployment and investigations. In this context, the main objective of the TENSOR project is to provide a powerful terrorism intelligence platform offering LEAs fast and reliable planning and prevention functionalities for the early detection of terrorist organised activities, radicalisation and recruitment. The platform integrates a set of…

MULTIFORESEE

MULTIFORESEE MULTI-modal Imaging of FOREnsic SciEnce Evidence The main objective of the COST Action CA16101, entitled MULTI-modal Imaging of FOREnsic SciEnce Evidence (MULTI-FORESEE)- tools for Forensic Science, is to promote innovative, multi-informative, operationally deployable and commercially exploitable imaging solutions/technology to analyse forensic evidence. Forensic evidence includes, but not limited to, fingermarks, hair, paint, biofluids, digital evidence, fibers, documents and living individuals. Imaging technologies include optical, mass spectrometric, spectroscopic, chemical, physical and digital forensic techniques complemented by expertise in IT solutions and computational modelling. Imaging technologies enable multiple physical and chemical information to be captured in one analysis, from one specimen, with information being more easily conveyed and understood for a more rapid exploitation. The enhanced value of the evidence gathered will be conducive to much more informed investigations and judicial…

MindSpaces

MindSpaces Art-driven adaptive outdoors and indoors design MindSpaces aims to create a novel approach to urban and architectural design by generating 3D-VR immersive and emotion-adaptive ‘neuro-environments’ that will help in designing emotionally-relevant urban spaces. The emotional aspects of an environment will be captured through the use of mobile EEG (Elecroencephalography) headsets, wearable bracelets/watches, and other physiological sensors that will be embedded with a VR-headset, so as to allow capturing the neuro-feedback of a VR-experience. The neuro-feedback will allow the virtual space to be adapted accordingly, resulting in an emotion-adaptive space. The ‘neuro-environments’ will be used at two levels of granularity. At the first level, artists and creatives will experience the ‘neuro-environments’ with the intention to improve the emotional-relevance of the urban space through the neuro-feedback of individuals, that have very-well…

WELCOME

WELCOME Multiple Intelligent Conversation Agent Services for Reception, Management and Integration of Third Country Nationals in the EU The objective of WELCOME is to research and develop intelligent technologies for support of the reception and integration of migrants in Europe. Unlike the majority of the projects that address the problem of migrant integration by frontends with “one-for-all” interfaces for information acquisition and services, WELCOME will offer a personalized and psychologically and socially competent solution for both migrants and public administrations. It will develop immersive and intelligent services, in which embodied intelligent multilingual agents will act as dedicated personalized assistants of migrants in contexts of registration, orientation, language teaching, civic education, and social and societal inclusion. The agents will be personalized in the sense that they will dynamically adapt their interaction…