Art&Media – Multimodal Data Fusion and Analytics Group

SUN

SUN Social and hUman ceNtered XR The SUN Horizon Europe project aims at investigating and developing extended reality (XR) solutions that integrate the physical and the virtual world in a convincing way, from a human and social perspective. The virtual world will be a means to augment the physical world with new opportunities for social and human interaction. Relevant limitations that will be addressed are: • Lack of solutions to develop scalable and cost-effective new XR applications. • Lack of convincing solutions for mixing of the virtual and physical environment. • Lack of plausible and convincing human interaction interfaces in XR. • Barriers due to resource limitations of end-user devices. The SUN project develops solutions to surpass the aforementioned limitations, including: • Scalable solutions to obtain plausible and convincing virtual…

ReSilence

ReSilence Retune the Soundscape of future cities through art and science collaboration The main objective of the ReSilence project is to support the development of art-driven technologies for designing the soundscape of future cities through the collaboration of artists, architects, urban designers, scientists, engineers and researchers, and targeting AI and XR tools addressing primarily challenges from the Urbanism/Mobility sector while exploring the borders between music and noise, and user experience measurements assessing a soundspace. The project ReSilence builds on the novel and prima facie paradoxical observation that elements of city life tend to become more silent, allowing us to question which sounds are necessary for safety and environmental awareness, and which sounds can be made more pleasant. Nowadays city planners, architects, engineers, and other scientific experts are urgently studying the…

XRECO

XRECO XR mEdia eCOsystem While media organisations increasingly support non-linear experiences for the consumer, those are still limited to single channels and media domains. Although several media organisations have recently succeeded in breaking data silos, data sharing is mostly limited to the organisation. Thus, there are challenges for producing content feeding multiple channels with different granularities and structures, mainly related to data discovery, management and (re-)use. XRECO will create a new data-driven ecosystem for the media industry, focusing on facilitating data sharing, search and discovery and supporting creation of news and entertainment content, in particular, the creation and (re-)use of location-related 2D and 3D assets and the creation of XR experiences. The ecosystem core, represented by a Neural Media Repository (NMR), will foster inter-organisation content sharing and provide increased access…

V4U

V4U Virtual technologies For (4) Universities: Supporting online learning with virtual technologies Since COVID-19, many universities have been forced to switch to online learning. This sudden change (also known as “The Great Online Pivot”) has been challenging for students and educators. This emergency pivot to online learning has impacted all educational levels. To salvage the academic year, educators with no prior online teaching experience were called to convert their courses to distance learning modules. Adapting curriculum on the fly, educators were forced to adjust student expectations and explore new instructional methods to replicate in-person classes. It was found that such initiatives had different level of success – deciding factors were educator skills and the academic field. Regarding the second factor, fields such as architecture and fine arts, imposed barriers on…

PrismArch

PrismArch Virtual reality aided design blending cross-disciplinary aspects of architecture in a multi-simulation environment The recent past of the AEC industry is characterized by digital breakthroughs that have dramatically shaped the design process. The powerful programs that are currently available offer unlimited possibilities, allowing complex and creative designs to be modelled accurately with ease while enabling collaboration and Building Information Modeling. The realization of structurally complex constructions is an inter-disciplinary process that passes through the close collaboration of architects and engineers. By employing advanced VR applications, designers can immersively perceive and interact with the current status of their creations and realize the consequences of their decisions. Unfortunately, these solutions focus on the aesthetic aspect of architecture while ignoring the driving cause hidden behind each architectural creation, functionality, that can only…

xR4DRAMA

xR4DRAMA Extended Reality For DisasteR management And Media plAnning The EU-funded xR4DRAMA project aims to create a solution that will improve the situational awareness of those user groups who are responsible for handling disasters, man-made crises or public events. The groups range from first responders, local authorities and security forces to media companies and event planners. Situational awareness is crucial for them to execute tasks efficiently, plan things and take the appropriate decisions. The proof of concept of this project will focus on first responders and media companies before extending the use cases to other domains. xR4DRAMA's methodology is based on X Reality (XR) technologies, and the tool can be used remotely and on site for an augmented and shared perception of a given situation. In particular, innovative technologies will…

V4Design

V4Design Visual and textual content re-purposing FOR(4) architecture, Design and video virtual reality games V4Design promotes the re-use of visual and textual content, such as documentaries, paintings and their critiques so as to be repurposed and be used for the inspiration of architecture and video game (VR) designing. Visual content analysis is performed through 3D reconstruction and modelling, localization of objects in visual data, automated extraction of aesthetics/style information and language understanding from visual and textual data. 3D objects are also enhanced with semantics and explanatory text descriptions. V4Design will develop a platform that provides architects, video game creators and designers of any expertise with innovative tools necessary to enhance and simplify the creative phase of the designing process. The main idea behind V4Design is to reuse (i) visual: movies,…

iV&L Net

iV&L Net The European Network on Integrating Vision and Language Combining Computer Vision and Language Processing For Advanced Search, Retrieval, Annotation and Description of Visual Data, COST Action IC1307, is designed to bring together two previously unconnected research communities, Computer Vision (CV) and Natural Language Processing (NLP). It is set up to allow researchers to meet, exchange ideas, expertise and technology, and form new partnerships. Building on the successful EPSRC Network on Vision and Language (V&L Net), the aim of the COST Action is to create a lasting interdisciplinary research community situated at the nexus between language processing and computer vision. This is hugely important because of the explosive growth of visual and textual data (both on the World Wide Web and held in private repositories by diverse institutions and…

MULTISENSOR

MULTISENSOR Mining and Understanding of multilinguaL contenT for Intelligent Sentiment Enriched coNtext and Social Oriented inteRpretation The consumption of large amounts of multilingual and multimedia content regardless of its reliability and cross-validation can have important consequences on the society. An indicative example is the current crisis of the financial markets in Europe, which has created an extremely unstable ground for economic transactions and caused insecurity in the population. The fact that the national mass media provide exaggerated and contradictory information and the inaptness to understand local contexts from different countries has a considerable share in the aggravation of the crisis. To break this spiral, we need multilingual technologies with sentiment, social and spatiotemporal competence that are able to interpret, relate and summarise economic information and news created from various local…

MindSpaces

MindSpaces Art-driven adaptive outdoors and indoors design MindSpaces aims to create a novel approach to urban and architectural design by generating 3D-VR immersive and emotion-adaptive ‘neuro-environments’ that will help in designing emotionally-relevant urban spaces. The emotional aspects of an environment will be captured through the use of mobile EEG (Elecroencephalography) headsets, wearable bracelets/watches, and other physiological sensors that will be embedded with a VR-headset, so as to allow capturing the neuro-feedback of a VR-experience. The neuro-feedback will allow the virtual space to be adapted accordingly, resulting in an emotion-adaptive space. The ‘neuro-environments’ will be used at two levels of granularity. At the first level, artists and creatives will experience the ‘neuro-environments’ with the intention to improve the emotional-relevance of the urban space through the neuro-feedback of individuals, that have very-well…