MULTISENSOR

Mining and Understanding of multilinguaL contenT for Intelligent Sentiment Enriched coNtext and Social Oriented inteRpretation

The consumption of large amounts of multilingual and multimedia content regardless of its reliability and cross-validation can have important consequences on the society. An indicative example is the current crisis of the financial markets in Europe, which has created an extremely unstable ground for economic transactions and caused insecurity in  the population. The fact that the national mass media provide exaggerated and contradictory information and the inaptness to understand local contexts from different countries has a considerable share in the aggravation of the crisis.

To break this spiral, we need multilingual technologies with sentiment, social and spatiotemporal competence that are able to interpret, relate and summarise economic information and news created from various local subjective and biased views and disseminated via TV, radio, mass media websites and social media. To address these challenges, MULTISENSOR will mine heterogeneous data from the aforementioned resources and apply multidimensional content integration. MULTISENSOR will go beyond the state of the art by proposing the following scientific objectives:

a) content distillation of heterogeneous multimedia and multilingual data;
b) sentiment and context analysis of content and social interactions;
c) semantic integration of heterogeneous multimedia and multilingual data;
d) semantic reasoning and intelligent decision support;
e) multilingual and multimodal summarisation and presentation of the information to the user.

To achieve multidimensional integration of heterogeneous resources, MULTISENSOR proposes a content integration framework that builds upon multimedia mining, knowledge extraction, analysis of computer-mediated interaction, topic detection, semantic and multimodal representation as well as hybrid reasoning.

The developed technologies will be validated with the aid of 2 main use cases: a) International mass media news monitoring and b) SME internationalisation.

PUBLICATIONS

  1. S. Vrochidis, A. Moumtzidou, I. Gialampoukidis, D. Liparas, G. Casamayor, L. Wanner, N. Heise, T. Wagner, A. Bilous, E. Jamin, B. Simeonov, V. Alexiev, R. Busch, I. Arapakis, I. Kompatsiaris. “A multimodal analytics platform for journalists analysing large-scale, heterogeneous multilingual and multimedia content”. Frontiers in Robotics and AI, 5, 123, 29 October 2018. OPEN ACCESS
  2. I. Gialampoukidis, A. Moumtzidou, D. Liparas, T. Tsikrika, S. Vrochidis, I. Kompatsiaris. "Multimedia retrieval based on non-linear graph-based fusion and partial least squares regression". Multimedia Tools and Applications, 15 May 2017. OPEN ACCESS
  3. F. Markatopoulou, A. Moumtzidou, D. Galanopoulos, T. Mironidis, V. Kaltsa, A. Ioannidou, S. Symeonidis, K. Avgerinakis, S. Andreadis, I. Gialampoukidis, S. Vrochidis, A. Briassouli, V. Mezaris, I. Kompatsiaris, I. Patras, "ITI-CERTH participation in TRECVID 2016", in Proc. TRECVID 2016 Workshop, Gaithersburg, MD, USA, Nov. 2016. OPEN ACCESS
  4. I. Gialampoukidis, S. Vrochidis, I. Kompatsiaris,"Incremental estimation of visual vocabulary size for image retrieval", in INNS Conference on Big Data, pp. 29-38. Springer International Publishing, 2016.
  5. I. Gialampoukidis, T. Tsikrika, S. Vrochidis, I. Kompatsiaris,"Community Detection in Complex Networks Based on DBSCAN* and a Martingale Process", in 11th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP), 2016, pp. 1-6. IEEE.
  6. B. Simeonov, V. Alexiev, D. Liparas, M. Puigbo, S. Vrochidis, E. Jamin and I. Kompatsiaris, "Semantic integration of web data for international investment decision support",  3rd international conference on Internet Science, Florence, Italy, Sept. 12-14 2016. OPEN ACCESS
  7. I. Gialampoukidis, D. Liparas, S. Vrochidis, I. Kompatsiaris, "Query-based Topic Detection Using Concepts and Named Entities", in 1st International Workshop on Multimodal Media Data Analytics (MMDA 2016), The Hague, Netherlands, August 30, 2016.
  8. I. Gialampoukidis, G. Kalpakis, T. Tsikrika, S. Vrochidis, I. Kompatsiaris,"Key player identification in terrorism-related social media networks using centrality measures", in Intelligence and Security Informatics Conference (EISIC), 2016 European, pp. 112-115. IEEE, 2016. DOI: 10.1109/EISIC.2016.029
  9. I. Gialampoukidis, S. Vrochidis, I. Kompatsiaris, "A Hybrid framework for news clustering based on the DBSCAN-Martingale and LDA", in Machine Learning and Data Mining in Pattern Recognition, pp. 170-184, Springer International Publishing, 2016. DOI: 10.1007/978-3-319-41920-6_13
  10. A. Moumtzidou, I. Gialampoukidis, T. Mironidis, D. Liparas, S. Vrochidis, I. Kompatsiaris, "A Multimedia Interactive Search Engine based on Graph-based and Non-linear Multimodal Fusion", in 14th International Workshop on Content-Based Multimedia Indexing (CBMI), 2016, IEEE, 2016. DOI: 10.1109/CBMI.2016.7500276
  11. I. Gialampoukidis, A. Moumtzidou, T. Tsikrika, S. Vrochidis and I. Kompatsiaris, "Retrieval of Multimedia objects by Fusing Multiple Modalities", in Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval, pp. 359-362, ACM. https://dl.acm.org/doi/10.1145/2911996.2912068
  12. I. Gialampoukidis, A. Moumtzidou, D. Liparas, S. Vrochidis, I. Kompatsiaris, "A hybrid graph-based and non-linear late fusion approach for multimedia retrieval", in 14th International Workshop on Content-Based Multimedia Indexing (CBMI), 2016, pp. 1-6, IEEE, 2016. OPEN ACCESS
  13. I. Gialampoukidis, S. Vrochidis, I. Kompatsiaris, "Fast Visual Vocabulary Construction for Image Retrieval using Skewed-Split k-d trees", In International Conference on Multimedia Modeling, pp. 466-477. Springer In-ternational Publishing, 2016. https://doi.org/10.1007/978-3-319-27671-7_39
  14. A. Moumtzidou, T. Mironidis, E. Apostolidis, F. Markatopoulou, A. Ioannidou, I. Gialampoukidis, K. Avgerinakis, S. Vrochidis, V. Mezaris, I. Kompatsiaris, I. Patras,"VERGE: A Multimodal Interactive Search Engine for Video Browsing and Retrieval", in Proc. Video Browser Showdown (VBS'16) at the 22nd Int. Conf. on MultiMedia Modeling (MMM'16), Miami, USA, 4 January 2016. https://doi.org/10.1007/978-3-319-27674-8_39
  15. C. Doulaverakis, S. Vrochidis, I. Kompatsiaris,"Exploiting visual similarities for ontology alignment", 7th In-ternational Conference on Knowledge Engineering and Ontology Development (KEOD 2015), Lisbon, Portugal, 12-14 November, 2015. DOI: 10.5220/0005588200290037
  16. F. Markatopoulou, A. Ioannidou, C. Tzelepis, T. Mironidis, D. Galanopoulos, S. Arestis-Chartampilas, N. Pittaras, K. Avgerinakis, N. Gkalelis, A. Moumtzidou, S. Vrochidis, V. Mezaris, I. Kompatsiaris, I. Patras, "ITI-CERTH participation to TRECVID 2015", in Proc. TRECVID 2015 Workshop, Gaithersburg, MD, USA, Nov. 2015.
  17. S. Vrochidis, I. Kompatsiaris, G. Casamayor, I. Arapakis, R. Busch, V. Alexiev, E. Jamin, M. Jugov, N. Heise, T. Forrellat, D. Liparas, L. Wanner, I. Miliaraki, V. Aleksic, K. Simov, A. M. Soro, M. Eckhoff, T. Wagner, M. Puigbo,"MULTISENSOR: Development of Multimedia Content Integration Technologies for Journalism, Media Monitoring and International Exporting Decision Support", in 2015 IEEE International Conference on Multimedia and Expo (ICME 2015), Turin, Italy, June 29 - July 3, 2015.
  18. T. Tsikrika, A. Latas, A. Moumtzidou, E. Chatzilari, S. Vrochidis and I. Kompatsiaris,"Discovery of Environmental Web Resources Based on the Combination of Multimedia Evidence", in 2nd International Workshop on Environmental Multimedia Retrieval (EMR 2015), Shanghai, China, June 23, 2015. https://doi.org/10.1145/2764873.2764876
  19. T. Tsikrika, K. Andreadou, A. Moumtzidou, E. Schinas, S. Papadopoulos, S. Vrochidis, Y. Kompatsiaris, "A Unified Model for Socially Interconnected Multimedia-Enriched Objects", in 21st MultiMedia Modelling Conference (MMM2015), Sydney, Australia, 5-7 January, 2015. https://doi.org/10.1007/978-3-319-14445-0_32
  20. A. Moumtzidou, K. Avgerinakis, E. Apostolidis, F. Markatopoulou, K. Apostolidis, T. Mironidis, S. Vrochidis, V. Mezaris, Y. Kompatsiaris, I. Patras,"VERGE: A Multimodal Interactive Video Search Engine", in Proc. 21st Int. Conf. on MultiMedia Modeling (MMM15), Sydney, Australia, Jan. 2015. https://doi.org/10.1007/978-3-319-14442-9_23
  21. N. Gkalelis, F. Markatopoulou, A. Moumtzidou, D. Galanopoulos, K. Avgerinakis, N. Pittaras, S. Vrochidis, V. Mezaris, I. Kompatsiaris, I. Patras,"ITI-CERTH participation to TRECVID 2014", in Proc. TRECVID 2014 Work-shop, Orlando, FL, USA, November 2014.
  22. D. Liparas, A. Moumtzidou, S. Vrochidis, I. Kompatsiaris,"Concept-oriented labelling of patent images based on Random Forests and proximity-driven generation of synthetic data", in COLING'14 Workshop on Vision and Language (VL'14), Dublin, August 23, 2014. DOI: 10.3115/v1/W14-5404
  23. Y. HaCohen-Kerner, S. Vrochidis, D. Liparas, A. Moumtzidou and I. Kompatsiaris,"Key-phrase Extraction using Textual and Visual Features", in 3rd Workshop on Vision and Language (VL), Dublin, Ireland, August 23-29, 2014. DOI: 10.3115/v1/W14-5421
  24. T. Tsikrika, A. Moumtzidou, S. Vrochidis and I. Kompatsiaris, "Focused Crawling of Environmental Web Resources: A Pilot Study on the Combination of Multimedia Evidence", in Proceedings of 1st International Workshop on Environmental Multimedia Retrieval (EMR 2014), Glasgow, UK, April 1, 2014. DOI: 10.1007/s11042-015-2624-3
  25. A. Moumtzidou, K. Avgerinakis, E. Apostolidis, V. Aleksic, F. Markatopoulou, C. Papagiannopoulou, S. Vrochidis, V. Mezaris, R. Busch, I. Kompatsiaris,"VERGE: An Interactive Search Engine for Browsing Video Collections", in Proc. Video Browser Showdown (VBS'14) at the 20th Int. Conf. on MultiMedia Modeling (MMM'14), Dublin, Ireland, Springer LNCS vol. 8326, pp. 411-414, January 2014.
  26. F. Markatopoulou, A. Moumtzidou, C. Tzelepis, K. Avgerinakis, N. Gkalelis, S. Vrochidis, V. Mezaris, and I. Kompatsiaris, "ITI-CERTH participation to TRECVID 2013", in TRECVID 2013 Workshop, Gaithersburg, MD, USA, 2013.