My research focuses on developing and applying machine learning methods for pattern recognition and analysis of big data. My primary interest is in visual data (images and video), including semantic understanding of visual content, predicting user perceptions (e.g., affect), and multimedia retrieval.
However, I have also worked with text data and very heterogeneous computer usage log data for proactive information retrieval of personal data.
In machine learning my main interests have been in neural networks (deep learning) and kernel methods.
Pytorch-based image captioning framework used in our research group.
[Source code] - source code snapshot on GitHub (Apache 2.0 license).
The idea of the Digital Me (DiMe) server is to collect your personal data from various loggers into a central place that you control. The focus is on tracking knowledge work and developing applications that can help the knowledge worker manage their working life better.
A web-based tool for performing pair-wise annotation of multimedia (videos or images), used in several MediaEval tasks. [Source code on GitHub].
Non-academic talks to a more general audience.
Proactive Information Retrieval by Capturing Search Intent from Primary Task Context
Markus Koskela, Petri Luukkonen, Tuukka Ruotsalo, Mats Sjöberg and Patrik Floréen. In ACM Transactions on Interactive Intelligent Systems, Vol. 8, No. 3, Article 20. August 2018.
Predicting Interestingness of Visual Content
Claire-Hélène Demarty, Mats Sjöberg, Gabriel Constantin, Ngoc Q. K. Duong, Bogdan Ionescu, Thanh-Toan Do and Hanli Wang. In Visual Content Indexing and Retrieval with Psycho-Visual Models, Multimedia Systems and Applications, Springer. March 2017.
Digital Me: Controlling and Making Sense of My Digital Footprint
Mats Sjöberg, Hung-Han Chen, Patrik Floréen, Markus Koskela, Kai Kuikkaniemi, Tuukka Lehtiniemi, and Jaakko Peltonen. In Proceedings of the 5th International Workshop on Symbiotic Interaction, Symbiotic 2016. Padova, Italy.
The MediaEval 2015 Affective Impact of Movies Task
Mats Sjöberg, Yoann Baveye, Hanli Wang, Vu Lam Quang, Emmanuel Dellandréa, Markus Schedl, Claire-Hélène Demarty, and Liming Chen. In Proceedings of the MediaEval 2015 Multimedia Benchmark Workshop, Wurzen, Germany, September 14-15, 2015.
[PDF] [Slides PDF]
VSD2014: A Dataset for Violent Scenes Detection in
Hollywood Movies and Web Videos
Markus Schedl, Mats Sjöberg, Ionut Mironica, Bogdan Ionescu, Vu Lam Quang, Yu-Gang Jiang, Claire-Hélène Demarty. In Proceedings of the 13th International Workshop on Content-Based Multimedia Indexing (CBMI) 2015, Prague, Czech Republic, June 10-12, 2015.
The MediaEval 2014 Affect Task: Violent Scenes
Mats Sjöberg, Bogdan Ionescu, Yu-Gang Jiang, Vu Lam Quang, Markus Schedl, and Claire-Hélène Demarty. In Proceedings of the MediaEval 2014 Multimedia Benchmark Workshop, Barcelona, Spain, October 16-17, 2014.
[PDF] [Slides PDF]
Content-based Prediction of Movie Style, Aesthetics
and Affect: Data Set and Baseline Experiments
Jussi Tarvainen, Mats Sjöberg, Stina Westman, Jorma Laaksonen, Pirkko Oittinen. In IEEE Transactions on Multimedia, 2014.
Using semantic features to detect novel visual
Mats Sjöberg and Jorma Laaksonen. In Proceedings of the 12th International Content Based Multimedia Indexing Workshop (CBMI 2014), Klagenfurt, Austria, June, 2014.
Large-Scale Visual Concept Detection with Explicit Kernel Maps
and Power Mean SVM.
Mats Sjöberg, Markus Koskela, Satoru Ishikawa and Jorma Laaksonen. In Proceedings of ACM International Conference on Multimedia Retrieval (ICMR 2013), Dallas, Texas, USA, April, 2013.
Analysing the structure of semantic concepts in visual
Mats Sjöberg and Jorma Laaksonen. In Proceedings of 8th International Workshop on Self-Organizing Maps (WSOM 2011), Espoo, Finland, 2011.
Optimal combination of SOM search in best-matching units and
Mats Sjöberg and Jorma Laaksonen. In Proceedings of 7th International Workshop on Self-Organizing Maps (WSOM 2009), St. Augustine, Florida, USA, 2009.
Improving automatic video retrieval with semantic concept
Markus Koskela and Mats Sjöberg and Jorma Laaksonen. In Proceedings of 16th Scandinavian Conference on Image Analysis (SCIA 2009), Oslo, Norway, 2009.
Inferring Semantics from Textual Information in Multimedia
Mats Sjöberg and Jorma Laaksonen and Timo Honkela and Matti Pöllä. In Neurocomputing, 2008.
My ORCID is 0000-0002-3157-7668.
From pixels to semantics: visual concept detection and its
Department of Information and Computer Science, Aalto University School of Science.