Dr Arno Solin

Assistant Professor in Machine Learning
Academy of Finland Postdoctoral Research Fellow

PublicationsSoftwareInvited TalksTeaching

I work as Assistant Professor in Machine Learning in the Department of Computer Science at Aalto University. I also hold an Academy of Finland Postdoctoral Fellowship (2017–) and work as technical advisor for IndoorAtlas Ltd.

Previously, I have been a Visiting Research Fellow with Department of Engineering, University of Cambridge (12/2017–06/2018), Senior Algorithm Developer and Team Lead with Indoor Atlas Ltd. (2015–2017), and Visiting Researcher with University of Sheffield (10/2013). I did my doctoral degree in Prof. Aki Vehtari's Bayesian Statistical Methods Group and my instructor was Prof. Simo Särkkä.

Research Activities

I have a broad interest in statistical modelling, machine learning, and signal processing. My key research interests are in Bayesian inference methods for stochastic dynamical systems, linear and non-linear Kalman filtering, and Gaussian process models and infinite-dimensional Kalman filtering for spatio-temporal models.

My application areas revolve around real-time inference applications on limited hardware such as smartphones. These include sensor fusion applications for tracking and localisation, and temporal and spatio-temporal modelling of, e.g., the weather and human brain.

Exciting Stuff

Arno Solin

Contact Information

arno.solin@aalto.fi
+358 40 577 6226
arno.solin

Street Address

Room A317
Tietotekniikan talo
Konemiehentie 2
Espoo, Finland

Postal Address

P.O. Box 15400
FI-00076 AALTO
FINLAND

Publications

The PDF preprints below are draft versions of the journal articles. They are here to give an opportunity to check the relevance of the articles before purchasing the final articles from the publisher. Complete bibliography information is given in the BibTeX format for each published article/report.

My Google Scholar profile.

Books

submitted

Simo Särkkä and Arno Solin (to appear). Applied Stochastic Differential Equations. Cambridge University Press, Cambridge.

Applied Stochastic Differential Equations

Simo Särkkä and Arno Solin

Journal Articles

submitted

James Hensman, Nicolas Durrande, and Arno Solin (2018). Variational Fourier features for Gaussian processes. Journal of Machine Learning Research, 18(151):1–52.

Variational Fourier features for Gaussian processes

James Hensman, Nicolas Durrande, and Arno Solin

This work brings together two powerful concepts in Gaussian processes: the variational approach to sparse approximation and the spectral representation of Gaussian processes. This gives rise to an approximation that inherits the benefits of the variational approach but with the representational power and computational scalability of spectral representations. The work hinges on a key result that there exist spectral features related to a finite domain of the Gaussian process which exhibit almost-independent covariances. We derive these expressions for Matérn kernels in one dimension, and generalize to more dimensions using kernels with specific structures. Under the assumption of additive Gaussian noise, our method requires only a single pass through the data set, making for very fast and accurate computation. We fit a model to 4 million training points in just a few minutes on a standard laptop. With non-conjugate likelihoods, our MCMC scheme reduces the cost of computation from O(NM2) (for a sparse Gaussian process) to O(NM) per iteration, where N is the number of data and M is the number of features.

submitted

Arno Solin, Manon Kok, Niklas Wahlström, Thomas B. Schön, and Simo Särkkä (2018). Modeling and interpolation of the ambient magnetic field by Gaussian processes. To appear in IEEE Transactions on Robotics.

Modeling and interpolation of the ambient magnetic field by Gaussian processes

Arno Solin, Manon Kok, Niklas Wahlström, Thomas B. Schön, and Simo Särkkä

submitted

Juho Kokkala, Arno Solin and Simo Särkkä (2016). Sigma-Point Filtering and Smoothing Based Parameter Estimation in Nonlinear Dynamic Systems. Journal of Advances in Information Fusion, 11(1):15–30.

Sigma-Point Filtering and Smoothing Based Parameter Estimation in Nonlinear Dynamic Systems

Juho Kokkala, Arno Solin and Simo Särkkä

We consider approximate maximum likelihood parameter estimation in nonlinear state-space models. We discuss both direct optimization of the likelihood and expectation-maximization (EM). For EM, we also give closed-form expressions for the maximization step in a class of models that are linear in parameters and have additive noise. To obtain approximations to the filtering and smoothing distributions needed in the likelihood-maximization methods, we focus on using Gaussian filtering and smoothing algorithms that employ sigma-points to approximate the required integrals. We discuss different sigma-point schemes based on the third, fifth, seventh, and ninth order unscented transforms and the Gauss-Hermite quadrature rule. We compare the performance of the methods in two simulated experiments: a univariate nonlinear growth model as well as tracking of a maneuvering target. In the experiments, we also compare against approximate likelihood estimates obtained by particle filtering and extended Kalman filtering based methods. The experiments suggest that the higher-order unscented transforms may in some cases provide more accurate estimates.

Solin-Sarkka-2013-PRE

Arno Solin and Simo Särkkä (2013). Infinite-dimensional Bayesian filtering for detection of quasiperiodic phenomena in spatiotemporal data. Physical Review E, 88(5):052909.

Infinite-dimensional Bayesian filtering for detection of quasiperiodic phenomena in spatiotemporal data

Arno Solin and Simo Särkkä

This paper introduces a spatiotemporal resonator model and an inference method for detection and estimation of nearly periodic temporal phenomena in spatiotemporal data. The model is derived as a spatial extension of a stochastic harmonic resonator model, which can be formulated in terms of a stochastic differential equation. The spatial structure is included by introducing linear operators, which affect both the oscillations and damping, and by choosing the appropriate spatial covariance structure of the driving time-white noise process. With the choice of the linear operators as partial differential operators, the resonator model becomes a stochastic partial differential equation, which is compatible with infinite-dimensional Kalman filtering. The resulting infinite-dimensional Kalman filtering problem allows for a computationally efficient solution as the computational cost scales linearly with measurements in the temporal dimension. This framework is applied to weather prediction and to physiological noise elimination in functional magnetic resonance imaging brain data.

[Codes available on request. For implementation details or questions, feel free to contact me.]

Sarkka-etal-2013-learning

Simo Särkkä, Arno Solin, Jouni Hartikainen (2013). Spatiotemporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing. IEEE Signal Processing Magazine, 30(4):51–61.

Spatiotemporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing

Simo Särkkä, Arno Solin, Jouni Hartikainen

Gaussian process-based machine learning is a powerful Bayesian paradigm for nonparametric nonlinear regression and classification. In this article, we discuss connections of Gaussian process regression with Kalman filtering and present methods for converting spatiotemporal Gaussian process regression problems into infinite-dimensional state-space models. This formulation allows for use of computationally efficient infinite-dimensional Kalman filtering and smoothing methods, or more general Bayesian filtering and smoothing methods, which reduces the problematic cubic complexity of Gaussian process regression in the number of time steps into linear time complexity. The implication of this is that the use of machine learning models in signal processing becomes computationally feasible, and it opens the possibility to combine machine learning techniques with signal processing methods.

Sarkka-Solin-etal-2012-DRIFTER

Simo Särkkä, Arno Solin, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, Simo Vanni, Fa-Hsuan Lin (2012). Dynamic Retrospective Filtering of Physiological Noise in BOLD fMRI: DRIFTER. NeuroImage, 60(2):1517–1527.

Dynamic Retrospective Filtering of Physiological Noise in BOLD fMRI: DRIFTER

Simo Särkkä, Arno Solin, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, Simo Vanni, Fa-Hsuan Lin

In this article we introduce the DRIFTER algorithm, which is a new model based Bayesian method for retrospective elimination of physiological noise from functional magnetic resonance imaging (fMRI) data. In the method, we first estimate the frequency trajectories of the physiological signals with the interacting multiple models (IMM) filter algorithm. The frequency trajectories can be estimated from external reference signals, or if the temporal resolution is high enough, from the fMRI data. The estimated frequency trajectories are then used in a state space model in combination of a Kalman filter (KF) and Rauch-Tung-Striebel (RTS) smoother, which separates the signal into an activation related cleaned signal, physiological noise, and white measurement noise components. Using experimental data, we show that the method outperforms the RETROICOR algorithm if the shape and amplitude of the physiological signals change over time.

Conference Proceedings

submitted

Santiago Cortés, Arno Solin, and Juho Kannala (2018). Deep learning based speed estimation for constraining strapdown inertial navigation on smartphones. Accepted for publication in the IEEE International Workshop on Machine Learning for Signal Processing (MLSP). Aalborg, Denmark.

Deep learning based speed estimation for constraining strapdown inertial navigation on smartphones

Santiago Cortés, Arno Solin, and Juho Kannala

Strapdown inertial navigation systems are sensitive to the quality of the data provided by the accelerometer and gyroscope. Low-grade IMUs in handheld smart-devices pose a problem for inertial odometry on these devices. We propose a scheme for constraining the inertial odometry problem by complementing non-linear state estimation by a CNN-based deep-learning model for inferring the momentary speed based on a window of IMU samples. We show the feasibility of the model using a wide range of data from an iPhone, and present proof-of-concept results for how the model can be combined with an inertial navigation system for three-dimensional inertial navigation.

submitted

Ari Heljakka, Arno Solin, and Juho Kannala (2018). Recursive chaining of reversible image-to-image translators for face aging. Accepted for publication in Advanced Concepts for Intelligent Vision Systems (ACIVS). Poitiers, France.

Recursive Chaining of Reversible Image-to-image Translators for Face Aging

Ari Heljakka, Arno Solin, and Juho Kannala

This paper addresses the modeling and simulation of progressive changes over time, such as human face aging. By treating the age phases as a sequence of image domains, we construct a chain of transformers that map images from one age domain to the next. Leveraging recent adversarial image translation methods, our approach requires no training samples of the same individual at different ages. Here, the model must be flexible enough to translate a child face to a young adult, and all the way through the adulthood to old age. We find that some transformers in the chain can be recursively applied on their own output to cover multiple phases, compressing the chain. The structure of the chain also unearths information about the underlying physical process. We demonstrate the performance of our method with precise and intuitive metrics, and visually match with the face aging state-of-the-art.

submitted

Santiago Cortés, Arno Solin, Esa Rahtu, and Juho Kannala (2018). ADVIO: An authentic dataset for visual-inertial odometry. Accepted for publication in European Conference on Computer Vision (ECCV).

ADVIO: An authentic dataset for visual-inertial odometry

Santiago Cortés, Arno Solin, Esa Rahtu, and Juho Kannala

The lack of realistic and open benchmarking datasets for pedestrian visual-inertial odometry has made it hard to pinpoint differences in published methods. Existing datasets either lack a full six degree-of-freedom ground-truth or are limited to small spaces with optical tracking systems. We take advantage of advances in pure inertial navigation, and develop a set of versatile and challenging real-world computer vision benchmark sets for visual-inertial odometry. For this purpose, we have built a test rig equipped with an iPhone, a Google Pixel Android phone, and a Google Tango device. We provide a wide range of raw sensor data that is accessible on almost any modern-day smartphone together with a high-quality ground-truth track. We also compare resulting visual-inertial tracks from Google Tango, ARCore, and Apple ARKit with two recent methods published in academic forums. The data sets cover both indoor and outdoor cases, with stairs, escalators, elevators, office environments, a shopping mall, and metro station.

submitted

Hannes Nickisch, Arno Solin, and Alexander Grigorievskiy (2018). State Space Gaussian Processes with Non-Gaussian Likelihood. Accepted for publication in International Conference on Machine Learning (ICML).

State Space Gaussian Processes with Non-Gaussian Likelihood

Hannes Nickisch, Arno Solin, and Alexander Grigorievskiy

We provide a comprehensive overview and tooling for GP modeling with non-Gaussian likelihoods using state space methods. The state space formulation allows for solving one-dimensional GP models in O(n) time and memory complexity. While existing literature has focused on the connection between GP regression and state space methods, the computational primitives allowing for inference using general likelihoods in combination with the Laplace approximation (LA), variational Bayes (VB), and assumed density filtering (ADF, a.k.a. single-sweep expectation propagation, EP) schemes has been largely overlooked. We present means of combining the efficient O(n) state space methodology with existing inference methods. We extend existing methods, and provide unifying code implementing all approaches.

submitted

Arno Solin, Santiago Cortes, Esa Rahtu, and Juho Kannala (2018). Inertial Odometry on Handheld Smartphones. Accepted for publication in Proceedings of the International Conference on Information Fusion (FUSION).

Inertial Odometry on Handheld Smartphones

Arno Solin, Santiago Cortes, Esa Rahtu, and Juho Kannala

submitted

Manon Kok and Arno Solin (2018). Scalable Magnetic Field SLAM in 3D Using Gaussian Process Maps. Accepted for publication in Proceedings of the International Conference on Information Fusion (FUSION).

Winner of the Best Paper Award at FUSION 2018!

Scalable Magnetic Field SLAM in 3D Using Gaussian Process Maps

Manon Kok and Arno Solin

submitted

Santiago Cortes, Arno Solin, and Juho Kannala (2018). Robust Gyroscope-Aided Camera Self-Calibration. Accepted for publication in Proceedings of the International Conference on Information Fusion (FUSION).

Robust Gyroscope-Aided Camera Self-Calibration

Santiago Cortes, Arno Solin, and Juho Kannala

Camera calibration for estimating the intrinsic parameters and lens distortion is a prerequisite for various monocular vision applications including feature tracking and video stabilization. This application paper proposes a model for estimating the parameters on the fly by fusing gyroscope and camera data, both readily available in modern day smartphones. The model is based on joint estimation of visual feature positions, camera parameters, and the camera pose, the movement of which is assumed to follow the movement predicted by the gyroscope. Our model assumes the camera movement to be free, but continuous and differentiable, and individual features are assumed to stay stationary. The estimation is performed online using an extended Kalman filter, and it is shown to outperform existing methods in robustness and insensitivity to initialization. We demonstrate the method using simulated data and empirical data from an iPad.

Solin-Cortes-Rahtu-Kannala-2018-WACV

Arno Solin, Santiago Cortes, Esa Rahtu, and Juho Kannala (2018). PIVO: Probabilistic inertial-visual odometry for occlusion-robust navigation. In IEEE Winter Conference on Applications of Computer Vision (WACV), pages 616–625. Lake Tahoe, NV, USA.

PIVO: Probabilistic Inertial-Visual Odometry for Occlusion-Robust Navigation

Arno Solin, Santiago Cortes, Esa Rahtu, and Juho Kannala

This paper presents a novel method for visual-inertial odometry. The method is based on an information fusion framework employing low-cost IMU sensors and the monocular camera in a standard smartphone. We formulate a sequential inference scheme, where the IMU drives the dynamical model and the camera frames are used in coupling trailing sequences of augmented poses. The novelty in the model is in taking into account all the cross-terms in the updates, thus propagating the inter-connected uncertainties throughout the model. Stronger coupling between the inertial and visual data sources leads to robustness against occlusion and feature-poor environments. We demonstrate results on data collected with an iPhone and provide comparisons against the Tango device and using the EuRoC data set.

Submitted

Eric Malmi, Aristides Gionis, and Arno Solin (2018). Computationally inferred genealogical networks uncover long-term trends in assortative mating. Proceedings of the International World Wide Web Conference (WWW), pages 883–892. Lyon, France.

Winner of the Best Paper Award at Young Demographers Conference!

Computationally inferred genealogical networks uncover long-term trends in assortative mating

Eric Malmi, Aristides Gionis, and Arno Solin

Genealogical networks, also known as family trees or population pedigrees, are commonly studied by genealogists wanting to know about their ancestry, but they also provide a valuable resource for disciplines such as digital demography, genetics, and computational social science. These networks are typically constructed by hand through a very time-consuming process, which requires comparing large numbers of historical records manually. We develop computational methods for automatically inferring large-scale genealogical networks. A comparison with human-constructed networks attests to the accuracy of the proposed methods. To demonstrate the applicability of the inferred large-scale genealogical networks, we present a longitudinal analysis on the mating patterns observed in a network. This analysis shows a consistent tendency of people choosing a spouse with a similar socioeconomic status, a phenomenon known as assortative mating. Interestingly, we do not observe this tendency to consistently decrease (nor increase) over our study period of 150 years.

Karvonen-etal-2017-ISMRM

Toni Karvonen, Arno Solin, Ángel F. García-Fernández, Filip Tronarp, Simo Särkkä, and Fa-Hsuan Lin (2017). Where is physiological noise lurking in k-space? Number 5300. Proceedings of ISMRM 2017, 25th Annual Meeting & Exhibition. The International Society for Magnetic Resonance in Medicine, Honolulu, HI, USA. Appeared as abstract and e-poster.

Where is Physiological Noise Lurking in k-Space?

Toni Karvonen, Arno Solin, Ángel F. García-Fernández, Filip Tronarp, Simo Särkkä, and Fa-Hsuan Lin
[See PDF for abstract]
submitted

Arno Solin, Simo Särkkä, Juho Kannala, and Esa Rahtu (2016). Terrain navigation in the magnetic landscape: Particle filtering for indoor positioning. Proceedings of the European Navigation Conference (ENC). Pages 1–9. IEEE.

Terrain navigation in the magnetic landscape: Particle filtering for indoor positioning

Arno Solin, Simo Särkkä, Juho Kannala, and Esa Rahtu

Variations in the ambient magnetic field can be used as features in indoor positioning and navigation. We describe a technique for map matching where the pedestrian movement is matched to a map of the magnetic landscape. The map matching algorithm is based on a particle filter, a recursive Monte Carlo method, and follows the classical terrain matching framework used in aircraft positioning and navigation. A recent probabilistic Gaussian process regression based method for modeling the ambient magnetic field is employed in the framework. The feasibility of this terrain matching approach is demonstrated in a simple real-life indoor positioning example, where both the mapping and positioning is done using a smartphone device.

submitted

Andreas Svensson, Arno Solin, Simo Särkkä, and Thomas B. Schön (2016). Computationally efficient Bayesian learning of Gaussian process state space models. Proceedings of the Nineteenth International Conference on Artifcial Intelligence and Statistics (AISTATS). JMLR W&CP, 51:213–221.

Computationally efficient Bayesian learning of Gaussian process state space models

Andreas Svensson, Arno Solin, Simo Särkkä and Thomas B. Schön

Svensson-Schon-Solin-Sarkka-2015

Andreas Svensson, Thomas B. Schön, Arno Solin, Simo Särkkä (2015). Nonlinear state space model identification using a regularized basis function expansion. Proceedings of the 6th IEEE International Workshop on Com- putational Advances in Multi-Sensor Adaptive Processing (CAMSAP). Pages 481–484. IEEE.

Nonlinear state space model identification using a regularized basis function expansion

Andreas Svensson, Thomas B. Schön, Arno Solin, Simo Särkkä

This paper is concerned with black-box identification of nonlinear state space models. By using a basis function expansion within the state space model, we obtain a flexible structure. The model is identified using an expectation maximization approach, where the states and the parameters are updated iteratively in such a way that a maximum likelihood estimate is obtained. We use recent particle methods with sound theoretical properties to infer the states, whereas the model parameters can be updated using closed-form expressions by exploiting the fact that our model is linear in the parameters. Not to over-fit the flexible model to the data, we also propose a regularization scheme without increasing the computational burden. Importantly, this opens up for systematic use of regularization in nonlinear state space models. We conclude by evaluating our proposed approach on one simulation example and two real-data problems.

Malmi-Solin-Gionis-2015-ECML-PKDD

Eric Malmi, Arno Solin and Aristides Gionis (2015). The blind leading the blind: Network-based location estimation under uncertainty. Machine Learning and Knowledge Discovery in Databases (ECML PKDD). Lecture Notes in Computer Science, 9285;406–421.

The blind leading the blind: Network-based location estimation under uncertainty

Eric Malmi, Arno Solin and Aristides Gionis

We propose a probabilistic method for inferring the geographical locations of linked objects, such as users in a social network. Unlike existing methods, our model does not assume that the exact locations of any subset of the linked objects, like neighbors in a social network, are known. The method efficiently leverages prior knowledge on the locations, resulting in high geolocation accuracies even if none of the locations are initially known. Experiments are conducted for three scenarios: geolocating users of a location-based social network, geotagging historical church records, and geotagging Flickr photos. In each experiment, the proposed method outperforms two state-of-the-art network-based methods. Furthermore, the last experiment shows that the method can be employed not only to network-based but also to content-based location estimation.

Malmi-Solin-Gionis-2015-ICCSS

Eric Malmi, Arno Solin and Aristides Gionis (2015). Reconstructing and analyzing family trees: Towards longitudinal computational social science. Accepted for publication at the International Conference on Computational Social Science (ICCSS). Appeared as abstract and poster.

Reconstructing and analyzing family trees: Towards longitudinal computational social science

Eric Malmi, Arno Solin and Aristides Gionis

Solin-Sarkka-2015-AISTATS

Arno Solin and Simo Särkkä (2015). State space methods for efficient inference in Student-t process regression. Proceedings of the Eighteenth International Conference on Artifcial Intelligence and Statistics (AISTATS). JMLR W&CP, 38:885–893.

State space methods for efficient inference in Student-t process regression

Arno Solin and Simo Särkkä

The added flexibility of Student-t processes (TPs) over Gaussian processes (GPs) robustifies inference in outlier-contaminated noisy data. The uncertainties are better accounted for than in GP regression, because the predictive covariances explicitly depend on the training observations. For an entangled noise model, the canonical-form TP regression problem can be solved analytically, but the naive TP and GP solutions share the same cubic computational cost in the number of training observations. We show how a large class of temporal TP regression models can be reformulated as state space models, and how a forward filtering and backward smoothing recursion can be derived for solving the inference analytically in linear time complexity. This is a novel finding that generalizes the previously known connection between Gaussian process regression and Kalman filtering to more general elliptical processes and non-Gaussian Bayesian filtering. We derive this connection, demonstrate the benefits of the approach with examples, and finally apply the method to empirical data.

Solin-Sarkka-2014-MLSP-competition

Arno Solin and Simo Särkkä (2014). The 10th annual MLSP competition: First place. Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing (MLSP). Reims, France.

Winner of the 2014 Kaggle Schizophrenia Classification Challenge.

The 10th annual MLSP competition: First place

Arno Solin and Simo Särkkä

The goal of the MLSP 2014 Schizophrenia Classification Challenge was to automatically diagnose subjects with schizophrenia based on multimodal features derived from their magnetic resonance imaging (MRI) brain scans. This challenge took place between June 5 and July 20, 2014, and was organized on Kaggle. We present how this classification problem can be solved in terms of a Bayesian machine learning paradigm known as Gaussian process (GP) classification. The proposed solution achieved an AUC score of 0.928, and it ranked first on the Kaggle private leaderboard.

Solin-Sarkka-2014-MLSP

Arno Solin and Simo Särkkä (2014). Gaussian quadratures for state space approximation of scale mixtures of squared exponential covariance functions. Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing (MLSP). Reims, France.

Gaussian Quadratures for State Space Approximation of Scale Mixtures of Squared Exponential Covariance Functions

Arno Solin and Simo Särkkä

Stationary one-dimensional Gaussian process models in machine learning can be reformulated as state space equations. This reduces the cubic computational complexity of the naive full GP solution to linear with respect to the number of training data points. For infinitely differentiable covariance functions the representation is an approximation. In this paper, we study a class of covariance functions that can be represented as a scale mixture of squared exponentials. We show how the generalized Gauss–Laguerre quadrature rule can be employed in a state space approximation in this class. The explicit form of the rational quadratic covariance function approximation is written out, and we demonstrate the results in a regression and log-Gaussian Cox process study.

Kokkala-Solin-Sarkka-2014

Juho Kokkala, Arno Solin and Simo Särkkä (2014). Expectation Maximization Based Parameter Estimation by Sigma-Point and Particle Smoothing. Proceedings of the 17th International Conference on Information Fusion (FUSION). Salamanca, Spain.

Expectation Maximization Based Parameter Estimation by Sigma-Point and Particle Smoothing

Juho Kokkala, Arno Solin and Simo Särkkä

We consider parameter estimation in non-linear state space models by using expectation–maximization based numerical approximations to likelihood maximization. We present a unified view of approximative EM algorithms that use either sigma-point or particle smoothers to evaluate the integrals involved in the expectation step of the EM method, and compare these methods to direct likelihood maximization. For models that are linear in parameters and have additive noise, we show how the maximization step of the EM algorithm is available in closed form. We compare the methods using simulated data, and discuss the differences between the approximations.

Solin-Sarkka-2014-AISTATS

Arno Solin and Simo Särkkä (2014). Explicit Link Between Periodic Covariance Functions and State Space Models. Proceedings of the Seventeenth International Conference on Artifcial Intelligence and Statistics (AISTATS). JMLR W&CP, 33:904–912.

Explicit Link Between Periodic Covariance Functions and State Space Models

Arno Solin and Simo Särkkä

This paper shows how periodic covariance functions in Gaussian process regression can be reformulated as state space models, which can be solved with classical Kalman filtering theory. This reduces the problematic cubic complexity of Gaussian process regression in the number of time steps into linear time complexity. The representation is based on expanding periodic covariance functions into a series of stochastic resonators. The explicit representation of the canonical periodic covariance function is written out and the expansion is shown to uniformly converge to the exact covariance function with a known convergence rate. The framework is generalized to quasi-periodic covariance functions by introducing damping terms in the system and applied to two sets of real data. The approach could be easily extended to non-stationary and spatio-temporal variants.

We are happy to announce that this method is now incorporated into the GPstuff software package for Matlab and Octave.

Solin-etal-2014-ISMRM

Arno Solin, Simo Särkkä, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, Fa-Hsuan Lin (2014). Catching Physiological Noise: Comparison of DRIFTER in Image and k-Space. Number 3068. Proceedings of ISMRM 2014, 22nd Annual Meeting & Exhibition. The International Society for Magnetic Resonance in Medicine, Milan, Italy. Appeared as abstract and poster.

Catching Physiological Noise: Comparison of DRIFTER in Image and k-Space

Arno Solin, Simo Särkkä, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, and Fa-Hsuan Lin
[See PDF for abstract]
Sarkka-Solin-2013-SCIA

Simo Särkkä and Arno Solin (2013). Continuous-Space Gaussian Process Regression and Generalized Wiener Filtering with Application to Learning Curves. Image Analysis. Lecture Notes in Computer Science, volume 7944, 2013, pages 172–181. (SCIA 2013)

Continuous-Space Gaussian Process Regression and Generalized Wiener Filtering with Application to Learning Curves

Simo Särkkä and Arno Solin

Gaussian process regression is a machine learning paradigm, where the regressor functions are modeled as realizations from an a priori Gaussian process model. We study abstract continuous-space Gaussian regression problems where the training set covers the whole input space instead of consisting of a finite number of distinct points. The model can be used for analyzing theoretical properties of Gaussian process regressors. In this paper, we present the general continuous-space Gaussian process regression equations and discuss their close connection with Wiener filtering. We apply the results to estimation of learning curves as functions of training set size and input dimensionality.

Solin-etal-2013-HBM

Arno Solin, Enrico Glerean, Simo Särkkä (2013). Time–Frequency Dynamics of Brain Connectivity by Stochastic Oscillator Models and Kalman Filtering. Number 1877. The 19th Annual Meeting of the Organization for Human Brain Mapping, Seattle, WA, USA. Appeared as abstract and poster.

Time–Frequency Dynamics of Brain Connectivity by Stochastic Oscillator Models and Kalman Filtering

Arno Solin, Enrico Glerean, Simo Särkkä
[See PDF for abstract]
Solin-etal-2013-ISMRM

Arno Solin, Simo Särkkä, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, Simo Vanni, Fa-Hsuan Lin (2013). Volumetric Space-Time Structure of Physiological Noise in BOLD fMRI. Number 3353. Proceedings of ISMRM 2013, 21st Annual Meeting & Exhibition. The International Society for Magnetic Resonance in Medicine, Salt Lake City, UT, US. Appeared as abstract and e-poster.

Volumetric Space-Time Structure of Physiological Noise in BOLD fMRI

Arno Solin, Simo Särkkä, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, Simo Vanni, and Fa-Hsuan Lin
[See PDF for abstract]
Sarkka-Solin-etal-2012-ISMRM

Simo Särkkä, Arno Solin, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, Simo Vanni, Fa-Hsuan Lin (2012). Identification of spatio-temporal oscillatory signal structure in cerebral hemodynamics using DRIFTER. Number 2842. Proceedings of ISMRM 2012, 20th Annual Meeting & Exhibition. The International Society for Magnetic Resonance in Medicine, Melbourne. Appeared as abstract and e-poster.

Identification of Spatio-Temporal Oscillatory Signal Structure in Cerebral Hemodynamics Using DRIFTER

Simo Särkkä, Arno Solin, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, Simo Vanni, and Fa-Hsuan Lin
[See PDF for abstract]
Sarkka-Solin-2012-SYSID

Simo Särkkä and Arno Solin (2012). On continuous-discrete cubature Kalman filtering. Proceedings of SYSID 2012, 16th IFAC Symposium on System Identification, Brussels, pages 1210–1215.

On Continuous-Discrete Cubature Kalman Filtering

Simo Särkkä, Arno Solin

This paper is concerned with application of cubature integration methods to Kalman filtering of discretely observed non-linear stochastic continuous-time systems. We compare two recently proposed variants of the continuous-discrete cubature Kalman filter (CD-CKF), which differ in the order how the discretization and the Gaussian approximation are done. Aside with theoretical analysis we test the performance of the different variants in a simulated application. The results indicate that the relative advantages of the approaches are application specific and neither one is unconditionally better than the other.

Sarkka-Nummenmaa-etal-2012-ISMRM

Simo Särkkä, Aapo Nummenmaa, Arno Solin, Aki Vehtari, Thomas Witzel, Toni Auranen, Simo Vanni, Matti S. Hämälainen, Fa-Hsuan Lin (2011). Dynamical statistical modeling of physiological noise for fast BOLD fMRI. Number 3592. Proceedings of ISMRM 2011, 19th Annual Meeting & Exhibition. The International Society for Magnetic Resonance in Medicine, Montreal. Appeared as abstract and e-poster.

Dynamical statistical modeling of physiological noise for fast BOLD fMRI

Simo Särkkä, Aapo Nummenmaa, Arno Solin, Aki Vehtari, Thomas Witzel, Toni Auranen, Simo Vanni, Matti S. Hämälainen, and Fa-Hsuan Lin
[See PDF for abstract]

Submitted Papers

submitted

Ari Heljakka, Arno Solin, and Juho Kannala. Pioneer Networks: Progressively Growing Generative Autoencoder. Submitted.

Pioneer Networks: Progressively Growing Generative Autoencoder

Ari Heljakka, Arno Solin, and Juho Kannala

submitted

Arno Solin, Pasi Jylänki, Jaakko Kauramäki, Tom Heskes, Marcel A. J. van Gerven, and Simo Särkkä. Regularizing Solutions to the MEG Inverse Problem Using Space-Time Separable Covariance Functions. Submitted.

Regularizing Solutions to the MEG Inverse Problem Using Space-Time Separable Covariance Functions

Arno Solin, Pasi Jylänki, Jaakko Kauramäki, Tom Heskes, Marcel A. J. van Gerven, Simo Särkkä

submitted

Arno Solin and Simo Särkkä. Hilbert Space Methods for Reduced-Rank Gaussian Process Regression. Submitted.

Hilbert Space Methods for Reduced-Rank Gaussian Process Regression

Arno Solin and Simo Särkkä

This paper proposes a novel scheme for reduced-rank Gaussian process regression. The method is based on an approximate series expansion of the covariance function in terms of an eigenfunction expansion of the Laplace operator in a compact subset of R^d. On this approximate eigenbasis the eigenvalues of the covariance function can be expressed as simple functions of the spectral density of the Gaussian process, which allows the GP inference to be solved under a computational cost scaling as O(nm^2) (initial) and O(m^3) (hyperparameter learning) with m basis functions and n data points. The approach also allows for rigorous error analysis with Hilbert space theory, and we show that the approximation becomes exact when the size of the compact subset and the number of eigenfunctions go to infinity. The expansion generalizes to Hilbert spaces with an inner product which is defined as an integral over a specified input density. The method is compared to previously proposed methods theoretically and through empirical tests with simulated and real data.

Theses

Solin-thesis

Arno Solin (2016). Stochastic Differential Equation Methods for Spatio-Temporal Gaussian Process Regression. Doctoral dissertation, Aalto University. Helsinki, Finland.

Stochastic Differential Equation Methods for Spatio-Temporal Gaussian Process Regression

Arno Solin

Gaussian processes (GPs) are widely used tools for non-parametric probabilistic modelling in machine learning, spatial statistics, and signal processing. Their strength lies in flexible model specification, where prior beliefs of the model functions are encoded by the GP model. This way they can also be interpreted as specifying a probability distribution over the space of functions. In signal processing GPs are typically represented as state-space models, whereas the kernel (covariance function) representation is favoured in machine learning. Under the kernel formalism, the naïve solution to a GP regression problem scales cubically in the number of data points, which makes the approach computationally infeasible for large data sets.

This work explores the link between the two representations, which enables the use of efficient sequential Kalman filtering based methods for solving the inference problem. These methods have linear time complexity with respect to the number of data points. The interest is in presenting an explicit connection between a large class of covariance functions and state- space models. This is done for one-dimensional (temporal) covariance functions and linear time-invariant stochastic differential equations. This class of models covers a wide range of both stationary and non-stationary GP models for encoding, for example, continuity, smoothness, or periodicity. The framework also extends to spatio-temporal models, where the GP is represented as an evolution type stochastic partial differential equation and inference conducted by infinite-dimensional Kalman filtering methods. Both separable and non- separable models are considered, and implementation techniques for numerical solutions are discussed.

The link between stochastic differential equations and standard covariance functions widens the applicability of Gaussian processes in combination with mechanistic physical differential equation models. Temporal and spatio-temporal Gaussian process models are useful in a multitude of data-intensive applications. Examples in this work include brain image analysis, weather modelling, financial forecasting, and tracking applications.

Solin-2012-MSc-thesis

Arno Solin (2012). Hilbert Space Methods in Infinite-Dimensional Kalman Filtering. Master's thesis, instructor Dr. Simo Särkkä, supervisor Prof. Jouko Lampinen. School of Science, Aalto University.

Hilbert Space Methods in Infinite-Dimensional Kalman Filtering

Arno Solin

Many physical and biological processes include both spatial and temporal features. Spatio-temporal modeling under the machine learning paradigm of Gaussian process (GP) regression has demonstrated prominent results. However, the appealing Bayesian treatment by GP regression is often difficult in practical problems due to computational complexity.

In this thesis, methods for writing spatio-temporal Gaussian process regression as infinite-dimensional Kalman filtering and Rauch-Tung-Striebel smoothing problems are presented. These scale linearly with respect to the number of time steps as opposed to the cubic scaling of the direct GP solution. Spatio-temporal covariance functions are formulated as infinite-dimensional stochastic differential equations. Furthermore, it is presented how infinite-dimensional models can be combined with a finite number of observations to an approximative solution. For this, a truncated eigenfunction expansion of the Laplace operator is formed in various domains, of which the n-dimensional hypercube and hypersphere are explicitly written out.

The approach in this thesis is primarily application-driven, and therefore three real-world case studies are presented as proof of concept. The feasibility of infinite-dimensional Kalman filtering is demonstrated by forming a spatio-temporal resonator model which is applied to temperature data in two spatial dimensions, and a novel way of modeling the space-time structure of physiological noise in functional brain imaging data is considered in both two and three spatial dimensions.

Solin-2010-BSc-thesis

Arno Solin (2010). Cubature Integration Methods in Non-Linear Kalman Filtering and Smoothing. Bachelor's thesis, Instructor Dr. Simo Särkkä, supervisor Prof. Harri Ehtamo. Faculty of information and natural sciences, Aalto University.

Cubature Integration Methods in Non-Linear Kalman Filtering and Smoothing

Arno Solin

Optimal estimation problems arise in various different settings where indirect noisy observations are used to determine the underlying state of a time-varying system. For systems with non-linear dynamics there exist various methods that extend linear filtering and smoothing methods to handle non-linearities.

In this thesis the non-linear optimal estimation framework is presented with the help of an assumed density approach. The Gaussian integrals that arise in this setting are solved using two different cubature integration methods. Cubature integration extends the weighted sum approach from univariate quadrature methods to multidimensional cubature methods. In this thesis the focus is put on two methods that use deterministically chosen sigma points to form the desired approximation. The Gauss-Hermite rule uses a simple product rule method to fill the multidimensional space with cubature points, whereas the spherical-radial rule uses invariant theory to diminish the number of points by utilizing symmetries.

The derivations of the Gauss-Hermite and spherical-radial rules are reviewed. The corresponding non-linear Kalman filter and Rauch-Tung-Striebel smoother algorithms are presented. Additionally, the relation between the cubature rules and the unscented transformation is discussed. It is also shown that the cubature Kalman filter can be interpreted as a refinement of the unscented Kalman filter.

Course material

SDE

Simo Särkkä and Arno Solin. Applied Stochastic Differential Equations. Lecture notes of the course Becs-114.4202 Special Course in Computational Engineering II held in Autumn 2014.

Applied Stochastic Differential Equations

Simo Särkkä and Arno Solin

Lecture notes of the course Becs-114.4202 Special Course in Computational Engineering II held in Autumn 2014.

Miscellaneous

Solin-2012-erkka

Arno Solin (2012). Tracking and Elimination of Periodic Noise in fMRI Using Bayesian Inference. Semester project undertaken as the course Mat-2.4108. Instructor Dr. Simo Särkkä, supervisor Prof. Harri Ehtamo. Department of Mathematics and Systems Analysis, Aalto University.

Tracking and Elimination of Periodic Noise in fMRI Using Bayesian Inference

Arno Solin

In this work the formulation of DRIFTER is studied. It is a model-based Bayesian method for estimation and removal of physiological noise, such as cardiac- and respiration-induced effects, in functional magnetic resonance imaging (fMRI). The method is due to Särkkä, Solin and colleagues, and this study aims to broaden some aspects discussed in the original article.

The background of the DRIFTER method is presented by providing some insight in stochastic resonator models and modeling of quasi- periodic signals. The method is based on first estimating frequency trajectories of physiological noise components by using the interactive multiple models (IMM) algorithm and reference signals. A retrospective image-based state space formulation is used to estimate the noise-induced components in the fMRI signal with Kalman filtering and Rauch-Tung-Striebel smoothing. Separate estimates are gained for cardiac- and respiration-induced noise components, the cleaned blood oxygenation level dependent (BOLD) brain signal and a white measurement noise estimate. In this study, two aspects of using the DRIFTER method are studied in more detail: the effect of slow sampling rates and signal aliasing, and an example of estimation of frequencies without physiological reference signals. A brief analysis of these questions is provided and the results are discussed.

Solin-Katsigiannis-Parkkila-Torabi-2011

Arno Solin, Michail Katsigiannis, Kaisa Parkkila, Bahare Torabihaghighi (2011). Modeling long-term electricity prices. Course work, Department of Mathematics and Systems Analysis, Aalto University. Undertaken as the course Mat-2.4177 in collaboration with Danske Markets, supervisor Prof. Ahti Salo.

Modeling long-term electricity prices

Arno Solin, Michail Katsigiannis, Kaisa Parkkila, and Bahare Torabihaghighi

Electricity has special characteristics which make it different from other commodities. Since it cannot be stored, standard procedures for modelling the forward price fails to give accurate estimates for its behaviour.

The objective of the study is to consider the dynamics and characteristics of electricity forward contracts. In order to capture some of this dynamical structure, we develop a CVAR (Cointegrated Vector Autoregressive) model for the generic electricity forward prices with time to maturity from two to five years and a delivery period of one year. The parameters of the CVAR model structure are estimated for all contracts separately.

In this study, we suggest that the contracts with longer time to maturity adapt market changes more slowly even though the changes follow the same general pattern. No seasonal components were identified, but the volatility of contract prices was dependent on the changes in the previous weeks. Our results suggest that there was a structural break in the price dynamics in mid-2008 that restricts the use of CVAR models. Additionally, the risk premium of the forward contracts is discussed. The preliminary results suggest that the risk premium is not constant over different times to maturity. This conclusion is heavily affected by the amount of studied data.

The impact of the evolving CO2 market and possible long-term structural changes are also discussed.

Malmi-Sainio-Solin-2010-MCM

Eric Malmi, Jussi Sainio, Arno Solin (2010). Natural Bayesian killers. In the one-weekend mathematical competition Mathematical Contest in Modeling by COMAP (SIAM/NSA/INFORMS). Awarded ‘Meritorious Winner’.

Natural Bayesian Killers

Eric Malmi, Jussi Sainio, and Arno Solin

In criminology, the real life CSI currently faces an interesting phase, where mathematicians are entering the cold-blooded field of forensics. Geographic profiling uses existing geocoded data to predict, solve and prevent crime. In this study, we examine probability distance strategies and a Bayesian approach used to predict the residential location of a serial offender. We compare different distance decay functions used for estimating the probability density of the residential location. Additionally, we derive a Bayesian approach for predicting the next target location of the serial offender following the formulation of O’Leary [2009].

For the measure of performance, we choose the search cost ranking. The model is tested using the leave-one-out cross-validation with a small example dataset that consists of serial murder cases. Different strategies of combining the six different methods are discussed. The tests suggest that the best predictions are achieved using the Bayesian approach with exponential decay function. However, this result is not statistically significant due to the lack of test data.

Solin-2005-paattotyo

Arno Solin (2005). Etäisyysmittalaite, joka kartoittaa ympärillään olevaa tilaa lasermittauksen avulla ja luo tietokoneelle kolmiulotteisen mallin. Päättötyö (final project in upper secondary high-school, in Finnish). Ohjaajina Stefan Johansson (Åbo Akademin hiukkaskiihdytinlaboratorio) ja Marjo Lahti. Turun Steiner-koulun lukio, Turku.

Etäisyysmittalaite, joka kartoittaa ympärillään olevaa tilaa lasermittauksen avulla ja luo tietokoneelle kolmiulotteisen mallin (literal translation: A device which measures its surrounding space with the help of a laser beam and creates a three-dimensional model of it)

Arno Solin

A report on how to build the mechanics, electronic circuits and program the drivers for a divide which measures its surrounding space with the help of a digital camera and a laser beam and constructs a three-dimensional model of the measurements.

This project report is in Finnish, and it was undertaken as the final project (an individual task that the student has interest in) during my last year in upper secondary high-school.

Software Packages

I have been involved in the development of the following software packages and Matlab/Python toolboxes. Codes for individual papers are listed under the papers.

Invited Talks

Slides (and other material such as videos) available upon request. This list is only sporadically updated.

Teaching

I have been involved in teaching at Aalto University as course assistant/lecturer on the following courses: