# Arno Solin

D.Sc. (Tech.), Academy of Finland Postdoctoral Researcher

I hold a position as an Academy of Finland Postdoctoral Researcher with the Department of Computer Science at Aalto University. I am also a Technical Advisor for IndoorAtlas Ltd, and I do projects as an independent consultant.

Currently, I work as a part of Prof. Juho Kannala's group at Aalto. During my doctoral studies, my affiliations were with the Department of Computer Science and the Department of Neuroscience and Biomedical Engineering, Aalto University. I was part of the Bayesian Statistical Methods Group lead by Prof. Aki Vehtari and my instructor was Prof. Simo Särkkä.

I have a broad interest in various fields of applied mathematics and machine learning—including, but not limited to, the research projects below that I am involved in.

## Research Activities

**Bayesian Inference Methods for Temporal, Spatial and Spatio-Temporal Systems**
- Bayesian inference methods for stochastic dynamic systems.
- Linear and non-linear Kalman filtering.
- Gaussian process regression and infinite-dimensional Kalman filtering for spatio-temporal models.

**Applications**
- Target tracking and indoor localization.
- Modeling of dynamical systems (e.g. the weather).
- Brain imaging (fMRI and MEG) data reconstruction.

## Publications

The PDF preprints below are draft versions of the journal articles.
They are here to give an opportunity to check the relevance of the
articles before purchasing the final articles from the publisher.
Complete bibliography information is given in the BibTeX format
for each published article/report.

My Google Scholar profile.

### Doctoral Dissertation

__Arno Solin__ (2016). *Stochastic Differential Equation Methods for Spatio-Temporal Gaussian Process Regression*. Doctoral dissertation, Aalto University. Helsinki, Finland.

### Stochastic Differential Equation Methods for Spatio-Temporal Gaussian Process Regression

**Arno Solin**
Gaussian processes (GPs) are widely used tools for non-parametric probabilistic modelling in machine learning, spatial statistics, and signal processing. Their strength lies in flexible model specification, where prior beliefs of the model functions are encoded by the GP model. This way they can also be interpreted as specifying a probability distribution over the space of functions. In signal processing GPs are typically represented as state-space models, whereas the kernel (covariance function) representation is favoured in machine learning. Under the kernel formalism, the naïve solution to a GP regression problem scales cubically in the number of data points, which makes the approach computationally infeasible for large data sets.

This work explores the link between the two representations, which enables the use of efficient sequential Kalman filtering based methods for solving the inference problem. These methods have linear time complexity with respect to the number of data points. The interest is in presenting an explicit connection between a large class of covariance functions and state- space models. This is done for one-dimensional (temporal) covariance functions and linear time-invariant stochastic differential equations. This class of models covers a wide range of both stationary and non-stationary GP models for encoding, for example, continuity, smoothness, or periodicity. The framework also extends to spatio-temporal models, where the GP is represented as an evolution type stochastic partial differential equation and inference conducted by infinite-dimensional Kalman filtering methods. Both separable and non- separable models are considered, and implementation techniques for numerical solutions are discussed.

The link between stochastic differential equations and standard covariance functions widens the applicability of Gaussian processes in combination with mechanistic physical differential equation models. Temporal and spatio-temporal Gaussian process models are useful in a multitude of data-intensive applications. Examples in this work include brain image analysis, weather modelling, financial forecasting, and tracking applications.

### Journal Articles

Juho Kokkala, __Arno Solin__ and Simo Särkkä. *Sigma-Point Filtering and Smoothing Based Parameter Estimation in Nonlinear Dynamic Systems.* Journal of Advances in Information Fusion, 11(1):15–30.

### Sigma-Point Filtering and Smoothing Based Parameter Estimation in Nonlinear Dynamic Systems

Juho Kokkala, **Arno Solin** and Simo Särkkä
We consider approximate maximum likelihood parameter estimation in nonlinear state-space models. We discuss both direct optimization of the likelihood and expectation-maximization (EM). For EM, we also give closed-form expressions for the maximization step in a class of models that are linear in parameters and have additive noise. To obtain approximations to the filtering and smoothing distributions needed in the likelihood-maximization methods, we focus on using Gaussian filtering and smoothing algorithms that employ sigma-points to approximate the required integrals. We discuss different sigma-point schemes based on the third, fifth, seventh, and ninth order unscented transforms and the Gauss-Hermite quadrature rule. We compare the performance of the methods in two simulated experiments: a univariate nonlinear growth model as well as tracking of a maneuvering target. In the experiments, we also compare against approximate likelihood estimates obtained by particle filtering and extended Kalman filtering based methods. The experiments suggest that the higher-order unscented transforms may in some cases provide more accurate estimates.

### Infinite-dimensional Bayesian filtering for detection of quasiperiodic phenomena in spatiotemporal data

**Arno Solin** and Simo Särkkä
This paper introduces a spatiotemporal resonator model and an inference method for detection and estimation of nearly periodic temporal phenomena in spatiotemporal data. The model is derived as a spatial extension of a stochastic harmonic resonator model, which can be formulated in terms of a stochastic differential equation. The spatial structure is included by introducing linear operators, which affect both the oscillations and damping, and by choosing the appropriate spatial covariance structure of the driving time-white noise process. With the choice of the linear operators as partial differential operators, the resonator model becomes a stochastic partial differential equation, which is compatible with infinite-dimensional Kalman filtering. The resulting infinite-dimensional Kalman filtering problem allows for a computationally efficient solution as the computational cost scales linearly with measurements in the temporal dimension. This framework is applied to weather prediction and to physiological noise elimination in functional magnetic resonance imaging brain data.

[Codes available on request. For implementation details or questions, feel free to contact me.]

Simo Särkkä, __Arno Solin__, Jouni Hartikainen (2013). *Spatiotemporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing*. IEEE Signal Processing Magazine, 30(4):51–61.

### Spatiotemporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing

Simo Särkkä, **Arno Solin**, Jouni Hartikainen
Gaussian process-based machine learning is a powerful Bayesian paradigm
for nonparametric nonlinear regression and classification. In this article,
we discuss connections of Gaussian process regression with Kalman
filtering and present methods for converting spatiotemporal Gaussian process
regression problems into infinite-dimensional state-space models. This
formulation allows for use of computationally efficient infinite-dimensional
Kalman filtering and smoothing methods, or more general Bayesian filtering
and smoothing methods, which reduces the problematic cubic complexity of
Gaussian process regression in the number of time steps into linear time
complexity. The implication of this is that the use of machine learning
models in signal processing becomes computationally feasible, and it opens
the possibility to combine machine learning techniques with signal processing
methods.

Simo Särkkä, __Arno Solin__, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, Simo Vanni, Fa-Hsuan Lin (2012). *Dynamic Retrospective Filtering of Physiological Noise in BOLD fMRI: DRIFTER.* NeuroImage, 60(2):1517–1527.

### Dynamic Retrospective Filtering of Physiological Noise in BOLD fMRI: DRIFTER

Simo Särkkä, **Arno Solin**, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, Simo Vanni, Fa-Hsuan Lin
In this article we introduce the DRIFTER algorithm, which is a new model based Bayesian method for retrospective elimination of physiological noise from functional magnetic resonance imaging (fMRI) data. In the method, we first estimate the frequency trajectories of the physiological signals with the interacting multiple models (IMM) filter algorithm. The frequency trajectories can be estimated from external reference signals, or if the temporal resolution is high enough, from the fMRI data. The estimated frequency trajectories are then used in a state space model in combination of a Kalman filter (KF) and Rauch-Tung-Striebel (RTS) smoother, which separates the signal into an activation related cleaned signal, physiological noise, and white measurement noise components. Using experimental data, we show that the method outperforms the RETROICOR algorithm if the shape and amplitude of the physiological signals change over time.

### Conference Proceedings

Toni Karvonen, __Arno Solin__, Ángel F. García-Fernández, Filip Tronarp,
Simo Särkkä, and Fa-Hsuan Lin (2017). *Where is physiological noise lurking in
**k*-space? Number 5300. Proceedings of ISMRM 2017,
25th Annual Meeting & Exhibition. The International Society for
Magnetic Resonance in Medicine, Honolulu, HI, USA. Appeared as abstract and e-poster.

### Where is Physiological Noise Lurking in *k*-Space?

Toni Karvonen, **Arno Solin**, Ángel F. García-Fernández, Filip Tronarp, Simo Särkkä, and Fa-Hsuan Lin
[See PDF for abstract]

__Arno Solin__, Simo Särkkä, Juho Kannala, and Esa Rahtu. *Terrain navigation in the magnetic landscape: Particle filtering for indoor positioning*. Proceedings of the European Navigation Conference (ENC). Pages 1–9. IEEE.

### Terrain navigation in the magnetic landscape: Particle filtering for indoor positioning

**Arno Solin**, Simo Särkkä, Juho Kannala, and Esa Rahtu
Variations in the ambient magnetic field can be used as features in indoor positioning and navigation. We describe a technique for map matching where the pedestrian movement is matched to a map of the magnetic landscape. The map matching algorithm is based on a particle filter, a recursive Monte Carlo method, and follows the classical terrain matching framework used in aircraft positioning and navigation. A recent probabilistic Gaussian process regression based method for modeling the ambient magnetic field is employed in the framework. The feasibility of this terrain matching approach is demonstrated in a simple real-life indoor positioning example, where both the mapping and positioning is done using a smartphone device.

Andreas Svensson, __Arno Solin__, Simo Särkkä, and Thomas B. Schön. *Computationally efficient Bayesian learning of Gaussian process state space models*. Proceedings of the Nineteenth International Conference on Artifcial Intelligence and Statistics (AISTATS). JMLR W&CP, 51:213–221.

### Computationally efficient Bayesian learning of Gaussian process state space models

Andreas Svensson, **Arno Solin**, Simo Särkkä and Thomas B. Schön
Andreas Svensson, Thomas B. Schön, __Arno Solin__, Simo Särkkä (2015). *Nonlinear state space model identification using a regularized basis function expansion*. Proceedings of the 6th IEEE International Workshop on Com- putational Advances in Multi-Sensor Adaptive Processing (CAMSAP). Pages 481–484. IEEE.

### Nonlinear state space model identification using a regularized basis function expansion

Andreas Svensson, Thomas B. Schön, **Arno Solin**, Simo Särkkä
This paper is concerned with black-box identification of nonlinear state space models. By using a basis function expansion within the state space model, we obtain a flexible structure. The model is identified using an expectation maximization approach, where the states and the parameters are updated iteratively in such a way that a maximum likelihood estimate is obtained. We use recent particle methods with sound theoretical properties to infer the states, whereas the model parameters can be updated using closed-form expressions by exploiting the fact that our model is linear in the parameters. Not to over-fit the flexible model to the data, we also propose a regularization scheme without increasing the computational burden. Importantly, this opens up for systematic use of regularization in nonlinear state space models. We conclude by evaluating our proposed approach on one simulation example and two real-data problems.

Eric Malmi, __Arno Solin__ and Aristides Gionis (2015). *The blind leading the blind: Network-based location estimation under uncertainty*. Machine Learning and Knowledge Discovery in Databases (ECML PKDD). Lecture Notes in Computer Science, 9285;406–421.

### The blind leading the blind: Network-based location estimation under uncertainty

Eric Malmi, **Arno Solin** and Aristides Gionis
We propose a probabilistic method for inferring the geographical locations of linked objects, such as users in a social network. Unlike existing methods, our model does not assume that the exact locations of any subset of the linked objects, like neighbors in a social network, are known. The method efficiently leverages prior knowledge on the locations, resulting in high geolocation accuracies even if none of the locations are initially known. Experiments are conducted for three scenarios: geolocating users of a location-based social network, geotagging historical church records, and geotagging Flickr photos. In each experiment, the proposed method outperforms two state-of-the-art network-based methods. Furthermore, the last experiment shows that the method can be employed not only to network-based but also to content-based location estimation.

Eric Malmi, __Arno Solin__ and Aristides Gionis (2015). *Reconstructing and analyzing family trees: Towards longitudinal computational social science*. Accepted for publication at the International Conference on Computational Social Science (ICCSS). To appear as abstract and poster.

### Reconstructing and analyzing family trees: Towards longitudinal computational social science

Eric Malmi, **Arno Solin** and Aristides Gionis
__Arno Solin__ and Simo Särkkä (2015). *State space methods for efficient inference in Student-t process regression*. Proceedings of the Eighteenth International Conference on Artifcial Intelligence and Statistics (AISTATS). JMLR W&CP, 38:885–893.

### State space methods for efficient inference in Student-*t* process regression

**Arno Solin** and Simo Särkkä
The added flexibility of Student-t processes (TPs) over Gaussian processes (GPs) robustifies inference in outlier-contaminated noisy data. The uncertainties are better accounted for than in GP regression, because the predictive covariances explicitly depend on the training observations. For an entangled noise model, the canonical-form TP regression problem can be solved analytically, but the naive TP and GP solutions share the same cubic computational cost in the number of training observations. We show how a large class of temporal TP regression models can be reformulated as state space models, and how a forward filtering and backward smoothing recursion can be derived for solving the inference analytically in linear time complexity. This is a novel finding that generalizes the previously known connection between Gaussian process regression and Kalman filtering to more general elliptical processes and non-Gaussian Bayesian filtering. We derive this connection, demonstrate the benefits of the approach with examples, and finally apply the method to empirical data.

### The 10th annual MLSP competition: First place

**Arno Solin** and Simo Särkkä
The goal of the MLSP 2014 Schizophrenia Classification Challenge was to automatically diagnose subjects with schizophrenia based on multimodal features derived from their magnetic resonance imaging (MRI) brain scans. This challenge took place between June 5 and July 20, 2014, and was organized on Kaggle. We present how this classification
problem can be solved in terms of a Bayesian machine learning paradigm known as Gaussian process (GP) classification.
The proposed solution achieved an AUC score of 0.928, and it ranked first on the Kaggle private leaderboard.

__Arno Solin__ and Simo Särkkä (2014). *Gaussian quadratures for state space approximation of scale mixtures of squared exponential covariance functions*. Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing (MLSP). Reims, France.

### Gaussian Quadratures for State Space Approximation of Scale Mixtures of Squared Exponential Covariance Functions

**Arno Solin** and Simo Särkkä
Stationary one-dimensional Gaussian process models in machine learning can be reformulated as state space equations. This reduces the cubic computational complexity of the naive full GP solution to linear with respect to the number of training data points. For infinitely differentiable covariance functions the representation is an approximation. In this paper, we study a class of covariance functions that can be represented as a scale mixture of squared exponentials. We show how the generalized Gauss–Laguerre quadrature rule can be employed in a state space approximation in this class. The explicit form of the rational quadratic covariance function approximation is written out, and we demonstrate the results in a regression and log-Gaussian Cox process study.

Juho Kokkala, __Arno Solin__ and Simo Särkkä (2014). *Expectation Maximization Based Parameter Estimation by Sigma-Point and Particle Smoothing*. Proceedings of the 17th International Conference on Information Fusion (FUSION). Salamanca, Spain.

### Expectation Maximization Based Parameter Estimation by Sigma-Point and Particle Smoothing

Juho Kokkala, **Arno Solin** and Simo Särkkä
We consider parameter estimation in non-linear state space models by using expectation–maximization based
numerical approximations to likelihood maximization. We present a unified view of approximative EM algorithms that use either sigma-point or particle smoothers to evaluate the integrals involved in the expectation step of the EM method, and compare these methods to direct likelihood maximization. For models that are linear in parameters and have additive noise, we show how the maximization step of the EM algorithm is available in closed form. We compare the methods using simulated data, and discuss the differences between the approximations.

__Arno Solin__ and Simo Särkkä (2014). *Explicit Link Between Periodic Covariance Functions and State Space Models*. Proceedings of the Seventeenth International Conference on Artifcial Intelligence and Statistics (AISTATS). JMLR W&CP, 33:904–912.

### Explicit Link Between Periodic Covariance Functions and State Space Models

**Arno Solin** and Simo Särkkä
This paper shows how periodic covariance functions in Gaussian process regression
can be reformulated as state space models, which can be solved with classical Kalman
filtering theory. This reduces the problematic cubic complexity of Gaussian process
regression in the number of time steps into linear time complexity. The representation
is based on expanding periodic covariance functions into a series of stochastic
resonators. The explicit representation of the canonical periodic covariance function
is written out and the expansion is shown to uniformly converge to the exact covariance
function with a known convergence rate. The framework is generalized to quasi-periodic
covariance functions by introducing damping terms in the system and applied to two sets
of real data. The approach could be easily extended to non-stationary and spatio-temporal variants.

We are happy to anounce that this method is now incorporated into the GPstuff software package for Matlab and Octave.

__Arno Solin__, Simo Särkkä, Aapo Nummenmaa, Aki Vehtari, Toni Auranen,
Fa-Hsuan Lin (2014). *Catching Physiological Noise: Comparison of DRIFTER
in Image and **k*-Space. Number 3068. Proceedings of
ISMRM 2014, 22nd Annual Meeting & Exhibition. The International Society for
Magnetic Resonance in Medicine, Milan, Italy. Appeared as abstract and poster.

### Catching Physiological Noise: Comparison of DRIFTER in Image and *k*-Space

**Arno Solin**, Simo Särkkä, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, and Fa-Hsuan Lin
[See PDF for abstract]

Simo Särkkä and __Arno Solin__ (2013). *Continuous-Space Gaussian Process Regression and Generalized Wiener Filtering with Application to Learning Curves*.
Image Analysis. Lecture Notes in Computer Science, volume 7944, 2013, pages 172–181. (SCIA 2013)

### Continuous-Space Gaussian Process Regression and Generalized Wiener Filtering with Application to Learning Curves

Simo Särkkä and **Arno Solin**
Gaussian process regression is a machine learning paradigm, where the
regressor functions are modeled as realizations from an a priori Gaussian
process model. We study abstract continuous-space Gaussian regression
problems where the training set covers the whole input space instead of
consisting of a finite number of distinct points. The model can be used
for analyzing theoretical properties of Gaussian process regressors.
In this paper, we present the general continuous-space Gaussian process
regression equations and discuss their close connection with Wiener
filtering. We apply the results to estimation of learning curves as
functions of training set size and input dimensionality.

__Arno Solin__, Enrico Glerean, Simo Särkkä (2013). *Time–Frequency Dynamics of Brain Connectivity by Stochastic Oscillator Models and Kalman Filtering*. Number 1877. The 19th Annual Meeting of the Organization for Human Brain Mapping, Seattle, WA, USA. Appeared as abstract and poster.

### Time–Frequency Dynamics of Brain Connectivity by Stochastic Oscillator Models and Kalman Filtering

**Arno Solin**, Enrico Glerean, Simo Särkkä
[See PDF for abstract]

__Arno Solin__, Simo Särkkä, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, Simo Vanni, Fa-Hsuan Lin (2013). *Volumetric Space-Time Structure of Physiological Noise in BOLD fMRI*. Number 3353. Proceedings of ISMRM 2013, 21st Annual Meeting & Exhibition. The International Society for Magnetic Resonance in Medicine, Salt Lake City, UT, US. Appeared as abstract and e-poster.

### Volumetric Space-Time Structure of Physiological Noise in BOLD fMRI

**Arno Solin**, Simo Särkkä, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, Simo Vanni, and Fa-Hsuan Lin
[See PDF for abstract]

Simo Särkkä, __Arno Solin__, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, Simo Vanni, Fa-Hsuan Lin (2012). *Identification of spatio-temporal oscillatory signal structure in cerebral hemodynamics using DRIFTER*. Number 2842. Proceedings of ISMRM 2012, 20th Annual Meeting & Exhibition. The International Society for Magnetic Resonance in Medicine, Melbourne. Appeared as abstract and e-poster.

### Identification of Spatio-Temporal Oscillatory Signal Structure in Cerebral Hemodynamics Using DRIFTER

Simo Särkkä, **Arno Solin**, Aapo Nummenmaa, Aki Vehtari, Toni Auranen, Simo Vanni, and Fa-Hsuan Lin
[See PDF for abstract]

Simo Särkkä and __Arno Solin__ (2012). *On continuous-discrete cubature Kalman filtering*. Proceedings of SYSID 2012, 16th IFAC Symposium on System Identification, Brussels, pages 1210–1215.

### On Continuous-Discrete Cubature Kalman Filtering

Simo Särkkä, **Arno Solin**
This paper is concerned with application of cubature integration methods to Kalman filtering of discretely observed non-linear stochastic continuous-time systems. We compare two recently proposed variants of the continuous-discrete cubature Kalman filter (CD-CKF), which differ in the order how the discretization and the Gaussian approximation are done. Aside with theoretical analysis we test the performance of the different variants in a simulated application. The results indicate that the relative advantages of the approaches are application specific and neither one is unconditionally better than the other.

Simo Särkkä, Aapo Nummenmaa, __Arno Solin__, Aki Vehtari, Thomas Witzel, Toni Auranen, Simo Vanni, Matti S. Hämälainen, Fa-Hsuan Lin (2011). *Dynamical statistical modeling of physiological noise for fast BOLD fMRI*. Number XXXX. Proceedings of ISMRM 2011, 19th Annual Meeting & Exhibition. The International Society for Magnetic Resonance in Medicine, Montreal. Appeared as abstract and e-poster.

### Dynamical statistical modeling of physiological noise for fast BOLD fMRI

Simo Särkkä, Aapo Nummenmaa, **Arno Solin**, Aki Vehtari, Thomas Witzel, Toni Auranen, Simo Vanni, Matti S. Hämälainen, and Fa-Hsuan Lin
[See PDF for abstract]

### Submitted Papers

__Arno Solin__, Santiago Cortes, Esa Rahtu, and Juho Kannala. *PIVO: Probabilistic Inertial-Visual Odometry for Occlusion-Robust Navigation*. Submitted.

### PIVO: Probabilistic Inertial-Visual Odometry for Occlusion-Robust Navigation

**Arno Solin**, Santiago Cortes, Esa Rahtu, and Juho Kannala
__Arno Solin__, Santiago Cortes, Esa Rahtu, and Juho Kannala. *Inertial Odometry on Handheld Smartphones*. Submitted.

### Inertial Odometry on Handheld Smartphones

**Arno Solin**, Santiago Cortes, Esa Rahtu, and Juho Kannala
James Hensman, Nicolas Durrande, and __Arno Solin__. *Variational Fourier features for Gaussian processes*. Submitted.

### Variational Fourier features for Gaussian processes

James Hensman, Nicolas Durrande, and **Arno Solin**
__Arno Solin__, Pasi Jylänki, Jaakko Kauramäki, Tom Heskes, Marcel A. J. van Gerven, and Simo Särkkä. *Regularizing Solutions to the MEG Inverse Problem Using Space-Time Separable Covariance Functions*. Submitted.

### Regularizing Solutions to the MEG Inverse Problem Using Space-Time Separable Covariance Functions

**Arno Solin**, Pasi Jylänki, Jaakko Kauramäki, Tom Heskes, Marcel A. J. van Gerven, Simo Särkkä
__Arno Solin__, Manon Kok, Niklas Wahlström, Thomas B. Schön, and Simo Särkkä. *Modeling and interpolation of the ambient magnetic field by Gaussian processes*. Submitted.

### Modeling and interpolation of the ambient magnetic field by Gaussian processes

**Arno Solin**, Manon Kok, Niklas Wahlström, Thomas B. Schön, and Simo Särkkä
### Hilbert Space Methods for Reduced-Rank Gaussian Process Regression

**Arno Solin** and Simo Särkkä
This paper proposes a novel scheme for reduced-rank Gaussian process regression. The method is based on an approximate series expansion of the covariance function in terms of an eigenfunction expansion of the Laplace operator in a compact subset of R^d. On this approximate eigenbasis the eigenvalues of the covariance function can be expressed as simple functions of the spectral density of the Gaussian process, which allows the GP inference to be solved under a computational cost scaling as O(nm^2) (initial) and O(m^3) (hyperparameter learning) with m basis functions and n data points. The approach also allows for rigorous error analysis with Hilbert space theory, and we show that the approximation becomes exact when the size of the compact subset and the number of eigenfunctions go to infinity. The expansion generalizes to Hilbert spaces with an inner product which is defined as an integral over a specified input density. The method is compared to previously proposed methods theoretically and through empirical tests with simulated and real data.

### M.Sc. and B.Sc. Theses

__Arno Solin__ (2012). *Hilbert Space Methods in Infinite-Dimensional Kalman Filtering.* Master's thesis, instructor Dr. Simo Särkkä, supervisor Prof. Jouko Lampinen. School of Science, Aalto University.

### Hilbert Space Methods in Infinite-Dimensional Kalman Filtering

**Arno Solin**
Many physical and biological processes include both spatial and temporal features. Spatio-temporal modeling under the machine learning paradigm of Gaussian process (GP) regression has demonstrated prominent results. However, the appealing Bayesian treatment by GP regression is often difficult in practical problems due to computational complexity.

In this thesis, methods for writing spatio-temporal Gaussian process regression as infinite-dimensional Kalman filtering and Rauch-Tung-Striebel smoothing problems are presented. These scale linearly with respect to the number of time steps as opposed to the cubic scaling of the direct GP solution. Spatio-temporal covariance functions are formulated as infinite-dimensional stochastic differential equations. Furthermore, it is presented how infinite-dimensional models can be combined with a finite number of observations to an approximative solution. For this, a truncated eigenfunction expansion of the Laplace operator is formed in various domains, of which the n-dimensional hypercube and hypersphere are explicitly written out.

The approach in this thesis is primarily application-driven, and therefore three real-world case studies are presented as proof of concept. The feasibility of infinite-dimensional Kalman filtering is demonstrated by forming a spatio-temporal resonator model which is applied to temperature data in two spatial dimensions, and a novel way of modeling the space-time structure of physiological noise in functional brain imaging data is considered in both two and three spatial dimensions.

__Arno Solin__ (2010). *Cubature Integration Methods in Non-Linear Kalman Filtering and Smoothing*. Bachelor's thesis, Instructor Dr. Simo Särkkä, supervisor Prof. Harri Ehtamo. Faculty of information and natural sciences, Aalto University.

### Cubature Integration Methods in Non-Linear Kalman Filtering and Smoothing

**Arno Solin**
Optimal estimation problems arise in various different settings where indirect noisy observations are used to determine the underlying state of a time-varying system. For systems with non-linear dynamics there exist various methods that extend linear filtering and smoothing methods to handle non-linearities.

In this thesis the non-linear optimal estimation framework is presented with the help of an assumed density approach. The Gaussian integrals that arise in this setting are solved using two different cubature integration methods. Cubature integration extends the weighted sum approach from univariate quadrature methods to multidimensional cubature methods. In this thesis the focus is put on two methods that use deterministically chosen sigma points to form the desired approximation. The Gauss-Hermite rule uses a simple product rule method to fill the multidimensional space with cubature points, whereas the spherical-radial rule uses invariant theory to diminish the number of points by utilizing symmetries.

The derivations of the Gauss-Hermite and spherical-radial rules are reviewed. The corresponding non-linear Kalman filter and Rauch-Tung-Striebel smoother algorithms are presented. Additionally, the relation between the cubature rules and the unscented transformation is discussed. It is also shown that the cubature Kalman filter can be interpreted as a refinement of the unscented Kalman filter.

### Course material

### Applied Stochastic Differential Equations

Simo Särkkä and **Arno Solin**
Lecture notes of the course Becs-114.4202 Special Course in Computational Engineering II held in Autumn 2014.

### Miscellaneous

__Arno Solin__ (2012). *Tracking and Elimination of Periodic Noise in fMRI Using Bayesian Inference*. Semester project undertaken as the course Mat-2.4108. Instructor Dr. Simo Särkkä, supervisor Prof. Harri Ehtamo. Department of Mathematics and Systems Analysis, Aalto University.

### Tracking and Elimination of Periodic Noise in fMRI Using Bayesian Inference

**Arno Solin**
In this work the formulation of DRIFTER is studied. It is a model-based Bayesian method for estimation and removal of physiological noise, such as cardiac- and respiration-induced effects, in functional magnetic resonance imaging (fMRI). The method is due to Särkkä, Solin and colleagues, and this study aims to broaden some aspects discussed in the original article.

The background of the DRIFTER method is presented by providing some insight in stochastic resonator models and modeling of quasi- periodic signals. The method is based on first estimating frequency trajectories of physiological noise components by using the interactive multiple models (IMM) algorithm and reference signals. A retrospective image-based state space formulation is used to estimate the noise-induced components in the fMRI signal with Kalman filtering and Rauch-Tung-Striebel smoothing. Separate estimates are gained for cardiac- and respiration-induced noise components, the cleaned blood oxygenation level dependent (BOLD) brain signal and a white measurement noise estimate.
In this study, two aspects of using the DRIFTER method are studied in more detail: the effect of slow sampling rates and signal aliasing, and an example of estimation of frequencies without physiological reference signals. A brief analysis of these questions is provided and the results are discussed.

__Arno Solin__, Michail Katsigiannis, Kaisa Parkkila, Bahare Torabihaghighi (2011). *Modeling long-term electricity prices*. Course work, Department of Mathematics and Systems Analysis, Aalto University. Undertaken as the course Mat-2.4177 in collaboration with Danske Markets, supervisor Prof. Ahti Salo.

### Modeling long-term electricity prices

**Arno Solin**, Michail Katsigiannis, Kaisa Parkkila, and Bahare Torabihaghighi
Electricity has special characteristics which make it different from other commodities. Since it cannot be stored, standard procedures for modelling the forward price fails to give accurate estimates for its behaviour.

The objective of the study is to consider the dynamics and characteristics of electricity forward contracts. In order to capture some of this dynamical structure, we develop a CVAR (Cointegrated Vector Autoregressive) model for the generic electricity forward prices with time to maturity from two to five years and a delivery period of one year. The parameters of the CVAR model structure are estimated for all contracts separately.

In this study, we suggest that the contracts with longer time to maturity adapt market changes more slowly even though the changes follow the same general pattern. No seasonal components were identified, but the volatility of contract prices was dependent on the changes in the previous weeks. Our results suggest that there was a structural break in the price dynamics in mid-2008 that restricts the use of CVAR models. Additionally, the risk premium of the forward contracts is discussed. The preliminary results suggest that the risk premium is not constant over different times to maturity. This conclusion is heavily affected by the amount of studied data.

The impact of the evolving CO2 market and possible long-term structural changes are also discussed.

Eric Malmi, Jussi Sainio, __Arno Solin__ (2010). *Natural Bayesian killers*. In the one-weekend mathematical competition Mathematical Contest in Modeling by COMAP (SIAM/NSA/INFORMS). Awarded ‘Meritorious Winner’.

### Natural Bayesian Killers

Eric Malmi, Jussi Sainio, and **Arno Solin**
In criminology, the real life CSI currently faces an interesting phase, where mathematicians are entering the cold-blooded field of forensics. Geographic profiling uses existing geocoded data to predict, solve and prevent crime. In this study, we examine probability distance strategies and a Bayesian approach used to predict the residential location of a serial offender. We compare different distance decay functions used for estimating the probability density of the residential location. Additionally, we derive a Bayesian approach for predicting the next target location of the serial offender following the formulation of O’Leary [2009].

For the measure of performance, we choose the search cost ranking. The model is tested using the leave-one-out cross-validation with a small example dataset that consists of serial murder cases. Different strategies of combining the six different methods are discussed. The tests suggest that the best predictions are achieved using the Bayesian approach with exponential decay function. However, this result is not statistically significant due to the lack of test data.

__Arno Solin__ (2005). *Etäisyysmittalaite, joka kartoittaa ympärillään olevaa tilaa lasermittauksen avulla ja luo tietokoneelle kolmiulotteisen mallin*. Päättötyö (final project in upper secondary high-school, in Finnish). Ohjaajina Stefan Johansson (Åbo Akademin hiukkaskiihdytinlaboratorio) ja Marjo Lahti. Turun Steiner-koulun lukio, Turku.

### Etäisyysmittalaite, joka kartoittaa ympärillään olevaa tilaa lasermittauksen avulla ja luo tietokoneelle kolmiulotteisen mallin (literal translation: A device which measures its surrounding space with the help of a laser beam and creates a three-dimensional model of it)

**Arno Solin**
A report on how to build the mechanics, electronic circuits and program the drivers for a divide which measures its surrounding space with the help of a digital camera and a laser beam and constructs a three-dimensional model of the measurements.

This project report is in Finnish, and it was undertaken as the final project (an individual task that the student has interest in) during my last year in upper secondary high-school.

## Software Packages

I have been involved in the development of the following software packages and Matlab/Python toolboxes:

## Teaching

I have been involved in teaching at Aalto University as course assistant/lecturer on the following courses:

- ELEC-E8105 Non-Linear Filtering and Parameter Estimation (5 cr) (2016, together with Dr. Simo Särkkä)
- Becs-114.4610 Special Course in Bayesian Modelling: Bayesian Estimation of Time-Varying Systems (5 cr) (2015, together with Dr. Simo Särkkä)
- Becs-114.4202/Mat-1.C Special Course in Computational Engineering II: Applied Stochastic Differential Equations (3 cr) (2014, together with Dr. Simo Särkkä)
- Becs-114.4610 Special Course in Bayesian Modelling: Bayesian Estimation of Time-Varying Systems (5 cr) (2013, lecturer Dr. Simo Särkkä)
- SCI-A0000/ENG-A1004 Introduction to Studies/Data Systems (2013, Swedish-speaking lecturer)
- T-106.1111 Introduction to Studies and Data Systems (2012, Swedish-speaking lecturer)
- Mat-1.1510 Grundkurs i matematik 1 (10 cr) (2009–2012, "Basic course in mathematics", lecturer prof. Gustaf Gripenberg, taught in Swedish)
- Mat-1.1520 Grundkurs i matematik 2 (10 cr) (2010–2012, "Basic course in mathematics", lecturer Dr. Georg Metsalo, taught in Swedish)