I'm an associate professor of computer games at Aalto University. My mission is to improve health through exergames and digitally augmented sports that promote physical activity. My research interests include exergame design and gameplay innovation through technologies such as AI, procedural animation, and computer vision. The video above features a quick analog exergame design exercise I did with my kids; see below for digital projects.

My background is in human-computer interaction, machine learning, computer vision, electronics, and audio signal processing technology, and before that in sound design, demoscene, short films, and theatre. I used to be fluent in AHDL, embedded system design, and several assembler dialects, but nowadays I mostly implement my game and interaction prototypes using C# and Unity 3D, and technology using Python and sometimes C++. I'm a hybrid researcher / engineer / designer.

Selected projects

	Mixed reality trampoline games (2012-2019). One of the main themes of my research is "mixed reality empowerment", i.e., boosting the player's abilities both digitally in the game world and in the real world using devices such as trampolines, in order to motivate exercise. CHI 2013 WIP video, CHI 2013 WIP paper, CHI 2014 HCI in sport workshop paper, IJCSS 2014 continuation paper, ValoJump commercial trampoline game system developed by Valo Motion, a spin-off from my group. The latest ValoJump game is Super Stomp, world's first multiplayer trampoline game, also accepted to CHI PLAY 2019.
	Exaggerating Avatar Flexibility in Virtual Reality (2018), a CHI PLAY 2018 paper that extends the superhero movement experiences or movement empowerment theme above to flexibility. Paper, Video.
	Proximal Policy Optimization with Covariance Matrix Adaptation (2018), a novel deep reinforcement learning method that combines ideas from Proximal Policy Optimization (PPO) and Covariance Matrix Adaptation Evolution Strategy (CMA-ES).
	Simulating humanoid climbing (2017-2019). Climbing is a complex motor skill that requires both full-body coordination and planning long movement sequences. We solve the problem using a combination of search, optimization, and learning approaches, controlling a physically based simulated climber without any reference movement data. SIGGRAPH 2017 paper & video, SCA 2018 paper & video, CoG 2019 paper (to appear).
	Combining online tree search and machine learning for continuous control (2017-2018). Our 3D characters learn stable locomotion gaits in less than a minute of wall-clock time on a modest 4-core computer. SCA 2017 paper & video, TVCG 2018 continuation paper.
	Predictive Physics Simulation in Game Mechanics (2017). This CHI PLAY 2017 paper explores the novel design space of game mechanics based on reproducible forward simulation of the game physics. We propose and evaluate a number of novel game prototypes, including both turn-based and real-time gameplay and simple and complex simulations. Paper, Video.
	Augmented Climbing Wall (2013-2016), a research project into digitally augmented sports, and now a spinoff startup. Viral video success with dozes of millions of views, CHI video showcase 2014, CHI 2016 paper with honorary mention, featured at wired.com, TechCrunch, New Scientist, Gizmodo... My role: supervising prof., computer vision for the first pilot (now continued by Joni Vähämäki), parts of the design and user studies, writing the CHI paper.
	Come As You Are (2016) An award-winning Finnish Game Jam game with the theme "ritual". It's a short audio puzzle game that plays with the notions of 1) circular motion as oscillation, 2) movement and sound synchrony and harmony. There's also some procedural sound and animation. My role: game mechanics design, programming, procedural audio, shaders. Video, Code.
	Gravity as a design resource (2015). Together with colleagues from Melbourne and Nottingham, we analyzed how gravity could be used in novel ways for digital and physical game design. The paper got a best paper honorary mention at CHI PLAY 2015, and also gave us a reason to try indoor skydiving =)
	Online Control of Simulated Humanoids Using Particle Belief Propagation (2015). A novel online trajectory optimization / tree search approach for real-time continuous control of 3D characters. Extends the work below to more diverse movements such as controllable locomotion, juggling, and climbing over obstacles. SIGGRAPH 2015 Paper, Video 1, Video 2, Windows demo.
	Online Motion Synthesis Using Sequential Monte Carlo (2014). We present the the world's first "active ragdolls" that can get back up when pushed over, in real-time, without breaking the physics simulation or using unrealistic forces, and without using any animation data or a time-consuming offline learning phase. SIGGRAPH 2014 Paper, Video, Windows demo.
	Benefits of 3D topos for information sharing and planning in rock climbing. We created an interactive photogrammetry model of the popular Koivusaari boulder in Helsinki, with various information overlays for climbers, and conducted a user study. Sports Technology 2014 paper, Windows executable
	Kung-Fu Live (2010), and Kung-Fu High Impact (2011). KFL is a PlayStation 3 combat game played by punching and kicking in front of the PS3 Eye camera, implemented using custom computer vision based body tracking. Kung-Fu High Impact is a sequel for Xbox 360 Kinect. My role: head of the computer vision team (wrote about 50k lines of the code myself), design and implementation of player's movement, controls & physics.
	Kick Ass Kung-Fu (2004-2007), an immersive game installation that transforms computer gaming into an immersive physical performance. The game lets you experience Kung-Fu movie action and aesthetics by merging the real and the virtual with a perceptive user interface. My role: Lead design & original idea, technology, music. Additional music by eBottle. Trailer , CHI 2005 paper. Here's also a full-page ad from WIRED magazine that invited the game to NextFest three times.
	Preceding Kick Ass Kung-Fu, I experimented with video "magic" mirrors that provide novel visual feedback for sports training. The simplest example is to point a camera at yourself and watch a delayed camera feed on screen while training. For example, when you practice spin kicks, you can't keep your eyes on a mirror when spinning around, so it's far more convenient to do the kick and then stop to watch the delayed view. Check out the video for more examples.
	Musical interface research (2004-2006). I've played various musical instruments, and during my doctoral studies I participated in the ALMA project. Together with Teemu Mäki-Patola, we had a NIME 2006 paper about an augmented djembe drum (my part: original idea, computer vision, sound synthesis), and an ICMC 2004 paper about how the perception of interface latency varies with musical content.
	Animaatiokone (2002), an installation for learning about stop-motion animation. The installation is built on an animation software and user interface I designed and implemented. Trailer, CHI 2004 paper.
	Kukakumma Muumaassa (2001-ca. 2004), the first game where one could express yourself as the avatar using both body movements and voice. This was my M.A. thesis project and also part of my doctoral research. My responsibilities were concept and interaction design (with Johanna Höysniemi), sound design and technology. Video, Game design case paper from 2002. During the process, Höysniemi and I also developed some game user research methodology utilizing peer tutoring and Wizard of Oz prototyping. I proposed the methods and developed the software needed for the studies.
	I've also done some audio DSP development, such as this lookahead limiter algorithm presented at DAFx 2002. I think this is quite an elegant solution to the problem. Paper, Matlab code.
	Software that turns a laptop keyboard into a velocity sensitive midi keyboard or drumpad, from around 2002. The sound input of the built-in microphone is analyzed and the loudness of keystrokes is mapped to loudness of synthesized sounds. Video, Windows executable.
	Design and programming of the sound engine of the Soittopeli (PlaySingMusic) musical edutainment game (in collaboration with Tero Tolonen) by Elmorex Ltd. Released in 2000, the game was the first to let you control an avatar by singing and learn to sing with help of immediate visual feedback. I also developed some other singing game concepts and later wrote a retrospective paper about singing game design. In 2003, the project was resurrected as Staraoke that won an interactive Emmy.
	MediaObjects, a modular audio/video processing tool. It's the computer vision and audio processing engine in Animaatiokone, Kukakumma, and Kick Ass Kung-Fu (back in early 2000's). I was going to release it, but never had the time, and now there's no point as pd has become the de facto tool in that space. As a curiosity, here's a gallery of self portraits generated when testing various video processing algorithms with a webcam.
	Design of soundscapes and audio technology for the Semana Santa exhibition at Helinä Rautavaara's museum, summer 2000. I designed and implemented a 20-channel 3D interactive sound animation system for behavioural modelling of large acoustic spaces. Homepage: http://mlab.uiah.fi/semanasanta.
	Electronics for Jalmari Helander's (of Rare Exports fame) early short film Maximillian Tarzan (1999), using a 8-bit Atmel microcontroller.
	Retordion (1998?), a distortion plugin for vst-compatible audio software such as Steinberg Cubase. It was conceived accidentally when I was experimenting with nonlinearities inside feedback loops in a Griesinger style allpass reverb algorithm. The result is quite out of control weird distortion. To use, download the dll to the vst plugin folder of your computer. Sound example: original and distorted drums.

Research and Design Awards & Shortlistings

Best paper award at ACM CHI 2019, with Maximus Kaos, Nick Graham and Ryan Rhodes

Finnish Applied Game of the Year (“vuoden hyötypeli”) award for Augmented Climbing Wall, 2017, with Valo Motion (my role: computer vision and parts of the design of the first version, supervising professor in commercialization phase)

Finnish Game Jam 2016 award for the jam game Come as you are. My role: game mechanics design, programming, procedural audio.

Augmented Climbing Wall shortlisted for Unity Awards 2016

Best paper honorary mention at ACM CHI 2016. With Raine Kajastila and Leo Holsti.

Best paper honorary mention at CHI PLAY 2015. With Raine Kajastila, Joe Marshall, Rich Byrne, Floyd Mueller.

Best Low-Cost, Minimally-Intrusive Solution award in 3DUI contest, Orlando, U.S., 2013. In collaboration with Tuukka Takala and Meeri Mäkäräinen

Kung-Fu High Impact (Virtual Air Guitar Company) won the “Control Design, 2D or Limited 3D” category in the 11th annual National Academy of Video Game Testers and Reviewers awards, 2011.

Kick Ass Kung-Fu (Perttu Hämäläinen, Ari Nykänen, Mikko Lindholm) won the Games Platforms category in Europrix Top Talent multimedia innovation competition in Vienna, Austria, 2004

Tampere Mindtrek Grand Prix for Kick Ass Kung-Fu, Tampere, Finland, 2004

Korjaamo Young Design Award for Animaatiokone (Mikko Lindholm, Perttu Hämäläinen, Ari Nykänen), Helsinki, Finland, 2004

Animaatiokone won the Pikku Kakkonen (best multimedia for children) and non-commercial categories at Tampere Mindtrek competition, Finland, 2003

Prix Spécial du Jury for Animaatiokone in the international Prix Möbius multimedia competition in Athens, Greece, 2003

Kukakumma Muumaassa (Perttu Hämäläinen, Johanna Höysniemi, Teppo Rouvi, Laura Turkki) was shortlisted for the Milia New Talent Competition in Cannes, France, February 2002.

Kukakumma Muumaassa won the Pikku Kakkonen category in Tampere Mindtrek competition (best multimedia for children), 2001

Exhibitions

Kick Ass Kung-Fu at Wired NEXTFEST, Los Angeles, US, 2007

Kukakumma Muumaassa at Ars Electronica Center, Linz, Austria, 2007

Kick Ass Kung-Fu at Karate World Championships, Tampere, Finland, 2006

Animaatiokone and Kukakumma Muumaassa at Ars Electronica campus exhibition, Linz, Austria,2006

Kick Ass Kung-Fu at Wired NEXTFEST, New York, US, 2006

Animaatiokone at Red Sticks international animation festival, Baton Rouge, Louisiana, US, 2006

Kick Ass Kung-Fu at Ultrasound festival, Huddersfield, UK, 2005

Animaatiokone at Japan Pop, Tennis Palace Art Museum, Helsinki, Finland, 2005

Kick Ass Kung-Fu at Wired NEXTFEST, Chicago, US, 2005

Kick Ass Kung-Fu at Tennispalatsi, Helsinki, Finland, 2005

Kick Ass Kung-Fu at pixelACHE festival, Helsinki, Finland, 2005

Kick Ass Kung-Fu at Korjaamo gallery, Helsinki, Finland, 2005

Kick Ass Kung-Fu at Nordic Exceptional Trendshop, Copenhagen, Denmark, 2004

Kick Ass Kung-Fu at Europrix Top Talent, Vienna, Austria, 2004

Kick Ass Kung-Fu at MindTrek, Tampere, Finland, 2004

Kick Ass Kung-Fu at Kiasma Theater, pixelACHE, Helsinki, Finland, 2004

Animaatiokone at Korjaamo gallery, Helsinki, Finland, 2004

Animaatiokone at Animex animation festival, Middlesbrough, UK, 2004

Kukakumma Muumaassa at Musta tuntuu exhibition, Rauma Art Museum, Rauma, Finland, 2004

Kukakumma Muumaassa at Game On exhibition, Tennis Palace Art Museum, Helsinki, Finland, 2003

Animaatiokone at Nordic Exceptional Trendshop, Copenhagen, Denmark, 2003

Animaatiokone at MindTrek, Tampere, Finland, 2003

Animaatiokone at Prix Möbius International, Athens, Greece, 2003

Animaatiokone at Annecy animation festival, Annecy, France, 2003

Animaatiokone at Tough Eye animation festival, Turku, Finland, 2003

Animaatiokone at Kunsthalle Lophem, Belgium, 2003

Animaatiokone at pixelACHE festival: Kiasma in Helsinki, Finland, Gerschwin Hotel in New York, USA, and SAT in Montreal, Canada, 2003

Animaatiokone at FF media festival, Rovaniemi, Finland, 2003

Animaatiokone at Tampere Film Festival, Tampere, Finland, 2003

Animaatiokone at Prix Möbius Preview Exhibition, Helsinki, Finland, 2003

Kukakumma Muumaassa in Design Museo, Helsinki, Finland, 2002

Animaatiokone at Leffakansio, Helsinki, Finland, 2002

Animaatiokone at Kettupäivät, Helsinki, Finland, 2002

Animaatiokone at Animatricks animation festival, Helsinki, Finland, 2002

Animaatiokone at Mainonnan Viikko, Helsinki, Finland, 2002

Animaatiokone at Love & Anarchy festival, Helsinki, Finland, 2002

Design of interactive soundscapes for Semana Santa exhibition at Helinä Rautavaara's museum, Espoo, Finland, 2000

Invited talks

Summer School on Experimental Play, RMIT University, Australia, 2018

Keynote at GamiFIN 2018, Pori, Finland

Keynote at Next Wave Conference, Seoul, South Korea, 2018

Opening Keynote of ITK conference, Hämeenlinna, Finland, 2013

“Teknologian ja käyttöliittymien mahdollisuudet opetuspeleissä”. Peliakatemia, Helsinki, Finland, 2013

“Designing (player motion in) motion games”, ManseDanse Festival, Tampere, Finland, 2011

“Exertion interfaces and digital sports”, University of Art and Design Helsinki Media Lab, Finland, 2006

“Exertion interfaces and digital sports”, University of Pori, Finland, 2006

“Designing Kick Ass Kung-Fu”, Flimmer international film festival, Norrköping, Sweden, 2005

“Mixed reality martial arts”, International symposium on Mixed Reality, RIXC Media Space, Riga, Latvia, 2005

“Interface driven edutainment”, University of Turku, Pori, Finland, 2004

“Computer vision based interfaces”, Experimental interaction workshop, University of Art and Design, Helsinki, Finland, 2004

“Experimental interaction in games”, Game On exhibition, Tennispalatsi art museum, Helsinki, Finland, 2003

“Computer vision based interfaces”, Experimental interaction workshop, University of Art and Design, Helsinki, Finland, 2003

“Experimental Interaction in games”, Pelin kosketus seminar day, YLE, Finland, 2002

“Designing QuiQui’s Giant Bounce”, Jyväskylän kesä festival, Jyväskylä, Finland, 2002

Audiovisual

DISCLAIMER: the work below is more than 20 years old now, from around 1995-2001.

I've done audiovisual stuff for the interactive projects above, such as the Kick Ass Kung-Fu trailer (music & editing), Animaatiokone info movie (sound & music) and Kukakumma Muumaassa intro animation soundtrack. Here's also some samples of my non-interactive audiovisual work.

	Sound and music for Animatricks 2001 trailer (mpeg1).
	Music for Golfpiste.fi TV commercial.
	Make-up, sound design, music and cinematography for the J5 surreal soap by the OUBS campus television of the Helsinki University of Technology. The series follows the life of students and mythical figures in a student community. Here's a piece of music, most of which has been lost.
	Cinematography, lights & music for the OUBS 1996 jingle. The jingle was based on my idea, but having just watched Greenaway's Pillow Book, it was maybe not that original...
	Music for the Speksi student theater orchestra, years 1996-1997. Samples: The speksi 1997 theme (synthesizer version used for rehearsing, I don't have a live recording) and a live recording of my arrangement of "You Don't Have To Say You Love Me".
	Make-up effects & part of the music of the short film Let's Fucking Die (1997). Fun fact: LFD was a first year film-school project of award-winning cinematographer Mika Orasmaa (Iron Sky, Rare Exports etc.). The opening scene features two pieces of my music. Here's also a climactic hymn where the hero meets the villain (the start overlaps preceding guitar score), trailer music (guitar melody played & composed by Henri Penttinen, noisy due to a damaged DAT tape) and the ending credits music.