Part 3: Biometrics As User Control

The idea of biometrics in XR isn’t a terribly new one. Methods like heart rate and eye tracking are pretty popular in the research community. Academic researchers have been using physiological signals to measure changes in user states for decades and some AR applications have begun integrating biosignals to inform changes to the system. What is a newer idea is using biosignals as a user controlled input method.

Imagine using your heart rate as a way to intentionally interact with a virtual space. Does that sound difficult? I probably would have had some doubts too, if I hadn’t tried Jamzone’s Stressjam demo at first hand. This is part three biometrics as user control.

At first glance, Stressjam looks like a platformer game in the same vein as those you might have played years ago, taking place in a jungle filled with ruins and tasking the player with navigating and exploring. What makes it unique is the heart monitor you have to strap to your chest before playing. While in the environment, gauges are visible on both controls and indicate the level of stress the system is sensing in that moment. The catch is that certain objects and items can only be interacted with when the player is calm or stressed. Unlike most mindfulness-based experiences, Stressjam requires the player not just to recognize when they are stressed but to control that stress with intent. Such an approach is unique in the space and it’s surprisingly easy to pick up. After about three minutes of acclimation time, I was pretty proficient at modulating the stress response and running around pyramids.

Why is this exciting? Because the control methods we’ve been using for years in domains like transportation or gaming can only go so far in XR. Using too many buttons makes learning too hard. Having a small number of buttons control a variety of actions causes creeping featurism. Visual-spatial processing can be easily overloaded in a VR environment and adding more spatial-manual demands only exacerbates this. Novel systems of control, like heart rate (or, more accurately, heart rate variability), might represent a way to bypass this limitation as well as teach users effective methods of stress management. It may also drive the industry to pay more attention to methods of biometric measurement. The majority of biometric measurement techniques are decades old. At best, they are largely sensitive to changes in user mental state but not terrible diagnostic of the changes being elicited (i.e., valence of emotional state, magnitude of reactions, etc). At worst, they are perpetually used techniques, long since retired in most scientific circles for questionable validity. New attention to methods of physiological measure may motivate the largest players in tech to begin investing in more accurate and lightweight versions of biometric measurement, allowing users to use naturalistic control methods. Imagine if a headset could tell what direction your pupils were facing and shift you field of view to accommodate or if blood oxygenation level could impact the speed at which an experience drove forward? With any amount of luck, you won’t have to for long.

__

This article is part of a three part series on VR AR XR, this was Part 3: Biometrics as User Control. Read the other parts:

Part 1: AI Smart Avatars
Part 2: Blockchain-Based VR