“The real game changer is when regular glasses become smart”
Sweating, facial expressions and increased heart rate. Our bodies send signals about our emotions – signals that can be picked up by sensors. The input can then be used to design our next workout, meal or learning experience. Luis Quintero’s research provides a sneak peek into the future.
“I’m a bit of a tech geek. My research delves into technology that’s not yet in use,” Luis Quintero says smilingly.
He notes that we’ve taken a giant technological leap in the last hundred years. People in the 1920s had mechanical clocks, needed wood to make a cup of coffee, and sent paper letters to communicate with distant friends. Today, we can solve most things with a few clicks on our mobile phones.
But Luis Quintero is interested in the next technological leap – a future that might be hard to imagine.
“We’re moving from two-dimensional screens to three-dimensional, personalised experiences. It’ll take time before the big breakthrough happens, though. I don’t think it will happen in my lifetime.”
Sensors and holograms
So, what changes does Luis Quintero foresee? He paints a picture of an ordinary day, starting with you waking up well-rested. Sensors measure your breathing, body temperature, and brain activity to ensure you’re not disturbed in your deepest sleep phase.
You reach for your smart glasses, which display the news in 3D as you move around the house. The glasses sense your head and eye movements, adapting what’s shown based on your situation and interest.
Breakfast is ready in the kitchen, tailored to match your nutritional needs. During the day, you work out with a personal trainer who appears as a hologram by your side. In the evening, you can wander through a rainforest – in your own living room. Or you can call a relative who will appear as a projection beside you on your couch.
We don’t want to walk around with clanky and expensive VR headsets in public
Luis Quintero is aware that this might sound like science fiction to many of us. Today’s VR headsets are mainly associated with gaming, but he believes the possibilities are much bigger than that.
“The real game changer is when regular glasses become smart. We don’t want to walk around with clanky and expensive VR headsets in public. And soon, those headsets will feel as outdated as clocks that need to be winded up manually.”
In his research, Quintero explores how technology can help us learn new things, be more productive, and improve our health. Data are collected by wearing sensors on our bodies, measuring heart rate, sweating, pupil dilation, head and hand movements, or facial muscle movements. The data show our emotions towards what we’re facing, and our skill levels in completing tasks. When the interpreted information is fed back into the system, the experience can be tailored to your mood or skill level.
One application could be learning how to play tennis. If the physical tennis court is far away, you can start practicing in VR to improve your movements. The system provides feedback and customised training at your level.
Another application might be overcoming arachnophobia. In a digital environment that monitors your stress level, you can be exposed to virtual spiders at a manageable pace.
Meta, Microsoft and Apple are investing heavily
A third application, which Quintero has been experimenting with in his research, involves firearm training for police officers in a VR environment.
“I’ve been building these types of systems for seven years. When I started, VR wasn’t as widespread as it is now. And there was really no commercial value in the technology when I began my doctoral thesis four years ago. But now, major tech companies like Meta and Microsoft are seriously investing, and Apple has announced plans to launch a VR and AR product for home use in 2024,” Luis Quintero explains.
He admits he was a bit naive at the start of his PhD journey.
“I thought it would be as simple as connecting some sensors, building prototypes, and testing. But I quickly realised there were no models that could capture people’s emotions and make adjustments based on them. Many didn’t even think that such models should exist.”
Major tech companies are eager to acquire your data
Quintero himself was convinced that personalised adaptations are necessary. Only then can the experience become personal and more powerful. However, along the way, he has become increasingly aware of the ethical aspects.
“Customised experiences are still superior, for example, in educational situations or in healthcare. You want your doctor to provide a personal diagnosis for you, not a general diagnosis that applies to the population at large.”
“At the same time, personal privacy is at risk when services become hyper-personalised. The major tech companies are eager to acquire your data, but they’re not interested in building better healthcare or education. As researchers, we have a responsibility to make that clear,” says Quintero.
Although he believes the major technological leap may take time, he sees changes coming in the near future.
“The first shift has to do with how we watch TV at home. Instead of focusing on a screen, the entire living room will become a projection surface where we can switch between different views and be in the midst of the action,” says Luis Quintero.
More about the research
Luis Quintero defended his PhD thesis on November 28, 2023, at the Department of Computer and Systems Sciences (DSV) at Stockholm University.
Quintero joined DSV after finishing a master’s degree at Karolinska Institutet, bringing experience from running a startup in his home country of Colombia.
His thesis title is “User Modeling for Adaptive Virtual Reality Experiences: Personalization from Behavioral and Physiological Time Series”. It comprises five scientific articles.
Luis Quintero’s main supervisor during the thesis work was Uno Fors, DSV. Supervisors were Panagiotis Papapetrou, DSV, and Jaakko Hollmén, DSV.
Jefrey Lijffijt, University of Ghent, Belgium, was the opponent during the defence.
Metaverse, immersive and interactive are some of the buzzwords in Luis Quintero’s field of research. The terminology also includes several abbreviations.
VR stands for Virtual Reality. By wearing a VR headset, you see a computer-simulated reality in front of you. Ýou can move around and take action in that world.
AR stands for Augmented Reality. AR technology adds fictional objects to our physical reality. One example is the Pokémon Go app, where players hunt digital monsters placed in the real environment.
MR stands for Mixed Reality. MR integrates VR and AR.
XR stands for Extended Reality. XR is a collective term for VR, AR, and MR.
Text: Åse Karlén
Last updated: January 3, 2024
Source: Department of Computer and Systems Sciences, DSV