Touching virtual worlds with your fingertips is one of the goals of Swiss start-up Sensoryx, which has focused its efforts on haptic devices: developing accessories that can improve our interactions in virtual reality. And to approach the precision of the real world! Meet the co-founder of Sensoryx, Rolf Adelsberger.
Tracking reality with Sensoryx
Dr. Rolf Adelsberger – The idea behind Sensoryx is a desire to work on immersive technologies after my studies at ETH Zurich. The basis was developed during my MSc in computer science thesis at MIT. During my Doctorate in Electrical Engineering I was focused more on measuring human motion for medical (elderly care) application. It was during my Postdoctoral studies that Sensoryx was incorporated, picking up the original idea of my MSc thesis. We had worked on a mobile motion capture device that could capture the movement of a whole body… back in 2008! When VR became more popular, we decided to create a company to implement our ideas through real-life cases. Sensoryx was born from this desire to facilitate interaction in virtual environments, with a dedicated tracking system that is as accurate as in the real world.
Dr. R. A. – Tracking technologies are clearly evolving in the fast-growing XR industry. The advent of more powerful devices and headsets are helping us in this movement. The overall XR infrastructure is now quite impressive, and our own devices can use the resources of headsets (for example) to operate. Especially since the use cases are multiplying, and require more and more from us! For the rest, it is mainly our sensors that are solicited, and there too we have been able to improve them significantly since 2020 for renewed performance.
Dr. R. A. – Sensoryx uses ultrasonic technology to find its way in space, and follows other manufacturers’ devices when it comes to cameras or microphones. These are sources of information above all, and whatever hardware is used allows our software to calculate the environment and the movement precisely. So using Sensoryx doesn’t require any additional hardware to the headsets you are already using, and instead opens up vast fields of exploration to produce your immersive experiences with many possibilities.
Going after science-fiction
Dr. R. A. – When I explained to my relatives my wish to work on haptic devices for VR, they summarised it with examples from movies! At Sensoryx, we focused primarily on creating haptic gloves, with the logic that hands are our primary tools when accessing virtual environments. VR often tries to imitate reality. And so, it is essential to have a fidelity to our real movements! Most of the existing models don’t fully achieve this, especially in terms of feedback. That’s why we worked on these accuracy criteria before announcing our own haptic gloves.
Dr. R. A. – Our work on a haptic glove model was also directed at studying force feedback. Obviously, if you don’t have anything in your hand, you can’t feel or weigh the object. We are getting closer to this sensation (imitating surfaces, feeling shapes…), with the objective of proposing the most realistic situation possible. In 2023, we propose a prototype that brings this frontier between science fiction and reality closer: a highly accurate haptic device that can be integrated into virtual environments for many existing use cases.
Dr. R. A. – The controllers that are currently offered with VR headsets are not obsolete, and I am the first to say that depending on the use, they can still be used. They can simplify your interactions at times, like in video games. Our haptic glove is interesting for applications that really require more finesse and precision – like industries, training, etc. We can train our body’s muscle memory via our body, by repeating gestures. We always need to learn, and this is one of the most important fields of investigation for us. Imagine learning to play music, for example! You can do it with controllers, but it will be less effective than a haptic device.
Understanding the audience for haptics devices
Dr. R. A. – Today, Sensoryx is a technology that measures the physicality of the experience – the environment, the gestures… We use artificial intelligence in part to improve some of the motion recognition, but beyond that it is really the reality of our gestures that is captured. The idea is to be able to avoid large onboarding phases where you have to repeat certain movements several times before using them, but to move towards a simple use of the gloves.
Dr. R. A. – We have to consider audiences with restricted access to our technologies and devices. Last year, for example, we participated in a university research project on Parkinson’s disease to understand where we could help people suffering from this disease. Haptic feedback is very relevant in these cases, and we are happy to support medical research in this direction (like the University of Bern). We put our technology at the service of these causes with great enthusiasm.
Dr. R. A. – In virtual worlds, there are many fascinating use cases. Painting, signing a document, etc. We want to remove a certain “virtuality” from these moments, offering something more concrete. With the development of virtual environments, we have noticed that curiosity can create a form of barrier. We need to make these experiences more instinctive to be able to immerse ourselves more intensely – and forget the technological side. We hope to quickly propose a prototype that can offer a minimum of force feedback, and improve the “natural” aspect of the interactions.
And the future?
Dr. R. A. – At Sensoryx, we are not limiting ourselves to the development of the haptic glove. We will soon propose a pen dedicated to VR uses. Here too, it’s to get away from the classic controllers and improve the precision of our gestures. For graphic designers, creatives or fields such as fashion, this is ultra important. We have all developed around the same technology, to obtain a pencil model with a very classic shape for a surprising result.
Dr. R. A. – The fantasy, always linked to the most popular science fiction, is to go beyond the screens. Everyone is waiting for the moment when we will wear connected glasses as Apple is supposed to do soon, and free ourselves from the physicality of our devices. With the Sensoryx technology, our devices remain low on energy, for the most free use possible, which leaves a lot of possibilities to come…
