The future of virtual headsets? Application method of brain-computer interface in AR/VR
A few weeks ago, Apple showed us today's highest-quality mixed reality device — Vision Pro.It's impressive with its finger and eye control interface.
However, a few months before it was announced, I had an even more shocking experience. I only need to clip the clip to my ear, then wear a device with a rubber tip sensor on my head, and wear a mask to be able to see my brain waves through virtual reality, and I can move objects in a virtual environment by only slightly exercising my facial muscles.This is my experience trying out OpenBCI's Galea device。
The future of virtual reality and augmented reality is steadily evolving, but we're not investing enough time and effort.Currently, the field is shifting from using physical controllers to hand and eye tracking technology. However, deeper, neurotechnology-related possibilities are also waiting to be explored.
I don't even know how to describe my experience trying out the Galea headset because it's a platform for exploring future developments.Also, as far as my experience with neurotechnology is concerned, I'm still in the early stages.
OpenBCI is a Brooklyn-based company focused on developing research tools for non-invasive brain-computer interface technology.They applied their own perception system to a mixed reality headset called Galea and are planning to launch it later this year。
At OpenBCI's Brooklyn office, I had the chance to try out a prototype version of Galea.I'm very curious about how brain-computer interfaces are used in virtual reality and augmented reality, and I'm also thinking about how the future of interacting with computers may bring about changes.
electroencephalogram (EEG)Used to measure the electrical activity of brain signals.OpenBCI uses a sensor with a rubber tip to connect to the scalp, similar to the NextMind hardware I tried in 2021.These electrodes work well in dry conditions, but need to avoid too much hair to ensure good signal reception.
In theory, you can even do very small movements, more like simple nerve impulses. Meta is also developing an EMG technology bracelet for future headsetsHowever, the bracelet only measures finger and hand movements through the wrist, while OpenBCI's sensors focus on the facial area.
Electrodermal activity (EDA)This is an electrical measurement of skin sweat, which is usually used to detect perspiration. For example, Fitbit integrates its own EDA sensor into the Fitbit Sense smartwatch to measureMeasure the level of stress. OpenBCI's EDA sensor is located on the headphone's forehead.
Photoelectric volume spectrometry (PPG)It's an optical heart rate sensing technology that's similar to what most smartwatches already have. On the ultimate Galea headset, PPGMeasurements are also taken on the forehead.But in my demo, I was wearing ear clips for PPG measurements.
This sensor array combines with the existing virtual reality and augmented reality headset Varjo XR-3 (or the lower-cost Varjo Aero) and connects to a computer to run software and analyze data.
Varjo's high-resolution display and mixed reality capabilities for perspective video provide OpenBCI's sensors with many software possibilities in virtual reality and augmented reality scenarios.However, OpenBCI's sensor array can work independently of virtual reality headsets, and can also be used in connection with other devices.
Apple's Vision Pro may also be an ideal platform for OpenBCI because of its powerful processing power and standalone features.According to Conor Russomanno, CEO and co-founder of OpenBCI,Developing augmented reality and virtual reality platforms similar to Vision Pro or the future is totally doable。He sees Apple's recent developments as an important focus on mixed reality in the computer sector, which coincides with OpenBCI's views on this opportunity.
OpenBCI's sensing array can simultaneously pursue multiple goals. It doesn't just focus on a specific goal; it promotes research through the system's sensors and opens up new possibilities for interaction with computers.
Recently, OpenBCI collaborated with Christian Beyerlein, a hacker with spinal muscular atrophy, who used OpenBCI's sensing array to control a drone through facial muscle pulses.This presentation in the TED talk shows how brain-computer interfaces can open new avenues for assistive functions and control of virtual reality technology。
My demo included an electromyogram (EMG) -based game called “Cat Runner” (Cat Runner).I move the cartoon character from side to side through tiny facial muscle movements, which are recognized by the EMG sensor in the Galea mask.
This game is similar to the neural input bracelet technology Meta has been using, and I saw it at Meta's Redmond Reality Lab headquarters last fall.However,Meta focuses on motion perception on the wrist, while Russomano of OpenBCI thinks there would be a better chance of putting the sensor on the head, as this will not interfere with existing camera-based hand tracking work.
Electromyography (EMG) technology is designed to sense weak electrical signals so weak that there is little muscle movement.However, this relationship between sensors, algorithms, and human input takes some time to fine-tune.OpenBCI's various types of sensors can provide vast amounts of data that can indicate future research directions or new interfaces.
They can also provide feedback on the effects of using virtual reality (VR) and augmented reality (AR) on the brain or attention.There have been previous attempts to study cognitive processes using sensors on VR headsets, including HP Omnicept, which has a heart rate sensor, or headsets that support eye tracking.
Another demonstration based on an electroencephalogram (EEG) sensor was to create a meditative “empathy room” where my different brain wave states were transformed into different colors of the surrounding light.My brain waves seemed to change the colors I saw.So I started experimenting with different colors through specific attention methods.
Does this seem to work?OpenBCI's many sensors are key features, and eventually they can be used to train and improve our ability to control things through our own nerve pulses。
Russomanno believes that Beyerlain uses electromyography to control drones by expanding the system to his own brain and body functions.Shows how neurofeedback will change the way we interact with computers, just like artificial intelligence.” That's not to say artificial intelligence isn't useful; it's just that it's not the only solution, nor the only holy grail that can change the world.”
Russomanno said, “Actually, we also value neural feedback, extremely smart user interfaces, and design, which optimize other directions in the feedback loop.Using technology to make computers better teach humans is another exciting revolution we are about to experience.”
This reminds me of the development trajectory of smartwatch technology. Optical heart rate sensors gradually opened up data streams, and then new health features appeared on the watch.Fitbit has added several new sensors to its Sense watch, which also includes an EDA pressure sensor.
Russomanno is convinced that OpenBCI's Galea and similar efforts will open a new path to the wearable sensors of the future, connecting them to the things we see, hear, and interact with our hands.He thinksThe arrival of better standalone virtual reality (VR) and augmented reality (AR) headsets, including Apple headsets, will open the way to new inputs and peripherals。“We won't really know until these headsets come out and people start building bi-directional apps.”
Russomanno told me via video chat a few months after our demo: “Until these headsets came out, we had no idea people were starting to build bi-directional apps.” Russomanno refers to the upcoming more advanced augmented reality and mixed reality devices. ”The cool thing about AR headsets is that they have any outside world sensors you want to know about the local environment. whereasWhat we do is the inner world. When these two data sets are combined, we still don't know what the possibilities will be.”
Although Russomanno compared neurofeedback to artificial intelligence, I also think the two are interrelated.Artificial intelligence requires data sets to work its magic;The same goes for future perceptual technology systems.With the development of neurotechnology, the possibilities for artificial intelligence to co-evolve with it are also increasing.
OpenBCI's Galea is actually a virtual reality (VR) and augmented reality (AR) headset, but its interface with Varjo hardware is just one part of it. This sensor array can also be used alone. This piqued my interest even more when I thought that future wearables could finally interact with other wearables on our bodies.
In this world,Our everyday interactions are likely to be enhanced with more advanced sensors. Although that future is still far away, the sensors that OpenBCI has assembled in Galea seem to be laying the foundation for this future.
Convincing people of the value of virtual reality, augmented reality, and wearable vision technology remains challenging.But improving the way we eventually interface with spatial computing or the real world may be one of the answers to evolving virtual reality/augmented reality into a more meaningful or even more impactful form.Personal technology is building a closer relationship with our senses and our brain, but as far as I can see, we haven't even really begun to move towards that future.
Original text byScott Steincompose, Chinese content compiled by the MetaverseHub (MetaverseHub) team，If you need to reprint, please contact us.
