Michael Abrash, Chief Researcher at Facebook Reality Labs, spoke about the development of the "superpowered perception" technology that will be implemented in the company's future smart headset. He made a special emphasis on the function of improving hearing using augmented reality methods. In a way, this is an analogue of a hologram, an information image on the user's glasses, but in the form of an audio signal.
The main advantage of the Facebook headset is the ability to intelligently analyze data from external microphones, cameras and a wide range of other sensors. For example, the system can recognize the manner of communication by the appearance of the interlocutor and his facial expressions, learn from LiveMaps that something is happening in a noisy restaurant, take into account the time of day, human traffic and even the weather in order to optimally filter the sound. This makes it possible to carry on a conversation in any conditions without raising your voice or straining your hearing, but there is enormous potential behind such a simple option.
If you combine a headset with AI, you can use audio analysis to instantly translate foreign speech. Or in order to isolate the voice of a crying child who is lost in the noise of the crowd. You can configure the system to search for specific words and phrases to quickly find a specific person in a large conference. In a word, "super hearing" opens up a lot of possibilities for users of the headset.