This fits well with a larger project that Facebook is working on, called the Metaverse. Metaverse is more than VR and AR technologies. Simply put, this is a new version of the internet or some other version of augmented reality, in which the real and virtual worlds intertwine, creating a whole new space of three-dimensional objects in interaction. .

Facebook Reality Labs is working on technology from the egocentric perspective with scientists from more than a dozen scientific institutions and presented the first result of this collaboration a few days ago. This is the Ego4D package (the 4D expression says this is an additional fourth dimension), which consists of two components. The first is an open dataset that records nearly 2,880 hours of video of what a first person perspective is.

Also check:

The second element of the package is a set of references designed for neural networks, intended by its creators to teach artificial intelligence how to perceive and interpret video from the point of view of the first person. Details of this innovative project were presented in a paper titled “Ego4D: Around the World in 2.8K Hours of Egocentric Video” by Facebook scientist Kristen Grauman.

The aforementioned dataset is to help scientists design neural networks, which, due to their enormous capabilities, can successfully support artificial intelligence in the process of building first-person perspective, the same. way than other datasets (such as ImageNet), which also support artificial data. intelligence, but only in the process of building an ordinary perspective.

Facebook explains that the main purpose of egocentric perception is to try to solve problems that neural networks currently cannot handle. They occur when the point of view of an image changes in a video. Most of the image recognition systems available today that deal with these situations have a high failure rate. Facebook’s new initiative and packages like Ego4D are poised to address this.