The word is out! Facebook is investing heavily in augmented reality, and for a good reason. The augmented reality market was sitting at a market value of $14.7 billion in 2020. It is expected to surpass the $88.4 billion mark by the end of 2026 [1]. 

At present, the popular social media giant is building its own augmented reality (AR) glasses in collaboration with Ray-Ban. At present, these gadgets can only record and share imagery. However, Facebook may have different ideas up its sleeve. We are well aware that Facebook is more than capable of dishing out new innovations. But what is the company’s stance on AR devices at the moment?

Let’s find out!

SEE ALSO: The Present and Future of Artificial Intelligence (AI) in Agriculture

Facebook’s AI team undertakes a new research project

Hands and futuristic smartphones in double color exposure effect in double color exposure effect

Facebook’s AI arm has kickstarted a new research project that gives a sneak peek into the company’s plans. Prima-face, the company is taking a deep plunge into the AI world by developing an AI system that sees, hears, and remembers everything people do to help them perform daily tasks in a streamlined manner. 

The company’s research team has provided a string of skills that it wants to incorporate into these AI systems including “episodic memory”. Basically, it wants to create an AI system that can answer questions such as “Where did I leave my wallet?”. Besides, it also wants to add “audio-visual diarization” – the ability to remember what was said by whom. 

At this stage, AI does not provide reliability to deliver these tasks with high precision. Currently, Facebook has said that this is a research project rather than a commercial one. However, that does not change the fact that it views such functionalities as the future of AR computing. 

Kristen Grauman, Facebook’s AI research scientist said that the company has augmented reality on its horizon and has a few ideas around the technology. He also added that this research project will help them explore the different opportunities that lie ahead in the augmented reality ecosystem. 

However, such endeavors come with a bunch of privacy implications. However, a host of privacy experts have expressed their concerns pertaining to Facebook’s augmented reality glasses. They are of the opinion that enables companies such as Facebook to covertly record data of the public via wearers of such devices. We also predict that such concerns will only increase in the future if the newer versions of the hardware will transform wearers into walking surveillance machines. 

Facebook has named its research project Ego4D. It is made of two major components – an array of challenges that the company deems fit for AI to solve and an open dataset of egocentric video. 

The current dataset is probably the largest ever created. Facebook has joined hands with 13 universities from around the world to gather information. At the moment, around 3,205 hours of footage has been recorded by 855 participants who live in 9 different nations. Although Facebook has been in the limelight, the universities were largely responsible for gathering data. 

A few paid participants were part of this project who wore GoPro cameras and AR glasses to record footage of unscripted activity. The content of the video ranges from socializing with friends to playing with pets to construction activities. 

Each piece of content was de-identified by universities. This included blurring the faces of onlookers and eliminating any piece of personally identifiable information. 

According to Grauman, the dataset is the first of its kind in terms of diversity and scale. A project that comes close has around 100 hours of first-person footage that was shot only in kitchens. 

The second part of the Ego4D project is a gamut of tasks that Facebook wants researchers around the globe to address using AI solutions trained on its native dataset. 

Some of these tasks include:

  • Forecasting – Predictions about the next course of action an individual is likely to take. 
  • Social interaction – Help people interact with each other seamlessly in a noisy environment. 
  • Episodic memory – Remembering what happened in the past. For example, “Where did I leave my wallet?”
  • Hand and object manipulation
  • Audio-visual diarization

Final words

Business hand robot handshake, artificial intelligence digital transformation

As mentioned earlier, tackling these problems is a hard task even for the current AI systems. However, as technology continues to mature, these challenges are likely to be addressed with ease in the future. As of now, creating datasets and benchmarks are proven techniques that are likely to accelerate development in the AI space. 

The recent AI boom is also attributed to the development of one dataset that is linked with the annual competition known as ImageNet. The ImageNet dataset contains images of a wide range of objects which researchers use to train the AI solutions for seamless identification of objects. 

Besides, as deep learning and machine learning continues to grow in stature, their adoption is only expected to move in the upward direction. Time will tell whether this huge influx of technology will become a matter of concern or make our lives better. 

But tell us what you think about Facebook’s plans to develop AI systems that see, hear, and remember everything you do. 

For more latest news and updates, keep reading iTMunch

SEE ALSO: 6 important questions to ask your AI security solutions vendor

Feature Image Courtesy:

Image 1 Courtesy:

Image 2 Courtesy:


[1] (2021) “The augmented reality market was valued at USD 14.7 billion in 2020 and is projected to reach USD 88.4 billion by 2026; it is expected to grow at a CAGR of 31.5% from 2021 to 2026” Yahoo Life [online] Available from:,The%20augmented%20reality%20market%20was%20valued%20at%20USD%2014.7%20billion,31.5%25%20from%202021%20to%202026 [accessed December 2021]