One of many realities: augmented reality 101

It seems that everyone has made predictions as we kick off the New Year, so I am going to jump on the bandwagon: In 2013, I predict that we’ll be hearing about augmented reality – a lot.

As I make this post, CES 2013 attendees are trying out the new Vuzix smart glasses M100, a wearable headpiece with floating eyepiece display that has several potential applications, including augmented reality navigation. There is already an online buzz about this Android-powered competitor to Google’s Project Glass.

So augmented reality will continue to be a hot topic this year across multiple industries – And naturally, we’ll also have more to say about our new augmented reality capability in the coming months.

As companies big and small strive to refine the many facets for this new technology, many people may still be asking a fundamental question: What is augmented reality anyway?

In short, augmented reality can be defined as augmenting a live scene with some 2D or 3D computer graphics elements. Augmented reality is not the same thing as virtual reality or even augmented virtuality. And while augmented reality and augmented virtuality fall under the same mixed reality continuum, virtual reality does not.

That’s a lot of realities to deal with in one paragraph. But I think it’s important to show the difference between these terms: the algorithms, workflow, and the applications are quite different between the different “realities” (for example, a “virtual reality” approach for an “augmented reality” application would fail).

So let me back up a bit.

The Mixed Reality Spectrum

Take a look at this diagram of the mixed reality spectrum.

This is a continuum of “how real” things are from left to right – i.e. when you hit the “real environment,” nothing is computer generated, but when you hit the “virtual environment,” everything is computer generated.  You will notice that augmented reality is more closely related to the real environment and augmented virtuality is more closely related to the virtual (computer generated) environment. This means that augmented reality applications will encompass more live elements than augmented virtuality applications.

Check out the example of augmented virtuality below. Note that the scene displayed on the tablet is virtual, and if you look closely you will see that the hands are holding real tiny objects to direct the virtual ball. In other words, the virtual is augmented with reality.

An example of augmented virtuality

With augmented reality, it is the opposite: the reality is augmented with the virtual. So, you could be looking at a real street and getting information about driving directions through graphical overlays.

Both augmented reality and augmented virtuality have applications in real-time and off-line situations. The best example of an off-line augmented reality situation is in film, where live scenes with live actors are augmented with 3D computer graphics characters – in this case, the objective of augmented reality is to have the 3D computer graphics almost indistinguishable from the live scene.  Many real-time augmented reality concepts have emerged in recent years — such as the driving directions example — where the 2D or 3D computer graphics are there to provide context or purpose to the live scene. Our own example of this can be seen in the figure below, where an engine part is computer graphics generated (in yellow) and over-laid on top of the live equipment.

Our capability is an example of real-time augmented reality

What about Virtual Reality?

This driver training system is an example of virtual reality

You may have noticed that virtual reality was not on the mixed reality spectrum. With virtual reality (applications like pilot or driver training) the subject is completely immersed in a world generated by computer graphics, with no live scene elements; therefore it does not fall under the mixed reality spectrum. A characteristic of the simulation/immersive environment is that the virtual object behavior and display performance must be in real time. The subject, like the driver of the virtual truck in this image, interacts with the virtual objects in a life-like fashion, and acts within that environment (unlike in an offline situation, like a movie, where the subject is passive and has no control over the scene elements).

So many realities….

I hope these definitions were useful or interesting. It’s important to recognize that augmented reality falls into a broader family of mixed realities. I am looking forward to exploring the many new advances in this area that will come into play in the coming months.