AR is the future of smartphones, starting with Apple’s AR glasses

I’ve been tapping into virtual reality (both figuratively and literally) for quite a while, starting 10 years ago as an early Oculus Rift Development Kit adopter. And over the course of these past 10 years, VR has become so much more accessible and incredible.Last year Oculus, now owned by Facebook, released the Oculus Quest 2, which is a VR headset running Android and powered by a Qualcomm Snapdragon processor. It’s a standalone VR headset powered by smartphone technology!

No need to connect the Oculus Quest 2 to a PC for it to work, as it has Android apps and games made specially for it. It even has its own VR web browser. Plus, until recently you could easily sideload normal, non-optimized Android apps on it, and there was a good chance those would run fine in VR. With some effort, you could get a decent VR Android experience, but more importantly, with some tweaking – an augmented reality Android experience.

So how is all of this related to Apple’s AR glasses? And how are those “the future”? Well, I believe AR (augmented reality) in general is the future of mobile technology, because I’ve been experiencing it for a while, and it’s got me excited. And Apple is shaping up to be the first company to introduce AR to the mainstream as a whole, finished product.

And as a trend-setter who has launched new types of devices into the mainstream in the past, such as the iPod, iPhone, iPad, Apple Watch and AirPods, when Apple launches its AR glasses, there’s a good chance those will have a strong impact on the consumer tech industry.

Hundreds of millions of people will suddenly become aware of AR and start using it in their daily lives. Other smartphone makers will follow the new trend and adopt consumer AR in the form of wearable glasses shortly after.

But back to the now. Even though we’re in the early days of accessible AR, let me share with you how exciting it is even today, and what a huge potential it has to change our entire way of being, just like the internet and smartphones did. And relatively soon.

Consumer-grade AR isn’t perfect today, but perfectly promising

As mentioned earlier, the Oculus Quest 2 is a gaming VR headset powered by Android, a fairly affordable one at that. It was technically designed only with VR gaming in mind, and in VR, you don’t get to see your actual surroundings. Your entire environment is replaced with a digital one.But, a while back the Quest 2 received a beta feature called “Passthrough”, which has just recently gone out of beta and is now easily accessible to all users. The feature in question utilizes two of the headset’s four cameras to show you your surroundings in 3D, almost as if you’re seeing it through your own eyes.

It’s not perfect – it’s grainy, the background is monochrome, and sometimes a bit distorted. But it’s fine. The Oculus Quest 2’s cameras weren’t meant to show you your environment in the first place, their sole purpose is to track your movement and controllers. But team Oculus was kind enough to let us use them for AR anyway, and it’s a decent experience. It gave me an opportunity to use Android in AR, and a good idea of what the future of smartphone technology might be like.

Currently, the Android AR experience you can squeeze out of the Oculus Quest 2 allows you to see your real environment with up to three floating windows in it. For example, you could have two browser windows, one playing a YouTube video, another on Wikipedia, and a Files app window where you can browse through your local photos, videos and music files.

It may be a rough Android AR multitasking experience, but it’s also promising and exciting. See the photo below to get an idea of what it looks like. Note that the background in the image is simulated, because the Oculus Quest 2 doesn’t allow taking screenshots of what the cameras see.

In any case, if Android AR in this early, buggy state is exciting, you can imagine what a polished Apple AR experience might be like.

And note that with Apple’s upcoming AR glasses you won’t need to worry about any cameras or their quality, as the digital content will be projected over transparent glass lenses right in front of your eyes. Or perhaps Apple’s AR glasses will project right onto your retinas. You won’t need to “second-hand see” your environment through any cameras.

You’ll be seeing the world through your eyes as you do with normal eyeglasses. And Apple’s AR glasses will simply project digital content over what you’re seeing, augmenting your reality. And thus the term.

What will the Apple AR glasses look like? Will they need an iPhone to work?

In the long run, Apple’s goal is presumably to create a device shaped like regular eyeglasses, that can hold all of its necessary components in its frames. Thus, it will be standalone and could potentially replace the iPhone someday.

But currently, this would be difficult to achieve from an engineering standpoint, so it’s very likely that Apple’s first AR headset, which has been rumored to be unveiled in 2022, will rely heavily on your iPhone.

Utilizing the full power of the iPhone, Apple’s AR glasses will leave any necessary computing to it, while their focus would be to display the resulting information to you and scan your surroundings, possibly with a LiDAR sensor and a camera.

It’s a genius way for Apple to sell you yet one more device on top of an iPhone. And we all know LiDAR on the iPhone and iPad Pro was never greatly utilized, but that doesn’t mean Apple is done with this technology. Perhaps LiDAR on the iPhone was just a trial run for something bigger.

So what will you be able to do with AR glasses?

Imagine texting your friends without a smartphone – you see their messages appearing at the corner of your eye while you’re having your morning coffee, and you can text them back with just your voice.Imagine watching a podcast while working out, but you’re seeing it floating in front of you and hearing it via speakers embedded in the frames of your glasses.

And imagine visiting a forein country, but the experience is eased on you as local signs or any other text gets translated to English in front of your eyes, as soon as you look.

Or picture using Apple Maps, but instead of having to stare at your physical phone, you get your directions in the form of arrows on the actual road, plus the names of the streets floating in the air.

This isn’t science fiction anymore, your iPhone can already do all of that AR magic to some degree. On iOS 15 you can already point your camera at any text in a different language and the iPhone will offer to translate it for you. You can already open Apple Maps in some areas and get immersive AR directions, scan buildings, translate signs…

Apple’s AR glasses will simply remove the need for you to hold a physical device and stare at it, instead of taking in your surroundings. The iPhone in your pocket will remain there, just quietly processing everything your AR glasses see and hear. Any information you need will be conveniently displayed right in front of your eyes.

AR is going to be a magical experience that will likely significantly change our lives, and it’s not that far from becoming a reality. The first few mainstream headsets may be basic in functionality, but someday the Apple AR glasses (and any potential competitors) could be all you need as you navigate the world and in your daily life, for socializing, entertainment and anything in between. Information from the internet and your phone will be mixed in with the real world, no longer fixed to a device in your pocket.

And because AR has the potential to completely replace our smartphones, tablets and smartwatches, even if that won’t be soon, it’s easy to see it as the future of mobile technology nonetheless.

And why am I personally so certain about that? Because even today’s basic, underdeveloped consumer-grade AR on the Oculus Quest 2 has managed to impress and excite me. It replaced my iPad for most of what I was using it for, including watching Netflix and YouTube while riding my stationary bike, or listening to a podcast while browsing a few web pages. Make that thing lighter and more functional and it will replace my smartphone too, or at least have me pull it out of a pocket way, way less.

I can only imagine the device Tim Cook gets to beta test after all these years of research and development Apple has invested in AR, and I’m looking forward to seeing it. Speaking of Tim Cook…

Apple’s CEO Tim Cook has shared his excitement for AR many times over the years

Apple, the trillion-dollar Cupertino company has been doing research and development on VR and AR for years, based on its many released patents, with new ones popping up to this day. So I’m certain the time we’re going to see results is nearing.

I can almost hear the Apple presentation…

“Everybody uses a smartphone and a smartwatch. And a question has arisen, is there room for a third category of device? Presenting the next logical step in mobile technology – Apple Glass!”

(Feel free to use this quote, Apple, it’s just a tweaked version of Steve Jobs’ 2011 iPad unveiling.)

The first AR device patents from the Cupertino company started showing up in 2017, so we can assume Apple initially began working on AR glasses around then. But Apple’s CEO Tim Cook has shared his excitement for AR even prior to that.

Below we have some Tim Cook quotes regarding AR from 2016 – five whole years ago. And note how even back then his thoughts on VR were very promising.

So there we have it – based on my own experience and Tim Cook’s promising words, I do believe AR is the future. A future where reality is mixed with the digital, and a future that could very well begin with Apple’s AR glasses, relatively soon.