Why the Apple AR headset’s secret weapon might be a smarter Siri

Apple will probably unveil its first mixed-reality headset later this year. The device will support both virtual (VR) and augmented reality (AR) experiences. The AR/VR headset should pave the way to AR-only glasses that are as elegant as regular glasses in the future, the kind of device that might replace the iPhone.

But the road to that ideal sort of AR experience is long, and there’s no estimate for the glasses’ arrival. For now, Apple is focusing on the AR/VR headset and creating experiences for it. And a report claims that Apple wants to let users create their own apps using voice instructions to Siri to code them. A smarter Siri might turn out to be one of the most exciting Apple innovations in years.

Apple has been working on the new software development tool since 2021, according to The Information (via MacRumors).

The initiative should make it even easier for regular consumers who do not know how to code to create apps for the AR/VR headset. That’s what the report calls the content, applications that would be available via the App Store for others to download:

With the software tools, Apple hopes that even people who don’t know computer code could tell the headset, via the Siri voice assistant, to build an AR app that could then be made available via Apple’s App Store for others to download. The tool, for example, could allow users to build an app with virtual animals moving around a room and over or around real-life objects without the need to design the animal from scratch, program its animations, and calculate its movement in a 3D space with obstacles.

But from the sounds of it, Apple is rather looking to let users create readily deployable AR and VR content via Siri. Apple is reportedly using technology from a startup called Fabric Software it bought in 2017.

The report said Apple’s tool would behave like Minecraft and Roblox, games that let anyone create 3D tools and worlds. The added functionality of Siri voice commands should simplify developing apps/content for the AR/VR headset.

There’s one caveat here. As exciting as the Siri-based coding software might sound, The Information’s sources last saw the tool in 2021. Apple could always change its mind. But creating any apps or content with the use of voice is the kind of breakthrough that would impress.

We’re on the verge of incorporating smart chatbots into tech experiences. Also, the importance of voice will only grow as smart computers of any size get better at understanding speech. The AR/VR headset would also benefit from a strong Siri assistant.

The revamped Siri could do more than answer questions and execute commands flawlessly. If Apple wants users with no knowledge of programming to create exciting content, then it first has to upgrade Siri to accommodate such a way to create apps.

Such a tool would also come in handy to experienced coders, as deploying code via Siri could help them speed up app development. This already sounds like the stuff of sci-fi movies. But the emergence of ChatGPT proves the future where smarter AI is ready to assist with all sorts of tasks is almost here. Apple must surely develop similar tools internally, including this rumored coding tool for Siri.

Finally, letting ordinary buyers create AR and VR experiences that can go viral once they become available in the App Store would fix one of Apple’s biggest problems with the mixed reality headset. Apple still needs to explain why we need to purchase AR/VR and AR devices as soon as possible. Access to lots of content is one way to make these gadgets must-have devices.

There’s no telling when Apple will unveil the AR/VR headset. Rumors say the wearable will be available in stores in the second half of the year. The first-gen gadget might cost around $3,000 and should be available in limited supply. But once that next-gen Siri arrives on a future version of the headset, it’ll also populate lots of Apple devices, including iPhone, iPad, and Mac. And Siri should have other sophisticated powers for these devices. Maybe even the ability to interact with traditional code to improve app development.