Apple’s ARKit: Cheat sheet

ARKit was released with iOS 11 at the Apple’s Worldwide Developer Conference in 2017. Augmented reality (AR) is at the forefront of specialized technology being developed by Apple and other high-tech companies, and this new kit helps developers bring AR to iOS devices. The development kit is integrated with the latest hardware to allow for a level of AR integration never seen before on iOS devices.

This article is an introduction to ARKit. We’ll update this resource periodically when there’s new information about the AR development framework. It is also available as a download, Cheat sheet: Apple’s ARKit (free PDF).

SEE: Augmented reality: Quick glossary (TechRepublic Premium)

Executive summary

• What is ARKit? iOS includes a framework in the iOS Software Development Kit (SDK) called ARKit. This framework allows developers to create augmented reality apps and take advantage of the hardware and software capabilities of iOS devices.
• Why does ARKit matter? Like virtual reality, augmented reality is a hot topic in tech. AR allows app users to place virtual objects in a real world, and then interact with them. AR apps will be especially popular for the gaming, shopping, and industrial sectors.
• Who does ARKit affect? iOS 11 was the first version of iOS that included ARKit, and it has been included in new iOS versions since, allowing developers to create AR experiences in their apps. This also affects users, because they can download and use ARKit apps on their iOS devices.
• When is ARKit available? ARKit has been available to developers as part of Xcode 9 since its beta release at WWDC ’17 in June 2017. ARKit apps has been available to consumers on the App Store with the release of iOS 11 on September 19, 2017.

SEE: All of TechRepublic’s cheat sheets and smart person’s guides

What is ARKit?

ARKit is an augmented reality framework included in Xcode that is compatible with iPhones and iPads. ARKit lets developers place digital objects in the real world by blending the camera on the screen with virtual objects, allowing users to interact with these objects in a real space.

It does this by using the camera on iOS devices to create a map of the area, detecting things like tabletops, floors, and the location of the device in the physical space using CoreMotion data. The user doesn’t have to do any calibration–that’s a breakthrough in this space.

ARKit can run on most modern iOS 11 iPhones and iPads, and can utilize Metal‘s features like SceneKit integration and integrate with third-party tools such as Unity and Unreal Engine.

ARKit can run on most modern iPhones and iPads, and can utilize Metal‘s features like SceneKit integration and integrate with third-party tools such as Unity and Unreal Engine.

For end users, ARKit will enable a new level of real-world integration never before seen on iOS devices. A new generation of apps will be able to be built that can do things like show furniture inside of your house while shopping in an app; show your dream car parked in your garage to see if it will fit; and many other inventive uses of this technology that developers haven’t discovered yet.

Apple created a Swift Playgrounds challenge that uses ARKit. The Swift Playgrounds app can be downloaded for iPad from the App Store. Developers and end users can see how these AR apps are created and used.

Additional resources

Why does ARKit matter?

AR has been a hotbed technology the past several years, and ARKit affirms this technology is here to stay and will become an ever-growing field for apps in the future. The use of AR in spaces other than gaming will be a game changer.

ARKit apps are currently in their infancy, but there is real potential in the apps that can be created with it. Apple has provided an amazing framework that is leaps and bounds better than what was previously available to create AR apps. Developers no longer have to develop a speciality, but instead can use this framework to create AR features in their apps without any prior experience working with AR.

SEE: Executive’s guide to the business value of VR and AR (free ebook) (TechRepublic)

With the iPhone X, Apple has created a true AR device with a better camera that will provide both facial tracking and better positioning and topology detection.

In the future, we could see apps that are used by industrial workers, medical workers, scientists, and other technical professionals to overlay critical information over a real-world scene.

Additional resources

Which devices and apps support ARKit?

ARKit-enabled devices are any iPhone or iPad running iOS 11 or later that have the Apple A9, A10, A11, or later processor; these devices are required to run the advanced Metal graphics. At launch, most of the apps that support ARKit are in the games category; as more developers work with AR, we will see an increase in the number of useful apps.

Some of the popular apps that are available at launch with ARKit support include RoomScan Pro, Human Anatomy Atlas 2018, and Night Sky 5, as well as the following.

Games

Tools and other apps

• Overstock.com (Free): Place life-size 3D furniture models in a room to see how they will look; you can even walk around it to get a sense of scale.
• Edmunds (Free): Place a life-size 3D model of a car that’s in their library in your driveway or garage to see if the car will fit.
• MeasureKit (Free): This is an AR utility that lets you measure a room or other object in real life using a virtual tape measure.

Additional resources

ARKit 2 improvements

ARKit 2 brought a new vivid augmented reality experience to apps that allowed them to interact with the real world in new ways. With ARKit 2, multiplayer AR games were possible, as well as tracking 2D images, and detecting known 3D objects like sculptures, toys, and furniture.

SEE: Virtual and augmented reality policy (TechRepublic Premium)

ARKit 3 improvements

ARKit 3 adds features that take augmented reality even further. You can show AR content in front or behind people using People Occlusion, which can track up to three faces. You can also now support collaborative sessions, enabling completely new AR shared gaming experiences.

Motion capture can be used to understand body position and movement, tracking joints and bones, allowing for new AR experiences involving people instead of just objects.

Other major improvements:

• Detection capabilities of up to 100 images at a time
• Estimation of physical size of an image
• 3D-object detection is more robust and can better recognize objects in complex environments
• Machine learning is used to detect planes in the environment faster, leading to faster AR environment setup
• Simultaneous front and back camera use means that you can interact with AR content in new ways with facial recognition
• Multiple face tracking on iPhone X, iPhone XS, iPhone XS Max, iPhone XR, and iPad Pro can now track up to three faces at once

ARKit 4 improvements

ARKit 4 was announced at WWDC ’20 and released alongside Xcode 12 and macOS 11 and iOS 14 later in 2020. With the introduction of LiDR on iPhones and iPads in 2020, Apple decided to add functionality to take advantage of these sensors into ARKit 4. Depth information from the LiDR sensor was added into this version of ARKit to improve tracking and object detection. 

In addition, ARKit 4 also added Location Anchors, which is a new feature that uses Apple Maps data to place augmented reality experiences on specific geographical coordinates. 

Reality Composer and RealityKit

These new tools available in ARKit can aid in the creation of AR scenes with little to no traditional AR design experience.

For more information, read about Reality Composer and Reality Kit on the Apple developer website.

Additional resources

When is ARKit available?

ARKit was available beginning in June 2017 to developers in the iOS 11 beta and the Xcode 9 beta. The final version of 1.0 of ARKit was released alongside iOS 11 in September 2017. Developers began making ARKit apps available on the App Store for iOS beginning September 19, 2017. ARKit is bundled with the latest iOS SDKs inside of the latest versions of Xcode.

Version changes

• June 2017: Apple released the beta of ARKit alongside Xcode 9 and iOS 11 betas.
• September 2017: Apple announced iPhone X with TrueDepth camera, which enables ARKit to detect position, topology, and the user’s facial expressions.
• September 2017: Apple released iOS 11 to the public and Xcode 9 to developers with the newly included Face Tracking features in ARKit.
• November 2017: Apple will release the iPhone X to the public, and developers will be able to take advantage of the facial tracking features in the TrueDepth camera system.
• June 2018: iOS 12 beta is released to developers at WWDC, bringing a beta version of ARKit 2.
• September 2018: Apple releases iOS 12 to the public, bringing ARKit 2 to iPhones and iPads, allowing developers to release ARKit 2 apps publicly.
• June 2019: iOS 13 beta is released to developers at WDWC, bringing a beta version of ARKit 3.

• September 2019: iOS 13 is released to the public, brining ARKit 3 to the public through new third-party apps and experiences.

• June 2020: iOS 14 beta is released to developers at WWDC, brining a beta version of ARKit 4, with LiDR improvements for the recently released 2020 iPad Pro with LiDR sensor.

• September 2020: iOS 14 is released to the public, bringing with it apps able to support ARKit 4 to the App Store. 

Additional resources

Apple Weekly Newsletter

Whether you want iPhone and Mac tips or the latest enterprise-specific Apple news, we’ve got you covered.
Delivered Tuesdays

Sign up today