Apple in less than two months is planning to enter a new product category, debuting its first mixed reality headset. Rumors suggest that the upcoming headset will support both AR and VR technology, and that it will have a number of features that will outshine competing products.
With the iPhone, iPad, and Apple Watch, Apple's hardware and software led it to dominate those categories within a few short years after entering a new market, and it's possible the same thing will happen with augmented and virtual reality. We've rounded up 10 features rumored for the AR/VR headset that will set it apart from competitors.
4K Micro-OLED Displays
Apple plans to use two high-resolution 4K micro-OLED displays from Sony that are said to have up to 3,000 pixels per inch. Comparatively, Meta's new top of the line Quest Pro has LCD displays, so Apple is going to be offering much more advanced display technology.
Micro-OLED displays are built directly onto chip wafers rather than a glass substrate, allowing for a thinner, smaller, and lighter display that's also more power efficient compared to LCDs and other alternatives.
Apple's design will block out peripheral light, and display quality will be adjusted for peripheral vision to cut down on the processing power necessary to run the device. Apple will be able to reduce graphical fidelity at the periphery of the headset through the eye tracking functionality being implemented.
Integrated Cameras
Apple is outfitting its AR/VR headset with more than a dozen cameras, which will capture motion to translate real world movement to virtual movement. It is said to have two downward-facing cameras to capture leg movement specifically, which will be a unique feature that will allow for more accurate motion tracking.
The cameras will be able to map the environment, detecting surfaces, edges, and dimensions in rooms with accuracy, as well as people and other objects. The cameras may also be able to do things like enhance small type, and they'll be able to track body movements.
Iris Scanning
For privacy and security, the AR/VR headset is expected to integrate an iris scanner that can evaluate the pattern of the user's eye, allowing for an iris scan to be used for payment authentication and as a password replacement.
Iris scanning on the AR/VR headset will be akin to Face ID and Touch ID on the iPhone, iPad, and Mac. It could allow two people to use the same headset, and it is a feature that is not available on competing headsets like Meta's new Quest Pro.
Facial Expression Tracking
The cameras in the AR/VR headset will be able to interpret facial expressions, translating them to virtual avatars. So if you smile or scowl in real life, your virtual avatar will make the same expression in various apps, similar to how the TrueDepth camera system works with Memoji and Animoji on the iPhone and iPad.
Control Methods
3D sensing modules will detect hand gestures for control purposes, and there will be skin detection. The headset will support voice control and the AR/VR headset will support Siri like other Apple devices. Apple has tested a thimble-like device worn on the finger, but it is not yet clear what kind of input methods we'll get with the new device.
Air Typing
For text input, the AR/VR headset will support "in-air typing" functionality, with the headset recognizing the fingers as they move using the built-in cameras.
Bloomberg's Mark Gurman says air typing is "finicky," but Apple will offer the feature when the headset launches.
Thin and Light Design
Apple is aiming for comfort, and the AR/VR headset is rumored to be made from mesh fabric and aluminum, making it lighter and thinner than other mixed reality headsets that are available on the market. Apple reportedly wants the weight to be around 200 grams, which would be significantly lighter than the 722 gram Quest Pro from Meta.
In March 2021, Apple analyst Ming-Chi Kuo said current prototypes were around 200 to 300 grams, but it is not clear if Apple was able to stick to that weight later in the development process.
External Battery Pack
Most of the AR/VR headsets on the market have an integrated battery, but Apple is planning to have the headset connect to a separate, external battery that is worn at the waist. The external battery will power the headset for approximately two hours, and it will be able to be swapped out for continuous usage while a second battery charges.
App Experiences
The headset is going to run a new operating system called xrOS, aka "Reality OS." Apple is designing unique apps made specifically for a virtual reality experience. Apple is said to be planning for a VR FaceTime-like experience with Animoji, where you might see a 3D Animoji or Memoji character version of a person instead of the person itself. The aforementioned facial expression detection would allow the headset to read facial expressions and features, matching that in real time for a lifelike chatting experience.
Apple is working with media partners for content that can be watched in VR, and it will integrate with Apple TV+. Users will be able to watch movies and TV shows in a virtual reality environments, with the display over a background like a desert or a mountain.
Sports content will be a focus, with Apple providing immersive viewing experiences for MLB and MLS content. Apple is working with third-party developers on gaming experiences, and there are 3D versions of standard iPhone apps like Safari, Calendar, Contacts, Home, Files, Messages, Notes, Photos, Music, Reminders, and more in development.
There will be a Fitness+ app that will allow users to work out while watching Fitness+ instructors in 3D, and Apple is creating a meditation app that will guide users through meditative experiences. A Camera app will be able to snap photos, a Books app will allow for reading in virtual reality, and a version of Freeform will be available for working on collaborative projects with others.
The headset will also be able to run thousands of existing apps that are designed for the iPad, with the apps showing up in a 2D format on the headset.
Apple Silicon Chip
Rumors suggest that Apple will use two Mac-level M2 processors for the AR/VR headset, which will give it more built-in compute power than competing products. Apple will use a high-end main processor and a lower-end processor that will manage the various sensors in the device.
With two Apple silicon chips inside, the headset will not need to rely on a connection to an iPhone or a Mac for power, and it will be able to function on its own.
Read More
For more on everything that we've heard about Apple's work on the AR/VR headset, we have a dedicated roundup that aggregates all of the rumors.
