Apple hopes the Vision Pro will fundamentally change how we interact with our devices. Freed from the constraints of a smartphone or tablet screen, Apple hopes we’ll embrace “spatial computing”, as its glitzy promo video shows. Strap the headset on around your forehead and you’ll see… what you saw without the headset, but with a row of app buttons overlaid on top of your field of vision. Gesture and eye tracking identifies where your focus is, allowing you to interact with apps without pressing buttons or a screen.
That could be great for consumers. But it’s a headache for Apple’s ecosystem of app developers. Apple was at pains to explain that existing apps designed for the iPad will work on visionOS, the operating system powering the Vision Pro, without any changes. But those iPad apps will simply be displayed within a metaphorical window, losing much of the functionality provided by a mixed reality headset.
To fully take advantage of the technology, these apps will need tweaking to unlock some of the opportunities available when taking them off a screen and into the real world, as you’d get in fully native, three-dimensional, augmented reality apps.
The announcement was a momentous one for René Schulte, head of 3D and quantum communities of practices at Italian company Reply, which designs 3D environments as part of its business. But he’s worried that much of what was shown in the demo videos presented by Apple were limited uses of the opportunities given by mixed reality.
“What I didn’t like was the focus on 2D content,” he says. Schulte has been working with Microsoft’s mixed reality glasses, the HoloLens, since 2015, and the Oculus Rift. He sees chances to overhaul the user experience for the Vision Pro that were missed.
In part that’s down to the challenges involved in redesigning apps for an entirely new interface. Schulte’s employers, Reply, published a white paper on how to transition apps from two dimensions into three last year. In it, they admit the change in mentality is not easy.
“Designers need to learn new methods and skills, and also get used to new tools,” says Schultz. “Designing for 3D is not simply mirroring 2D concepts into three-dimensional space.” Yet that’s what he saw in—for instance—the presentation of Adobe Lightroom and Microsoft Office being presented as 2D apps within a 3D space.
Denys Zhadanov is a board member and ex-vice president of Readdle, a Ukrainian app development company that produces a suite of popular productivity apps across iOS. He’s enthused by the promise of the Vision Pro, but also recognizes it’ll require retooling Readdke’s apps.