Apple Vision Pro for $999? An engineer built the Vision Pro’s eye + hand-tracking interface for the Meta Quest Pro

If every tech reviewer who got to try on the Vision Pro after Apple’s WWDC event can be considered a reliable source, the Vision Pro is absolutely ‘magical’. Almost everyone who got to try it on (even Disney’s CEO Bob Iger) has the same feeling of being simultaneously sucked in and blown away by how incredibly immersive and intuitive the tech is. The resolution is flawless, the eye-tracking is brilliant, and the overall experience has changed the minds of quite a few skeptics. There’s a downside, however… This magical experience costs a whopping $3500 USD.

For YouTuber ThrillSeeker, this downside seemed a little too rich. Ultimately, the Apple Vision Pro’s unique interface could be boiled down to three distinct features – Passthrough (the ability to see the world through your headset), Eye Tracking, and Hand Tracking… and the $999 Meta Quest Pro had all those three features. “I’ve been in VR for half a decade, and have been making videos about AR and VR for most of that time,” said the YouTuber, “I struggle to believe that Apple has somehow created something so radically superior, so transformative, that it warrants the use of the word Magical.” A lot of the Vision Pro’s magic is the result of its highly intuitive UI, which lets you interact with elements simply by looking at them and pinching your fingers. The Meta Quest Pro is capable of doing all these things too, although nobody at Meta really built them out… so ThrillSeeker decided to give things a go.

Designer: ThrillSeeker

ThrillSeeker started first by shooting a tweet to Meta’s CTO, Andrew Bosworth hoping for some leads and support, but understandably never heard from him (I assume everyone at Meta was just taking a while to recover from the Apple Keynote). Deciding to then take matters into his own hands (and eyes), he then went on to build the eye and hand-tracking system, designing a mock app drawer (the Vision OS home page) to test out his UI.

Hey @boztank , Still love Quest Pro but it would be amazing if used as a testbed to try new types of UI/UX given its hardware.

Is there anything on the way for Qpro owners to test out UX experiments utilizing eye tracking (like the Vpros look and pinch)

We’d all be thankful.

— 𝙏𝙝𝙧𝙞𝙡𝙡 (@Thrilluwu) June 7, 2023

The entire interface was designed and coded within Unity, where ThrillSeeker tapped into the Quest Pro’s eye-tracking abilities and turned them into a controller of sorts. Most VR headsets ship with controllers, and these controllers use invisible lasers to point at objects, which the headset then recognizes as a cursor. ThrillSeeker simply turned the wearer’s eyesight into a laser pointer, allowing app icons to pop forward when you look at them (just like on the Vision Pro). Tapping your fingers would select/grab the icon, allowing you to manipulate it and move it around.

The pop-out 3D app icons

Even though highly preliminary, ThrillSeeker proved one thing – that Apple’s magical UI isn’t entirely inconceivable – it’s just that nobody at Meta (or Sony or HTC) ever thought of it in the first place. His demonstration proves that this eye and hand-controlled interface is absolutely possible with existing tech in a $999 Quest Pro device. ThrillSeeker is planning on making the APK for this demo available in the near future for all Meta Quest Pro users. We’ll add the link here as soon as he does!