Redesigning how users point, aim, and interact in AR space
Hand-based targeting on Spectacles had a fundamental issue. The original cursor rendered at a fixed depth — a set distance in front of the user. When the user's hand ray intersected a UI panel at a different distance, the cursor jumped abruptly to that depth. There was no visual continuity, no spatial logic the user could internalize.
The result was that users couldn't build a mental model for how targeting worked. They couldn't predict where the cursor would be or how it would behave. Every interaction felt slightly uncertain. For a platform where hand tracking is the primary input, that uncertainty was a serious problem.
On top of that, the cursor was always visible — whether you were targeting something interactable or not. A constant presence that added visual noise without adding information.
Original cursor behavior · Schematic recreation · Character rig: Rain by Blender Studio, modified (reposed, animated, lit) · CC BY 4.0
Make targeting feel like a natural extension of intent — not a visible mechanism users have to think about. The system should communicate just enough to guide action, and disappear once the user knows what to do.
The new cursor only appears when the user is near something interactable — fading in smoothly as their hand enters the object's interaction zone. When not targeting anything, it disappears entirely.
This required a foundational change to the Spectacles Interaction Kit: all interactable objects now define an interaction zone that drives cursor visibility. It wasn't just a visual change — it was a systemic one that required every interactive object in the platform to carry this new behavior.
The design principle: the cursor should be informative, not decorative. It should appear when it has something to communicate and disappear when it doesn't.
New cursor — proximity-aware fade-in · Schematic recreation · Character rig: Rain by Blender Studio, modified (reposed, animated, lit) · CC BY 4.0
The second change was adding a visible ray — a line from the user's hand to the point of targeting. This gave users a clear spatial reference for where their hand was pointing.
The ray length is dynamic — it extends to the surface it's hitting and no further. Instead of a floating dot at an arbitrary depth, users now had a line that told them exactly how far away the target was.
The underlying ray casting logic didn't change — we updated the visuals and introduced the interaction zone. The ray itself was immediately intuitive to users from the start, with no confusion about what it was communicating.
Both targeting visuals — cursor and ray — are configurable per object. Developers can choose which visual each interactable uses, and can combine both approaches within the same experience. The design intent was: use the cursor for 2D panels and objects at similar depth; use the ray when objects are scattered across world space at different depth layers. Mixing both within the same view is possible but not recommended — consistency of targeting feedback within a single context keeps the experience legible.
Ray behavior · Schematic recreation · Character rig: Rain by Blender Studio, modified (reposed, animated, lit) · CC BY 4.0
In the original Interaction Kit, when a user pointed at a panel but landed on negative space — text, an image, anything non-interactive — the cursor had nothing to lock onto and would snap through the panel to whatever interactable existed behind it in space. The panel you were looking at essentially had no depth wall.
The fix was an interaction plane — a depth boundary attached to the panel itself. Interactables inside the panel automatically inherit it, keeping the cursor locked at the correct depth even when pointing at empty areas of the UI. The result: targeting feels contained and predictable, no matter where on the panel you're pointing.
I built two functional prototypes in Lens Studio — one with the original cursor behavior, one with the new system — and wrapped both in the same balloon-popping task to create a controlled comparison. Same task, same environment, different targeting systems. The TypeScript implementation was a collaborative effort, since the interaction logic required code beyond my own scripting ability.
The prototypes went through corridor studies with the UX research team. The results were clear: the new design produced significantly more successful hits than the original.
The qualitative feedback told the same story — users felt more confident and in control. The uncertainty was gone. I presented the findings and the prototype comparison directly to Evan Spiegel and received his sign-off to proceed.
The new targeting system shipped as a foundational update to the Spectacles Interaction Kit. Every interactable object on the platform now carries the interaction zone behavior I designed. It's not visible to users as a single feature — it's infrastructure. But it's the reason targeting on Spectacles feels reliable rather than frustrating.