Twenty Milliseconds Logo

This is Twenty Milliseconds, a site documenting what works and what doesn't in virtual reality design.

Look-to-Select Interface: Benefits and Drawbacks

Having players select items by looking at them is simple to code and works on all machines. Designers should provide visual feedback to indicate that you're supposed to look at items to select them, and require a button press to select the target.

Many games have players select options by looking at them. The user centers the button in their field of view. In some variants of this UI design, the option will be selected after spending a small amount of time with the the object centered in the field of view. In other variants, the user must press the space bar or an equivalent key on a different input device to select the button.

Gear VR UI The Gear VR Store uses a look-to-select interface.

Advantage

The advantage of look-to-select is that it works with every input device. Moving your head is a core functionality of a head mounted display, so it’s supported with everything. On-screen tools like a mouse might not be visible or available everywhere. But every HMD supports changing the field of view, and every input device has a primary button.

Disadvantages

First, with a look-to-select interface, there are no visual cues that you are supposed to look at items to select them. In some demos, I was left wildly trying different keyboard keys, or clicking/moving my mouse, in the hopes that something on the screen would change.

Second, I would recommend against choosing an option if the user only hovers over it. In this interface paradigm, your head is serving as the mouse. On a desktop, changing application state based on the location of a hovering mouse is generally a no-no. More information can be made visible (sidebars that pull out from the side, or drop-down menus that become visible on hover), but you shouldn’t ever be shown a new page, or have a form submitted, based on mouse hover. Many people place the mouse over content they are reading or examining, and may not intend to submit a form or navigate to a new page.

The equivalent design rule for VR is to only take an action if the user confirms they want to select the given link. Users may focus a button in their field of view to read it, or may be reading text slightly off-center of their field of view, and it would be wrong to assume they meant to select the item they are looking at. Every input device has a primary button (space bar, right click, A button, etc) and you should listen for this button to confirm the user’s intent.

A smaller note is that it’s not wise to force the user to move their head. Head motion induces motion sickness, at least compared to keeping the head still. Too much head motion can cause fatigue in the neck muscles, as well. If a user is just choosing a few options and then doing a stationary-neck activity such as watching a movie, this is probably fine. But forcing the user to move their head for long periods is not wise.

Finally, it’s possible to choose a target that is too small for the user to look at. I am doing research into acceptable “look” target sizes and will report on results when I have them.

Recommendations

You can use a targeting reticle to indicate to the user that their gaze can be used to select items in the world. The reticle also can reduce error rates when selecting content by making it clear whether the user’s head is appropriately hovering over the target.

The disadvantage of using a reticle with look-to-select is that it is permanently fixed in the center of the field of view, and must be projected onto whatever object the user is looking at. This can require lots of work, and if the reticle is not visible when the user isn’t looking at an object in the foreground, you lose the cue that tells users that the direction of their gaze is important.

Another option is to have a “focus” state where the button style changes when the user looks at it. On the Web, buttons generally become darker when you hover over them - imagine the mouse covering a button in shadow. With Oculus, it may be more appropriate to make a button become lighter on hover, as if your eyes are the light source. You could also take advantage of the 3D environment to “pop” the selected menu item towards the user. You should experiment with this to see what looks and feels good.

Another option is to provide redundant options for selecting UI. In Titans of Space, you can look at an option and press enter to toggle it. Or you can press a key on the keyboard to toggle the corresponding menu option. I would probably have chosen a key other than the F[1-12] keys, as these are harder to select than letters when you can’t look at them, but this is a solid idea for making sure everyone can toggle settings as they see fit.