Conducting User Tests With an Oculus RiftOctober 9, 2014
When conducting a user test, it’s essential to be able to see what the user is seeing. This way you can compare what they are saying to the view that they have, and steer the conversation in a productive way.
If the testing device is a monitor or a phone you can simply look over the user’s shoulder. Not so with the Oculus Rift, or Sony Morpheus, or other VR device, where the screen is strapped to the user’s face.
Fortunately for the Oculus Rift, it has enough PC infrastructure that we can record. I’m going to assume you are running Windows. (Running Windows in a VM on a Mac would result in horrible frame rates and is not recommended). I would recommend Windows 8 for optimal performance. I’m also going to assume you have the Oculus DK2 set up and have been able to run demos with it; if not, this setup document is going to be your best resource.
The basic idea is we are going to run the Oculus as an external monitor, then record the screen using the screen capture tool Open Broadcaster Software (OBS).
The procedure goes something like this.
Change the “Rift Display Mode” to “Extend Desktop to the HMD” via the Oculus Config Utility in the Windows toolbar. There’s currently no way to record the Direct Mode.
Set the Rift display to be the main monitor in the “Screen Resolution” settings page for Windows. This ensures that “DirectToRift” demos open on the correct screen. However this also means any new windows open in the Rift, which is annoying; learn to use Windows+Shift+Left and Windows+Shift+Right to switch windows out of the Rift and into your desktop monitor.
Open the desired demo. On my machine I open the “DirectToRift” executable and it opens in the Rift. I am not sure if this is normal behavior; you may want to use the other executable.
Configure OBS to always run as the administrator. You can do this by right-clicking on the OBS icon, clicking “Properties” in the window that comes up, opening the “Compatibility” window, then checking the box that says “Run this as administrator”.
Configure OBS. I lifted the settings that eVRydayVR uses to record play-throughs. On my not-top-of-the-line graphics card, I needed to turn the FPS down to 15 in OBS to avoid dropped frames in the Oculus. It is unacceptable for the user’s experience to degrade because you’re recording. They’ll report sickness that’s not because of the software, it’s because of the recording.
Add a source in OBS. You want to add a “Game Target”, then in the dropdown menu that appears, select the window that’s running your demo.
Select “Preview Stream” if you just want to watch, or select “Start Recording” if you’d like to save recording for later.
You will probably also want to record what the user is saying and doing. The simplest possible thing is to use the recording function on your smartphone to capture audio. If you’d like to sync with video, you’ll want to place a phone over the user’s shoulder. I was able to find a number of stands on Craigslist for about $20, and you can buy an adapter for your phone (or just use a camera) fairly cheaply.
Be sure to place the stand far enough away that the user won’t hit it as they move around.
If you’d like to participate in a user test, or try out the Oculus Rift, I am conducting tests, and you should sign up!. I’ll also walk you through the process of conducting a test.
Tips for Testing
Many of the Oculus tools that have come out so far are demos, purely to be experienced, with little reliance on any form of input or interaction from the user. In this case it’s tougher to do goal directed tasks like “book a flight to Seattle” or whatever. Instead you can ask people to walk around and observe their interactions, or tell you what they are thinking about.
You may also test games that have more of a UI focus, or are more built up. Elite: Dangerous and Euro Truck Simulator are two good examples of games you may want to test people on.
Keep in mind that users may not want to tell you how sick they are, and users will become accustomed to your game over time, so you will want to bring in new people for tests each time you make a change in your game.
Moving around: How comfortable are people with the motion paradigm? How do 90 degree turns feel? Do users grasp the movement model immediately?
Moving outside the view frustum: How is it handled when users move too far to the left and the right? Are people comfortable with it? If the player has an avatar and they move three feet backwards (behind the head of the avatar), what happens?
Pausing the experience: How can you pause during gameplay?
Exiting the game: How do you exit? Is there progress to be saved?