Users Lie in VR Usability TestsSeptember 20, 2014
I had a very interesting conversation with Tom Forsyth last night about user testing in VR. Sometimes Tom would test users, and they would become nauseous from playing the game. However, they would insist that they were fine, because they wanted to keep playing, even if they began sweating and you could observe their skin turning pale.
This is a pressing problem for people testing games, who need accurate feedback from test users in order to determine whether their application is making people sick.
However, this is also a problem that websites and phones have when testing. There are a few steps you can take to mitigate this:
Present the software if a third party made it. Pretend that you (the tester) are just trying to get a feel for another person’s games, even if you are actually testing your own game. The user should feel less pressure to “please” in this situation.
Remind the user that they are the expert - the software is under test, not the user, and if it’s bad they should say so.
Administer a questionnaire afterwards - The questionnaire should ask users to rate how they feel on a number of axes - nausea, sweating, headache, etc. Users may be more honest on a questionnaire than they are when speaking to you.
It’s important to remember that you cannot ask the user how sick they are feeling during a user test - it biases the user by asking them to think about how sick they are, and in addition people who are thinking about how sick they are are more likely to become sick.
Don’t trust your own intuitions
I asked Tom whether he trusts his own intuition about what makes him sick, and he violently shook his head “No.” As a developer, you’ve experienced a ton of content in the Rift and become more accustomed both to the platform and to your own game. Furthermore, people vary widely in what makes them sick and how sick they get.
Because users will become more accustomed to your software every time you try it, you’ll have to continually ask new users to try your application. This is going to be a difficult problem on a modest budget. Try finding people in a local electronics store, or in the beginning you’ll probably be able to ask people to try it because they want to get their hands on a headset.