When you make a computer game, you try your best to make it easy to get started with: making menus clear, giving hints when needed, making the controls intuitive, starting out at a difficulty level that's easy but not boring. But because you tend to play the game in a particular way, and play it a lot, there end up being a lot of gaps where things are needlessly hard or confusing.
This is where hallway testing comes in: grab a bunch of random people from the hallway - or from the cafeteria, in my case - put them down in front of your software, and observe as they try to use it. And very soon, you'll find that things that were totally obvious to you are not obvious to anyone else in the universe - or at least confusing to a big proportion of people.
Two examples:
Right at the start of the first level, once you have figured out how to move your creature, a hint appears next to the target reticle, telling you to click to shoot. What it doesn't tell you is that you can use the mouse to aim. As a result, I had several people trying to play the game with the angle of their shots fixed to something awkward, and getting absolutely nowhere. That you can use your mouse to aim seems obvious to me, and maybe to you, but it's clearly not obvious to everyone. Nor is it a mistake that only non-gamers would make: two students who tried it were frequent gamers and didn't realize.
The conclusion from this one: The hint should say use the mouse to aim as well as click to shoot.
Another small thing occurred with one tester, who, noticing that he had two weapons available, tried to click on the other weapon to activate it. This did absolutely nothing, because it had simply not occurred to me to put in that option: you can select weapons with the number keys, and cycle through them with Q and E, but you can't use your mouse. As with the one above, this is a five-minute fix, and it will save at least a few people some confusion and frustration.
After four hours and six people, I have ended up with a lot of notes. Thankfully, there's no huge problems: everyone got into the game, rapidly became better at it, and enjoyed themselves. What remains is a long list of little fixes to make the game just a bit more pleasant.
Hover-Elephant is good friends with your mother.
Nice article! I should try this.
Interesting method, never thought of this. Did you communicate with the players, or were you strictly observing? I would be tempted to talk to them, but this would mean that I might influence them (even with some questions), which defeats the purpose of the hallway testing to begin with.
I don't think it would defeat the purpose, you would still learn things. But I see what you are saying. I think you'd need a balance. Clearly you shouldn't help them too much, but if they got stuck you would help them and at the same time learn what they didn't understand.
I tried my best to not talk to them at least at the start, so I could observe their reactions to the game and see if they got stuck anywhere. If they did get stuck somewhere, I would help them, of course. After a few minutes, we tended to naturally get to chatting about the game, though.