Tips for Playtesting news - Echo Lake

In this post I give an insight into the playtesting process used for Observatorium. Hopefully these steps will be useful to you in your own projects.

What is the point of playtesting?

The point of playtesting is to highlight (1) design flaws and (2) major issues in your game. The process of discovering and fixing these bugs should strengthen your product.

When should I start playtesting?

As early as possible! A good rule of thumb is to start testing when there's something worth playing. If all you have implemented is a button that causes the player's avatar to jump you're probably not there yet; if you've built an endlessly scrolling level that throws obstacles at your player and have a basic scoring system in place you may be ready to test. Aim for 5-10 minutes of play.

Why so early?

Your average game development cycle requires input from the following disciplines.

• (1) Design
• (2) Code
• (3) Art
• (4) Audio

Ultimately, the design of the game will be your most scrutinised aspect when it comes to players/reviewers and it knocks onto code, art and audio decisions. Early playtesting can save you time and money and de-risk the later stages of your development cycle. Code is time consuming so you'll want to reduce coder workload where possible. Art and audio are low risk and can change right up until release. Design however needs to be locked down as early as possible.

Suggested Phases for Playtesting

This is a suggested approach to playtesting: the idea being to start small and branch out with each new stage - adding more people to your test pool and improving the quality of feedback you receive.

(1) Test Yourself

This may seem counter-productive but before getting anyone else on board you should test it yourself.

It's hard to analyse your game impartially being so close to development but some things you can look out for at this stage are:

• Aim - does the demo have a goal? Does it provide enough of a challenge or is there an adequate progression in place? If not, subsequent tests may not be very useful to you right now.
• Showstoppers - anything that seems like a major bug will hamper subsequent playtests and shift the focus from playability to stability. Prioritise glaring bugs that are easily reproduced and fix them. Infrequent critical bugs or any minor issues can wait
• Tutorials - ask yourself if you have trained the player adequately for the challenges they face. If not - add more details: you're already thinking about design improvements for first time players!

You don't want to spend too long on this phase as you should be testing your game daily as you add new features. This stage is about making sure you're ready for the next step...

(2) Test with Friends

Next, ask close friends to play your game and watch them as they do it. Chances are you will have missed alot of major issues in phase (1) and will need to go back. The point however is to get people outside the development team to play your game and get a useful list of feedback to help you focus.

From here on you'll want to start taking notes and refraining from interacting where possible. The approach I use on Observatorium is:

• Observation - let your friend play the game and observe them silently
• Vocalisation - ask your friend to vocalize their thoughts as they play as much or as little as they like. This gives an insight into their thought process and how they are interpreting the game. Believe it or not what you see them doing and what they tell you they're trying to do may be completely different (!)
• Notes - make a note of any playability issues they encounter: this will depend on your game design but some examples of things you'll want to look out for are (1) clarity - do they understand the concept, menus and gameplay? (2) flow - are they experiencing the game as you intended? (3) fun - are they enjoying the experience as you intended?

Make a note of bugs but flag them as such - again, you can deal with these later. Do not start the playtest by explaining your game to them: you want them to come at this cold to gauge the usefulness of your prompts.

If your friend gets stuck don't abandon the test (though for all intensive purposes your game is most definitely not ready for market right now) - let them sweat it out for a while. If they still can't progress I find it useful to ask them what they think they're trying to do rather than to help them immediately. The need to ask this question and the response they give you will pave a path towards a solution.

(3) Test with Friends of Friends

Why friends of friends? You'll want to test with someone who isn't worried about hurting your feelings. These tests will also take a bit longer to organize and you'll want to use them wisely so I would suggest fixing as many issues from phase (2) as possible before moving on.

This process is very similar to (2) but you should treat it as much more formal. Imagine these people are part of your target audience. At the end of the playtest, it's useful to have Q+A about the session and anything you're trying to quantify or improve. Some example questions:

• How useful did you find the tutorials?
• How intuitive did you find the controls?
• How helpful did you find the in-game readouts?
• General thoughts on gameplay?
• General thoughts on story?
• How usable did you find the menus?
• When would you have stopped playing the game and why?
• Any other comments/suggestions?

You could ask these questions at phase (2) also; the point of Q+A is to open a forum for feedback and get an insight into problems/suggestions you may not have even considered yet. Consider all feedback and use it to improve your game.

(4) Test with the Public

The final phase is to test with a broad community: either through releasing a demo online or by attending an indie game event. Being able to directly observe your testers or viewing a video of their gameplay session is infinitely more useful than trying to interpret comments on a forum but don't be afraid to ask follow up questions if you don't understand someone's response.

This phase should be considered hands-off: the core goal is to get a large volume of testers to play your game and expose new issues or underline old issues you haven't quite fixed yet. Be available to fix/bypass any showstoppers of course but try not to interfere. Take notes - as always - and feed this back into your development cycle.

Other Points

This seems like a lot of work - and it is - but the point of playtesting is to prevent design issues from snowballing. Some other points to consider:

• Keep your original goals in mind and how you can tweak your design to achieve them
• Playtest important milestones - first playable, vertical slice and alpha are especially important
• Substantial changes to existing mechanics or adding new ones requires more playtesting
• People unfamiliar with your game are more valuable than previous testers
• Try not to go round in circles - every fix should move the game forward
• Try to look for trends - these are undoubtedly the "must fix" items
• Try to prioritise feedback before proceeding and set reasonable goals for addressing it
• Don't hide behind fancy art/sound in early demos - you want to expose design flaws
• Be wary of the audience you are targeting: don't try to make a ‘game for everyone'
• Quality assurance should not be the sole stage where playtesting occurs
• Stay positive - take every page of feedback as a sign your game needs more time

Conclusion

The topic of playtesting is complex and time consuming and I have no doubt missed a lot of important points but this is the process used on Observatorium and one that allowed me to go from this...

...to this...

...in just 3 months.

If you have any other thoughts or suggestions then add them below. I'm interested to hear how other Indie DB members approach playtesting.

Thanks for reading!

Clive Lawrence
The Man Who Flew Away