Top tips: Commercial games testing

Foolproofers love sharing great ideas and debating all things UX related so, each week, we hold a Smart Lunch where we get together to feed our brains and tummies at the same time.

The most recent topic was “The unique challenges of commercial games testing.” Here’s my top takeaways from this session; great for those wanting to get more involved in the games industry or just looking to brush up their general UX consultancy skills.

Learn the lingo, avoid UX jargon
The games industry, like many industry sectors, is full of specialist terminology so make sure you get to grips with it so you can communicate effectively with your client and their customers.

Sometimes inexperienced UX consultants make the mistake of trying to impress clients by using UX jargon and buzzwords, games designers in particular aren’t interested. Don’t try to educate them, just help them to make their game the best it can be.

Keep pace, be flexible
The games industry is particularly fast paced so putting development on hold to wait for the recommendations from user research often isn’t an option.

One way of coping with unforgiving timelines is to invite clients into your ‘wash-up’ meetings so you can give them the headline findings they can use to continue their work whilst you dig deeper into the analysis. That way you can still provide them with the really valuable insights they need to help them provide a great experience for their customers and they don’t have to put everything on pause until the debrief.

Be succinct and clearly prioritise
Games designers don’t have time to read lengthy reports. Make your key messages very clear and try to get your point across in single line bullet points. It’s up to you to clearly prioritise your findings and recommendations.

Spreadsheets are king
PowerPoint reports are great for getting buy-in from producers and other senior stakeholders, but it’s the developers who actually have to make the changes you are recommending. Excel spreadsheets are a great way of giving developers a practical list of clearly prioritised issues and recommendations.

Observe from a distance – no more think aloud
UX consultants love to get users to ‘think aloud’ and ask them lots of questions whilst they’re using a product. This kind of technique works great for websites, where the interaction is mostly task based, but constant interruptions are not so great for most games where you’re trying to explore how ‘fun’ and ‘immersive’ they are to play.

Instead, try to gather feedback at the end of the session, during natural breaks in play or after key developments in the game. Alternative methodologies such as diary studies can also be extremely useful ways of gathering insights.

Ratings are useful indicators but caveat well
Where multiple rounds of user research are taking place, it can be helpful to use rating scales to indicate how changes to the game have affected the overall playing experience. However as with any qualitative style research and since you’re unlikely to be testing with hoards of people make sure you heavily caveat their statistical significance.

Since games get given ratings by reviewers it is also important to emphasise that just because users gave your game a rating of 8/10 at alpha testing stage it doesn’t automatically mean it’s going to get the same rating by critics when it’s fully finished.

Build a library of knowledge
As UX designers it’s great to keep an annotated scrapbook of good stuff you come across which you can draw on for inspiration in future projects and there are many great tools out there to help such as Little Snapper and Patternry.

Pin pointing and documenting the magic formula of a game is harder though as it’s more about the overall experience than specific interface elements, so annotated screenshots aren’t as much help. An alternative idea is to write a brief synopsis of games after you play them. Remember to note bits of both best and worst practice and try to capture the more emotional aspects. What was it that made it fun to play?

Use your power wisely
As with any product or service as a UX consultant you possess an immense power to radically improve the experience and enjoyment of games. However, remember that power is a double edged sword, be aware that your insights and recommendations may help decide the fate of the game and those who worked on it.

With games, poor decisions are amplified because games are reviewed so much; even a slightly weak experience can make a game bomb. So only report/propose things you are 100% confident in whether that is delivering findings and recommendations or proposing a novel study design. The team would rather have reliable results than supposition.

It’s doubly important when testing games to make sure you set out clear goals and objectives of the research before you start to ensure your research is able to uncover answers to the key questions. If you get asked questions that you can’t conclusively answer don’t bullshit, remind them of the scope/goals of the project and don’t be afraid to propose conducting further research to provide them with an answer if it’s important to them.
 

What’s your experience of testing games, leave a comment.

Read more about the games industry
The Art of Game Design by Jesse Schell

No comments yet.