Rapid Playtesting

The scenario: You’re a lone gameplay designer working on a project that’s been on the back burner for too long (years), and you want to make the damn thing production ready ASAP.  The problem is, the levels are a mess, and worse still you’ve got a dozen more to work through afterwards!  Deadlines are looming and you have nothing to show for it.

What follows is the approach I’ve developed over the years, modified with a few techniques and ideas I picked up at work and elsewhere.  This refining process is best suited for levels and projects that are on that last ‘20% is 80% of the work stage’ – i.e. built, scripted, but very buggy and needs improvement.  It’s not intended for initial design work, which requires proper planning; likewise we are not adding new features or content, unless the benefits and workload are clearly understood.  You’ll need a notepad (essential for any design work) and ideally an alarm clock (I use my mobile’s timer function).

The overall idea is to work in passes and refine the work with each iteration.  For your first pass, play through the level for around 25 minutes or so, and note down everything that stands out as wrong, not right, a little off, and needs work.  I recommend that you focus on the critical path to begin with – that is, play the level as it was intended to be played.  At this stage, there’s not a lot of point rooting out those obscure bugs if the core level design has issues, and remember that our key objective here is to make the level production ready as soon as possible.  Nevertheless, it’s still best to jot down everything that stands out so you don’t forget it later.  I tend to use a single notepad page per level / pass, and write down each issue on a new line.  I use pen and paper because I’m working alone and the process is supposed to be simple and fluid.  You can easily flick back a few pages to see where you were, but the idea is to establish a prompt working rhythm and not interrupt that with over-complication.

Once you’ve done this, you need to break down the issues in terms of importance and estimated difficulty/time.  I class issues as: trivial, minor, and major, like we do at work, for familiarity; but given that this is an informal (and rapid) version of the ticketing process, I recommend that you use a system that makes sense to you.  For me, a trivial issue is one that can either be resolved in a matter minutes or can be written off; a major or critical issue is something that needs a bit more thought or absolutely has to be resolved before your level is production ready.  Adding content should always be considered a major issue, because it will likely require significant time and effort to code and test, and risks introducing new bugs into the project.

With the tasks estimated, the final step is simply to address and resolve them.  I’m not going to be prescriptive here – you may wish to resolve all the minor issues first – that’s fine.  However, there’s one final trick you can employ here to speed things along – set a time limit.  Specifically, give yourself 25 minutes and use an alarm clock or an egg timer to keep time.  Then blitz through the list and try resolve as many of the issues as you can before the alarm goes off.  Then stop, finish what you’re doing and take a five minute break.  You don’t need to be a certified Pomodoro Master to appreciate the benefits of this approach;  while achieving a state of “flow” is great; in my experience, it’s hard to control, and there’s a definite risk of flowing down the wrong creek.  Using a timer and a prioritised list keeps you focused, and taking frequent breaks reduces fatigue and keeps you fresh.

After the break you can either return to the list for another 25 minutes, or if enough of it is finished, begin a new pass, starting with another playtesting session.  When working through the list, I tick off each item that has been addressed, and during the next testing phase, I strike-through all the issues which appear to be resolved (thereby closing them).  Once all issues have been resolved to your satisfaction, you can either continue to seek out more obscure errors, or call it a wrap and move on to the next task.

Essentially the method described above is a sped-up version of Agile development, with each micro-sprint being limited to no longer than 25 minutes, and the core phases of QA testing and task estimation condensed down to a quick notepad exercise.  The simplicity and speed of the process makes it ideal for tasks like playtesting and polishing, while keeping you focused on testing and resolving genuine issues that will prevent your level from being shipped.  Lastly, the iterative nature of this approach means that the level should be becoming more refined with each pass as you continue to invest time eliminating problems.

The above graph is absolutely non-scientific or set in stone.  The point is to illustrate the logarithmic nature of time working vs. quality.  That’s not to say you should skimp on polish – hell no, it’s by far the most important phase, but there will come a point where your changes start to have marginal improvement, or may even be detrimental to the final product.  The trick is recognising when you’re at this stage.  I’m quite fond of Mike Birkhead’s mantra:

“Add things until it starts sucking, take things away until it stops getting better”

Potential quality is the game you’d like to make, or  imagine it to be like finished if only we could just…  I’m not going to say that it’s impossible for games to meet or exceed their potential, but certainly in my experience there’s always been a gap between the game you wanted to make, and the game you actually end up releasing. Good as it may be, they’re not the same thing; so you need to learn when to let go and call it a job well done.

Anyway, once you’ve got your level to a point that you’re happy with, and can no longer see any scope for improvements, the final phase before release is to get real feedback from other playtesters.  I cannot  stress enough how important this phase is – second opinions will give you a much clearer idea on what works and what doesn’t, highlight issues you’d never considered, and will help you prioritise the issues that really matter.

For the Real World, I used Dropbox and invited about a dozen people to try three levels and give me feedback.  This process is much slower, as people can take up to a week to get back to you; however, the core benefit of using dropbox is that you can address issues almost immediately: “Too difficult you say? Okay, try now – is it better or worse?”  This interactive approach is quite effective, because playtesters can suggest ideas and immediately see the results of their suggestions implemented in the game.

To give you an example, in Trinity’s level,  we had a crate blocking part of the corridor you’re supposed run down to escape the Agent.  On paper, this seems like a legitimate obstacle to frustrate the player’s escape a little, and create a bit of dramatic tension. This was an oversight on my part – having played through the level countless times, I’d simply learnt to avoid it.  The first round of playtesting immediately identified this as a flaw in the design, and as a result it was moved out of the player’s way.

In practise I did about three or four rounds of playtesting (1 month approx)– the biggest issue was the difficulty, and it was important that these complaints were addressed before releasing the level to the public.  This was done in stages – a limited release candidate was made available to download via a QR code, to a small audience – this was to make sure there were no showstopper issues with the mod (e.g. compatibility issues), and a week later the ‘final’ release was uploaded and mirrored.

Overall, I would say that the final release was better for this adopting this approach, and enabled me to develop, polish, playtest, and release a 4 level mod within a 2-3 month timeframe.  I’m pretty happy with the final result (which you can try here), and the majority of feedback has been positive.  Of course there’s always scope for improvement, but again, we return to that balance between meeting potential and actually releasing content for people to play.  At any rate, this is just the first release – I’ve left the door open for future refinement.


2 thoughts on “Rapid Playtesting

  1. I have been so happy to have found out a week or so ago that the progress and technology that made Katana, did not die a slow death. I want to whole-heartedly support this endeavour, but sadly I lack any substantial developing skills.
    But I still want to contribute, so I would like to donate. Where can I do this?

Comments are closed.