Getting your game play-tested

Hey Kirk!
I am trying to organize play-testing day/weekend in January here in So-Cal if you are interested. Let me know.
Daniel@radarlockgames.com
dano

You're going to have to let us know how this goes. I've been seriously considering using the service. Which package did you select (I would do the 15+)?

I went with the "Exploratory" session of 6 play-throughs with a light party-style game just to test the service. I want to see what kind of feedback they give, how it's formatted, and what they offer by way of suggestions to improve.

Cost of this level of testing is about $150.00, I also had them quote a deeper game (Second Line, which has been discussed here at length in other topics) and the quote there was $300.0 for the same "Exploratory" level of testing. Not cheap by any means...but depending on the return, may be worth it just to get an intensive evaluation that would otherwise cost a ton of personal time and effort. They know that meaningful playtesting by professionals is extremely valuable...we'll see how valuable when the reports come in.

The process and communication so far had been very professional and pleasurable, and I have nothing but confidence in them at this point.

My game is being sent to Gil Hova of Formal Ferret Games, and he will be lead tester, with around 15 total testers giving feedback.

I'll let everyone know how it goes!

I enjoyed my experience, however brutal the evaluation was. I sent a cartoony party game called "Death By Dice" for their testing, and while the feedback was very critical, it was also succinct and helpful in identifying where the game fell short of expectations.

In my case, the art was a big hit, and kept the players engaged. The "deaths" which had unique and meaningfully tied player powers (the basic play is roll to survive the "death," then gain its power to use against other players) were also well-received; but those cards which had repetitive powers or were less naturally/cleverly tied to the "death" were seen as lacking.

Most importantly, only about 1/4 players who liked the game said they would buy it...which was a big negative and what is really what caused me to shelf the game. Overall it was reviewed as being good, but not good enough. SOme players were turned off immediately due to the game being a light party-style game. Their response was noted, but tests were conducted with players who were open to such a game; so CGS makes sure your testers aren't detrimentally biased before they play.

As for numbers, a total of 19 unique players tested the game over 6 sessions, for a cost of around $135.

I would use them again, but plenty of testing by the designer is still required beforehand...unless you've got a few hundred bucks to throw around.

Keep in mind, when you ask people what they would do in such-and-such circumstances, they're often wrong. No, the don't lie, it's just been shown that often they don't do what they say they will do.

I'm skeptical that ANY game could actually achieve one quarter sales among randomly-selected playtesters.

I wouldn't let this alone put you off.

I've posted the player feedback for "Death By Dice" for anyone who would like to read it here:

https://www.dropbox.com/s/f4qux0dxm2srwu7/CG-B-061%20-%20Critical%20Eval...

Where do you live? Are there no meetups, conventions, or etc in the area?

I live in Honduras, and my wife is my only playtester. So, right now I'm only designing games that work really well with 2 players and aren't overly complex.

Perhaps you should try to design solitaire games that you can test on your own.

Thanks for the kind words, Chris! Glad you were able to get something good out of the service.

This is absolutely spot on. Engagement bias is real. Not only that, but our sample sizes are much too small to be relied on as definitive market research. I know that Gil took steps to make sure Chris's focus groups were diverse in their tastes, but this shouldn't be used as any kind of method to predict sales or your game's worth.

We actually find volunteer playtesters out in the real world. These are often friends, family, and total strangers. Our developers are all game designers themselves, and often use their own circles of testers when taking on a project for Coalition. We space tests out to avoid the kind of saturation you're talking about.

The payment goes to the developer, but we don't actually pay the testers themselves for a couple of reasons. First, even if we paid them minimum wage, the service would be absurdly expensive. Secondly, paying a tester is a bit counterproductive--not because it will make them lie, but because we're looking to pass on the value of blind playtesting. If someone accepts money to play a game (especially in American culture), they're likely to apply a work ethic to the situation, and this will change their natural reaction to the game. The only incentives we offer are usually just food/drink/hospitality, which is more common courtesy than tangible payment.

Unfortunately, our data isn't on this kind of scale. The kind of marketing statistics you're looking for just aren't affordable or practical for independent designers.

I can say that, on a purely subjective comparison, 25% willing to buy has been on the low end of average for the games we've evaluated so far. Take that with a grain of salt though! Willingness to purchase is very difficult to evaluate because it's such a difficult question to answer accurately on a survey. Many factors play into a purchase.

I'm sorry your quote felt unreasonable! Pricing is something I've definitely struggled with. I want to make our services as cheap and accessible as possible, but I also want to make sure my guys are compensated fairly and that we can pay the bills.

Our rates scale to the length (in minutes), rules weight (a factor determined by manual word count), player count (some upcharges apply beyond 5 players, just because it's more difficult to arrange the sessions), and the number of sessions you request. Our services are also modular, so you can build the package that best fits your needs--blind testing with an evaluation report is less expensive than iterative testing with development and design collaboration.

You can find all the math behind our services in our rate reference at http://coalitiongames.com

Send 'em our way! We have six developers in North America and Europe, each with a network of dozens of regular testers.

Another thing to keep in mind is that you can have Coalition only test certain parts of your game instead of the whole thing. If you had them playtest everything but the Spacewars scenario, you wouldn't have to worry about it increasing your average game length.

I live in Latvia. The population of entire country is less than 2 million. Boardgamers are rare to come by.

I use Tabletopia to playtest. I`m only building actual physical prototypes if I go to an expo or something.

Everybody has internet and anybody can use Skype.

I like to think playtesting should be done in 3-phases:

1. Solo Playtesting and with close friends (who are patient enough to deal with a rougher game). This should be done as soon as you have a workable game concept and rough prototype. This phase will point out the MAJOR issues with the game.

2. Playtesting with broader friends and family. You should have a moderately functional game. Hopefully you've done a lot of Phase #1 playtesting previous to this (rough estimate, at least 5-15 playtests). This phase will point out the MAJOR and MINOR issues with the game. This phase will take awhile, so don't skimp on it.

3. Blind Playtesting and Sending to Reviewers. Expect many many many playtests before getting to this point (I'd say at least 25-50, depending on the depth of the game and your experience). But this step is important, because it will validate (or not) that you have an enjoyable game that may do well in the market. Additionally, blind playtesters and reviewers don't care about your feelings (which is good). They will be blunt and give you the most constructive feedback. Do not underestimate how important this phase is.

I haven't used Coalition Game Studios, but I want to for our first game. If you truly plan to publish your game, $150-300 is a great investment. If they give you extremely positive feedback, you can feel that much better about your game. If they give you negative feedback, you can make improve your game before wasting a lot of time and money trying to publish it. It's a win/win.

Looking through the feedback, they seem to be insightful and thorough. I'd take their feedback to heart.

Best of luck!