Skip to Content
 

GDS Scoring System 'flaw'

33 replies [Last post]
Hamumu
Offline
Joined: 12/31/1969
GDS Scoring System 'flaw'

Xaqery wrote:

Thats not how I see our current system. Our current system says nothing about which games people think are bad. In our current system people pick the games they think should be in the top 3. If you are not in the top 1/3 of the picks it only means people didn’t think you should be in the top 3. You really can not make many more conclusions.

Yes, the above is why I think the current system does not leave anybody feeling dumped on. I mean, you do feel dumped on if you get zero votes, but if you take a moment to give it logical thought, you can see that it implies nothing other than that there were at least 3 entries considered better than yours by everyone (which is kind of sad, but being 4th best all around is good!). Whereas with a "These are worth a mention", you can logically infer (if you get no "mentions") that everybody truly hated your game.

Not that I'm arguing for super PC "everybody wins!" pats on the head, just that the "mention" concept is unnecessarily harsh.

I keep seeming to agree with Xaqery. Hopefully this means he voted for my entry. Speaking of which, I better submit my votes, since it seems nobody is going to explain Zoo Gang to me! Sorry, Zoo Gang author, can't vote for a game I can't figure out.

Ideally, I would like to get a little summation of thoughts on my entry from tons of people each month. But I know the reality is most people aren't the lazy self-employed scum that I am who enjoy sitting around writing up critiques. So I would definitely think it would be a bad idea to incorporate any kind of feedback requirement, or harder voting than something like the current "pick your top 3". This allows the fast-paced jetsetting social butterflies among us to gloss over it and pick whatever sticks out, while the pathetic basement dwellers can crank out detailed summaries and charts (Note: that includes me, although I have no basement). Anything more would hurt voter turnout, I think.

And on a more floaty note, I think having multiple categories of voting/winners would dilute the results. People will be less excited to have a win on their 'resume' if they won "most like the requirements" or "included the most mentions of sheep" or "best use of octagonal tiles". I think the whole thing would be a little watered down if it's not just "WINNER GDS 12-05!"

I say that with some trepidation, because the "best gadget" side vote was fun, and having a side vote like that in every contest would be fun. But I think there is a certain subtle harm in it. I don't know. I tend to overdo things, layer stuff on, and I always find that as simple as possible is better. Which is where we currently stand, and I think any modifications will muck it up some.

"Tain't broke!" says I, but like most of the nonbroke clan, I am nonchalant on the whole matter. Do what you will, I will participate.

Zomulgustar
Offline
Joined: 07/31/2008
GDS Scoring System 'flaw'

Quote:

Can anyone add any more?

We're agreed that we should all vote sincerely in the interest of giving feedback rather than winning, but that's like agreeing not to make symmetric moves in certain abstracts. Why make another rule when you can remove the incentive? If nothing else, there will be times when someone is unable to vote even if they intend to.

If you don't mind my transforming your list, we should probably include things the current system gets right that we want to maintain...so let's make it goals for the optimal system rather than problems with the current one.

1) Enough resolution to express preferences without forcing distinction between designs deemed similar.
2) Enough dimensions to express preferences on multiple aspects of designs, but still a single overall winner is chosen.
3) Entrant's votes can't affect their own game's relative standing.
4) Voters' opinions will have roughly equal weight in determining the outcome.
5) Voting is simple enough that the effort isn't discouraging.
6) Adminstration is simple enough that Bryk's Spontaneous Combustion Quotient remains within acceptable parameters.
7) The balance between praise and criticism should be such that the winners get an ego boost, but low scorers have little reason for depression.

Any more?

Right now, I'm really wishing I could split my 9 points 2-2-2-1-1-1...

Brykovian
Brykovian's picture
Offline
Joined: 07/21/2008
GDS Scoring System 'flaw'

Maybe I can make it a bit more simple for everyone.

I'm glad that people (for the most part) did not approve of my overly-convoluted new voting system ... shortly after I had it all down in words I wasn't overly taken with it. But I threw it out there as something to debate.

To be honest, I'm not real good with this everyone-debates/committee style of going about things ... if any of you would see me at work, you'd be surprised how against my grain all of this discussion goes ... but I'd rather it were a community-driven thing, and so we have the debate every other month. ;-) The discussion has confirmed a few things for me, however, for which I am better off.

Voting should be for the best game that meets the Challenge criteria ... so, in effect, you vote for *both* things -- a good game, and "meets the criteria". Each voter can weigh for himself how much to ding a game that is shakey on the critieria. It's all very subjective, which is why we use open voting.

I do not think that entrants should be forced to vote. There are infinite reasons what someone isn't able to or doesn't care to vote. If those reasons are "selfish", then that's something that person needs to work out for themselves. If someone doesn't vote, but still has submitted the best game -- that game should still be recognized for what it is.

I think that entrants should consider the "game" to end once their entry has been submitted ... the design Challenge *is* the game, the voting is *not* a game. The voting is simply a way for entrants to be helpful, nice, and in some cases altruistic ... some people might consider this sort of nicey-nice thing "stupid" -- and, again, they'll have to work through that.

Once your entry is in, consider the voting thing to simply be your way to give feedback on *everyone else's* entries. Please consider incentives from outside the system as well ... such as the mutual back-scratching that makes a community like this work. It is not a self-contained, zero-sum system -- even if you can prove it mathematically.

If this sort of squishy, non-formal type of thing causes you some intellectual uneasiness ... then maybe it's not your cup of tea. That's okay ... different strokes, etc.

However, I have come away with 1 change that might be helpful, and that is to allow voters to distribute their own voting points, instead of the fixed 5/3/1 format. So, starting with the January Challenge, I propose to make the following changes to the voting as long as there aren't any obvious holes in it:

  • Each voter has 10 points to distribute among 1 or more games ... the voter does not have to give out all 10 points
  • No game can be given more than 5 points
  • The voter cannot give points to more than 5 different games

This doesn't address Zom's original problems at all, but does address some things I've heard some voters mention from time to time.

-Bryk

[/]
Syndicate content


forum | by Dr. Radut