[Icehouse] Reliability of the contest results and thoughts

[Jorge Arroyo]

Hi all,

I went ahead and checked how removing some ballots affected the position of
my game, and found that by removing just 2 (the ones where it was voted the
lowest) it jumped from 7th place to being tied for 2nd place. What this
tells me is that we're playing with such a small number of rankings that
just one or two people can make the results change drastically. This doesn't
seem too good, but I can't really see a way to fix it. Of course, if all the
contests before this one had similar number of ballots sent in (except for
the last one, in summer) then that says a lot about the reliability of those
results too.

Given the current situation, a designer with some friends that sent votes
for his game has the potential of taking the competition home unfairly, and
this just shouldn't be possible. It's also possible to vote strategically
instead of sincerely and be able to affect the results heavily. As an
example, if I change my ranking and instead of having Martian12s as first, I
put it as last (what I would call voting strategically given that I already
knew M12 was the one game that could win), I could force a tie between M12
and WreckTangle. Just one person can do that! Note that I'm not saying this
is what has happened in this contest, I'm just pointing out the
vulnerability of the system given the low number of votes. For the record, I
told my gf not to vote because I didn't want to make it seem like she was
voting my game just because it was mine... now I feel stupid.

I can't offer a solution, but I'll offer a suggestion to at least make the
competition more useful. Make it a requirement to offer feedback before
sending in the votes, or offer feedback along with the votes to be released
when the results are. This will make sure, bad ratings are explained and at
least the designers can see what people didn't like about their games and
how to make them better.

And I'm sorry if I sound like a sore loser. I just think I came up with a
nice and original idea for a game that is really different from what is out
there and interesting. It probably needs refinement, of course, but again, I
don't think it deserved to be ranked 7th...

BTW, I've started a new piecepack competition where instead of games,
reviews of games are entered into the contest. We'll see how that turns out,
but I'm hoping it will be useful for designers whose games have no feedback.
Maybe in the future we could try that with the Icehouse system. Here's the
link: http://www.boardgamegeek.com/article/2096186

Cheers,

-Jorge
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.looneylabs.com/pipermail/icehouse/attachments/20080220/83aa9a34/attachment.htm