Subscribe to this feed
Keeping IdeaWarz Fresh - Feedback Wanted
Once in a while, when the dust settles between releases, we actually get to do some thinking. This week we have been thinking a lot about IdeaWarz - specifically how to make it better. We know IdeaWarz isn’t perfect (and we thank all of you who keep reminding us every day 
This post is about sharing with you some of the thinking we’ve been doing, and getting some feedback. So without further ado, let’s review some of the areas IdeaWarz could be improved, goals for the next version, and what we propose going forward.
What could possibly be wrong?
The biggest problem with current version of IdeaWarz is that there is no finality.
All the ideas are in an open ended battle with no end in sight. We did not expect to have over 5000 community members and 2200 ideas this fast. When we launched back in June, our goal for the website was 100 community members and 1000 ideas. We’re not complaining - this is a good problem to have . 2200 ideas however, is too much to rank all at once.
Having so many ideas battling all at once means the truly good ideas aren’t getting enough eyeballs in front of them. We want the top ideas battling each other, not random weak ones.
The other challenge with an open ended battle tournament is the complexity of the ranking algorithm. Trying to create a ranking algorithm that takes into account the age of an idea, it’s relevance or weighting (whatever that means), and determining who it should battle next gets non-trivial really quick (and it still doesn’t nail the essence of what we’re trying to do).
The current version of IdeaWarz also lacks real incentive for people to pick the winners. For the wisdom of crowds to work properly, we need people to have a vested interest in the outcome. Currently it’s too easy for people to vote without caring about the result. We need to make the crowd care.
Finally, adding some rails to the process to give battles a sense of finality is something we have wanted to add from the beginning and are only now getting around to.
What can we do better?
First thing we want to do is make the ranking process more transparent. People should be able to clearly see the life cycle of their idea from the second it is submitted, to the moment it is picked for commercialization.
Secondly we want to find a way of reducing the number of ideas battling at once. Currently there are just too many. We need to be able to find the good ones fast, and let the bad ones go even faster. This will make the battles more efficient and interesting.
Third, we want to add a little more rhythm and ritual into the battles. Having battles and tournaments on some kind of schedule will give people that sense of finality, and leave them looking forward to the next skirmish.
Here’s what we’re thinking
Basically we want to move from an open ended free for all to a tournament model. That means groups of ideas battling each other over a certain period of time, with one winner declared at the end. This is similar to what threadless.com does with their t-shirt design competitions. Time boxing also adds rhythm and ritual. It closes the loop and gives people finality on their idea’s journey.
We think the life cycle of an idea could look something like this:
Week 1 : Accept ideas
- For an entire week just accept new ideas.
- There is no ranking at this point.
- They are simply collected and stored.
Week 2: Rank collected ideas
- Collected ideas are now opened up for their first taste of battle.
- Table napkin style battles would randomly choose ideas to fight.
- Results would be collected and rankings kept private until the end of the week.
- The top 32 ranked ideas (or top 5% which ever is greater) would progress to the next battle.
Week 3: Skirmish
- Out of the top 32 ideas, voters get to pick three.
- People’s votes are collected throughout the week.
- Idea submitters get an opportunity to expand and promote their idea to entice voters.
- Once again voting results are kept private to avoid group think.
- The winners then proceed to the 3-way.
Week 4: 3-Way
- Out of the three ideas left, voting members are asked to pick the best idea.
- This idea is the chosen one. It represents the single best idea submitted this week.
- There can be only one
The winner is then eligible to be commercialized via a market test. Hallelujah!
How often would we run the tournament?
We are thinking weekly or bi-weekly.
If we ran it weekly that would mean a new product market test every week. The nice thing about the weekly tournament is it keeps the number of ideas battling reasonable. The disadvantage of a weekly tournament is we lose some of our hype (it’s not as exciting having a winner every week).
Thoughts?
The Experience
Last thing I want to share for now is a little bit about the experience. We want the IdeaWarz tournaments to be fun, rewarding, full of fame, and all about the thrill of getting your idea commercialized. A key part of that will be giving people the tools and data to watch their ideas progress through the skirmishes and battles.
To this end we are looking at adding data feeds where people can receive stats and updates on things like:
- Battles won
- Battles lost
- Comments added
- # of views
Then people can specify how often they would like e-mail updates (hourly, daily, weekly) to keep tabs on how their ideas are doing.
Things we still need to work out
Wild Cards
MJ really wants us to have the ability to inject a wild card idea into a tourney at any stage. This gives us an opportunity to mix things up, and do tests on ideas we think are hot, but not sure about.
Existing ideas
Whatever new system we go with, we know we have 2200 existing ideas that need to be worked in. We haven’t decided how to do this yet. We could select the top ones and bring them over to seed the first battles. We could systematically take some ideas every week and battle them. Or we could throw them all into one big tournament (as a kind of a one off) and see how they do. We have lots of options here. Don’t worry, we haven’t forgotten about these ideas.
What happens to the losing ideas?
They get the boot. If cases where we have more then one truly outstanding idea, we may make an exception and promote two. But for the most part we really want to find that one diamond in the rough. People can always re-submit ideas, but we would encourage tweaks and changes to give them a better chance of surviving.
What about a stock exchange for ideas?
We are still playing with the idea of having a stock exchange where people actually trade on the value of an idea like a stock. This is still a ways off, but once we get our battles sorted out, this may be the next step.
What do you think?
Here are the top questions we would love your feedback on (along with anything else you can think of):
- Should scores be kept private during weekly battles to prevent herding?
- What should the length between tournaments be (1, 2, or more weeks)?
- Would you like to see a trading system for ideas / Royalty Points?
- What role would link / e-mail promotes would play in this style of tournament?
- What other experiences do you think would be fun and rewarding?
We know there are a thousand details still remaining to be worked out, but before we go pro on the idea, we wanted to hear from you.
Note: For further discussion on this topic please use the Cambrian House forum there here.
August 20th, 2006 at 6:38 pm
Okay guys, let me start off by saying props to you for recognizing this problem (I sorta complained about the “too many ideas” problem in an earlier comment, if you recall, so I am fully behind you on this) and tsk tsk to you for not expecting this much outpouring of ideas. With your belief in abundance, I’m surprised you expected such a paucity of input. But everybody doubts themselves sometimes, right?
ANYWAY, let’s get down to it.
Your 4-week model is great. I think it’s a wonderful way of encouraging people to “check in” on the situation, and really rally behind ideas that they either submitted or believe strongly in (or both). I also VERY strongly support your “3 votes only” later stage, which is similar to how Sun’s “Bug Parade” for Java works (everybody can cast up to 3 votes for as many as 3 unique bugs, but can throw all their votes to one bug if they wish).
So, following that logic, I think that if you want to create a true “vested interest” in choosing winners, you should let people have 3 votes that they can spread around as they wish. When it’s all said and done, some fixed-size pool of royalty points would be distributed to all of the “winning voters” proportional to the number of votes they submitted to the winning idea. Man, I’d be so all over that.
Okay, a few more comments…
Just an observation: Lots of “open” sites seem to go through this stage of thinking that you can keep everything forever and it will have value. SourceForge took FOREVER to figure out that sometimes you have to actually delete a project that’s dead and has no future. LiveJournal went through the same thing. The fact is, some things are just crap and should be flushed as such. At least you guys are coming to that conclusion in your Beta stage rather than years down the road…
“Link/Email Promotes”. Ugh. I say drop these. These are just going to lead to cheating, spamming, and god knows what else. You guys are good enough at marketing that you shouldn’t need to do this in order to get people voting/visiting. Please drop this “outside promotion” of ideas. I really feel it adds very, very little.
Finally, and I don’t know how this would fit in, but it’d be nice if people who submitted ideas had a DIRECT way of responding to criticisms of them. As it is, people respond in the comments section, but because of the comment-rating approach, the order gets scrambled so you can hardly tell who’s responding to what. Take a look at how comments work on LiveJournal. Look at my journal if you wish. (http://cybercerberus.livejournal.com) See how the comments can be responses to other comments? This is a feature many sites overlook, but creates “conversations” and is HUGELY valuable.
Okay…I’ll shut up for now.
August 20th, 2006 at 10:35 pm
And for those who Digg, here’s the link I submitted:
http://digg.com/tech_news/Cambrian_House_Rethinking_Voting_System
Support CH. Digg the story at that link!
August 24th, 2006 at 1:08 am
Hi guys,
how about adding a “tag cloud”. This will group all similar ideas together so that similar ideas can compete against each other. Apples versus apples. It will also act as a filter to prevent replication of ideas.
A tag cloud (more traditionally known as a weighted list in the field of visual design) is a visual depiction of content tags used on a website. Often, more frequently used tags are depicted in a larger font or otherwise emphasized, while the displayed order is generally alphabetical. Thus both finding a tag by alphabet and by popularity is possible. Selecting a single tag within a tag cloud will generally lead to a collection of items that are associated with that tag.
The first widely known tag cloud appeared on Flickr, the photo sharing site. That implementation was based on Jim Flanagan’s Search Referral Zeitgeist, a visualization of web site referrers. Tag clouds have also been popularised by Technorati, among others.
The first published appearance of a tag cloud can be attributed to the “subconcious files” in Douglas Coupland’s Microserfs (1995).
Source: http://en.wikipedia.org/wiki/Tag_cloud
/me
August 26th, 2006 at 11:09 am
Hi Cocoluv,
Great suggestion. We have been wanting to add tagging to ideas for some time, but I hadn’t thought of the tag cloud idea. Thanks for the suggestion. I would definitely like to explore this.
Cheers - JR
August 30th, 2006 at 3:07 am
I think it’s valuable to know how many voted an idea up and down. One idea may have 5 down votes and no up votes and that is totally different from another idea that has 100 up and 105 down. Some ideas will not appeal to everyone but what counts is how many people they do appeal to.
I also think it will still be valueable to save the losing ideas in a seperate section since sometimes I want to search and see if my idea has already been posted before and maybe contact others thinking about related ideas.
September 9th, 2006 at 11:00 am
Let members have RSS feeds for their ideas.
ie: I could put an RSS feed for http://cambrianhouse.com/engtech/ideas/feed in the sidebar on my blog.
The RSS feed would show the current ranking for my battling ideas, and also show historic/archived ideas by rank.
Lots of free press and inbound links for Cambrian House.
The first published appearance of a tag cloud can be attributed to the “subconcious files” in Douglas Coupland’s Microserfs (1995).
Hey, I was the one who added that to wikipedia. Cool.
September 25th, 2006 at 7:52 pm
Hi there!
Dont want to be negative, but I really think there’s a conceptual problem with Idea Warz.
While fun and highly interactive, Idea Warz is only good up to a certain level.
In the new way you are suggesting, just imagine: you’ll come up with 3 great ideas, that has overcome every major filter that can be imagined. And you will have to kill 2 of them!
Do you realize? You will be discarding 2 of the best ideas sent to CH!
Makes no sense to me.
My recommendation is to keep IW just in the first part of the process, that is to help the CH team detect the ideas that emerge from crowd.
After that, the job has to be less massive and more specialized.
Maybe the CH team should handle it. Maybe you can invite some qualified members to help you evaluate and rank ideas in order to have external opinions. You can reward them with some RP from the winning idea, so they will be really intrested in picking up a bunch of great products.
That’s my POV.
Un abrazo,
Robby!
September 5th, 2007 at 9:36 pm
Nice
September 15th, 2007 at 5:47 pm
Nice
December 28th, 2007 at 2:00 pm
Nice
January 5th, 2008 at 8:58 am
Nice!
January 9th, 2008 at 3:15 am
Interesting…
February 2nd, 2008 at 1:15 am
Nice!
February 2nd, 2008 at 4:53 pm
interesting
March 9th, 2008 at 3:04 am
Chippewafallssportsgrid3056$anchor$basketball Betting,final Four,final Four Betting,final Four Gambling,final Four Sports Book,final Four Sportsbook,march Madness,march Madness Betting,march Madness Gambling,march Madness Sports Book,march Madness Sp…
Chippewafallssportsgrid3056$anchor$basketball Betting,final Four,final Four Betting,final Four Gambling,final Four Sports Book,final Four Sportsbook,march Madness,march Madness Betting,march Madness Gambling,march Madness Sports Book,march Madness Spor…