Rating campaigns
Moderator: Forum Moderators
Forum rules
Before posting a new idea, you must read the following:
Before posting a new idea, you must read the following:
-
Clint Eastwood
- Posts: 13
- Joined: December 27th, 2005, 5:22 pm
Rating campaigns
The campaign server is chaotic, there is many different campaigns, some good, some bad, some unfinished, some are multiplayer eras, other are not.
Campaigns, maps, and multiplayer eras should be separated, and there should be a way of rating campaigns, like putting a campaign on the top of the campaign server called "most popular campaigns", that links to a website were users can vote from 1 to 10 on each campaign using their forum account. This way new players could know earlier wich campaign they are most likelly to enjoy.
Campaigns, maps, and multiplayer eras should be separated, and there should be a way of rating campaigns, like putting a campaign on the top of the campaign server called "most popular campaigns", that links to a website were users can vote from 1 to 10 on each campaign using their forum account. This way new players could know earlier wich campaign they are most likelly to enjoy.
"I tried being reasonable, I didn't like it. "
Clint Eastwood
Clint Eastwood
-
Casual User
- Posts: 475
- Joined: March 11th, 2005, 5:05 pm
-
Eleazar
- Retired Terrain Art Director
- Posts: 2481
- Joined: July 16th, 2004, 1:47 am
- Location: US Midwest
- Contact:
It would seem a shame for a campaign to get low marks because it was placed on the server earlier than another campaign, and thus piled up some low marks. On the other hand one of the criterion that player will look for is completeness. On the other, other hand, i only counted 14 campaigns that claimed 1.0 status or above.
Since feedback is important to getting a campaign polished and finished, i think some sort of rating for unfished campaigns would be helpful to the community.
The question deserves some though, but even a flawed rating system is more useful than nothing...
Since feedback is important to getting a campaign polished and finished, i think some sort of rating for unfished campaigns would be helpful to the community.
The question deserves some though, but even a flawed rating system is more useful than nothing...
Feel free to PM me if you start a new terrain oriented thread. It's easy for me to miss them among all the other art threads.
-> What i might be working on
Attempting Lucidity
-> What i might be working on
Attempting Lucidity
I'm sorry, but I strongly disagree with this... sure, right now, there are many bad campaigns mixed in with the good. But if we have a flawed rating system, it will mean that some good campaigns get marked as bad ones. That is, IMHO, much worse than just not sorting the campaigns at all.Eleazar wrote:but even a flawed rating system is more useful than nothing...
IMHO, it is necessary to be able to change your rating of a campaign. It is almost certain the quality will go up, so votes from 0.1 shouldn't apply to 1.0 of a campaign. That, or having ratings "expire"... only the last 30 or so ratings are counted for the average.
For I am Turin Turambar - Master of Doom, by doom mastered. On permanent Wesbreak. Will not respond to private messages. Sorry!
And I hate stupid people.
The World of Orbivm
And I hate stupid people.
The World of Orbivm
How about this?
All ratings persist. Any rating for the current version never expires. Ratings for all previous versions are weighted less based on how old they are.
I'm pretty sure you can degrade it exponentially and not have to keep track of the age of each group of votes.
All ratings persist. Any rating for the current version never expires. Ratings for all previous versions are weighted less based on how old they are.
I'm pretty sure you can degrade it exponentially and not have to keep track of the age of each group of votes.
Hope springs eternal.
Wesnoth acronym guide.
Wesnoth acronym guide.
I don't know it all seems good and bad (more on the bad though) in a sense.
I mean sure it'd be good to get someone else's opinion but...
How do u know that you agree with the rating until you download and play the campaign, your own idea could be completely different 2 the next person.
Plus as much as this is probably going 2 sound stupid or someone will find a flaw, i do believe that some people who have tried, tested, painted and further more wasted their own time 2 create a campaign would probably be most offended if their campaign got the lowest rating
.
I mean sure it'd be good to get someone else's opinion but...
How do u know that you agree with the rating until you download and play the campaign, your own idea could be completely different 2 the next person.
Plus as much as this is probably going 2 sound stupid or someone will find a flaw, i do believe that some people who have tried, tested, painted and further more wasted their own time 2 create a campaign would probably be most offended if their campaign got the lowest rating
.
This is off-topic, but really...WolfyX wrote:I don't know it all seems good and bad (more on the bad though) in a sense.
I mean sure it'd be good to get someone else's opinion but...
How do u know that you agree with the rating until you download and play the campaign, your own idea could be completely different 2 the next person.
Plus as much as this is probably going 2 sound stupid or someone will find a flaw, i do believe that some people who have tried, tested, painted and further more wasted their own time 2 create a campaign would probably be most offended if their campaign got the lowest rating.
Is it such an effort to write the word "to"?
Putting "2" in everywhere does not make you appear any more intelligent, original or innovative. If anything, quite the opposite.
And anyway, it's not as if you are saving time. "To" involves only one more keystroke then "2".
And not everyone here speaks English as their first language. We have quite a few people from France and northen Europe. Filling your posts with downright senseless numbers just because you can't seem to type a two-letter word, will not make it any easier for anyone who has a less-then-perfect graps on English.
[/righteous speech]
-
Duke Guillermo
- Posts: 127
- Joined: October 25th, 2005, 9:32 pm
-
Eleazar
- Retired Terrain Art Director
- Posts: 2481
- Joined: July 16th, 2004, 1:47 am
- Location: US Midwest
- Contact:
The problem with that is, as soon as a designer updates a campaign, his rating is gone. (what other way is there "not to count?") A campaing once ranked at the top would drop to the bottom. That would serously penalize good designers who frequently update their campaigns.Duke Guillermo wrote:I propose that a voting system be implemented where only votes for the current version of a campaign count.
Feel free to PM me if you start a new terrain oriented thread. It's easy for me to miss them among all the other art threads.
-> What i might be working on
Attempting Lucidity
-> What i might be working on
Attempting Lucidity
-
Duke Guillermo
- Posts: 127
- Joined: October 25th, 2005, 9:32 pm
Good point.
A campaign designer should have the ability to label a campaign as 'in progress'. The campaign could still be rated like any other campaign. When the campaign is labeled 'complete' the campaign editor should have the ability to choose whether the ratings he/she previously received still apply. Even if the designer chooses for these to count they count for only 1/2 a vote. Once a campaign is labeled 'complete' all votes count. However, votes for past versions count as less with the release of each new version. (Votes for the current version count as 1 full vote, those for the version before it count for 2/3 of a vote, those for the version before it count for 1/3 of a vote, versions before this count for nothing) Once two versions of a complete campaign are released all votes from when the campaign was 'in progress' do not count at all.
For Example:
I release a campaign and label it as 'in progress'. It receives modest attention and receives an average of 8 from 10 voters. I then modify it and label it 'complete'. I feel that the 8 is a relatively accurate depiction of how good my campaign is so I decide for those votes to count. My campaign receives a 10 from 10 people. These 10s are counted as more than the 8s, twice as more. So, my campaign now has a rating of 9.33. I release a new version of the campaign. The new version isn't received well (the music was annoying). I receive an average of 6 from 8 people. These votes count as 1 vote each and the 10s count as 2/3 votes each and the 8s count not at all. This leaves me with a rating of 7.82. Etc.
This system allows campaigns to retain their past votes while making new versions count as more.
A campaign designer should have the ability to label a campaign as 'in progress'. The campaign could still be rated like any other campaign. When the campaign is labeled 'complete' the campaign editor should have the ability to choose whether the ratings he/she previously received still apply. Even if the designer chooses for these to count they count for only 1/2 a vote. Once a campaign is labeled 'complete' all votes count. However, votes for past versions count as less with the release of each new version. (Votes for the current version count as 1 full vote, those for the version before it count for 2/3 of a vote, those for the version before it count for 1/3 of a vote, versions before this count for nothing) Once two versions of a complete campaign are released all votes from when the campaign was 'in progress' do not count at all.
For Example:
I release a campaign and label it as 'in progress'. It receives modest attention and receives an average of 8 from 10 voters. I then modify it and label it 'complete'. I feel that the 8 is a relatively accurate depiction of how good my campaign is so I decide for those votes to count. My campaign receives a 10 from 10 people. These 10s are counted as more than the 8s, twice as more. So, my campaign now has a rating of 9.33. I release a new version of the campaign. The new version isn't received well (the music was annoying). I receive an average of 6 from 8 people. These votes count as 1 vote each and the 10s count as 2/3 votes each and the 8s count not at all. This leaves me with a rating of 7.82. Etc.
This system allows campaigns to retain their past votes while making new versions count as more.
In summation, you're wrong.
I like the exponential curve better.
Oh, I forgot to mention... another thing I really dislike about rating systems out is that any half-decent campaign can easily get at least a 5... there's no way to fix this, but it seems to me like if you're never even going to use the bottom half of the scale (except for truly horrendous campaigns) why have it?
That's why I prefer to have ratings be out of 5, not out of 10, and allow half stars. It comes out to the same, but psychologically, I believe it will make ratings be more diverse...
Oh, I forgot to mention... another thing I really dislike about rating systems out is that any half-decent campaign can easily get at least a 5... there's no way to fix this, but it seems to me like if you're never even going to use the bottom half of the scale (except for truly horrendous campaigns) why have it?
That's why I prefer to have ratings be out of 5, not out of 10, and allow half stars. It comes out to the same, but psychologically, I believe it will make ratings be more diverse...
For I am Turin Turambar - Master of Doom, by doom mastered. On permanent Wesbreak. Will not respond to private messages. Sorry!
And I hate stupid people.
The World of Orbivm
And I hate stupid people.
The World of Orbivm
Here's the algorithm. I think 6 months may be too long of a half-life.
Store the following:
(current version)
(current version release date)
a = current # of votes
b = current avg score
c = previous # of votes
d = previous average score
weight = exp(-x/x0)
x = (today - current_version_release_date) (in # of days)
For 180 days, x0 = 259.9
For 90 days, x0=129.8
score = (a*b + weight*c*d) / (weight*c + a)
When a new release is posted,
c = c + a
d = score
a = 0
b = 0
The current average is calculated every time someone votes
b = (a*b + vote) / (a+1)
a = a+1
The value of 'vote' is a delta. If I previously scored the campaign as a 5 and now I think it's a 4, vote = 4-5=-1. If I never voted, vote = 4-0 =4.
The client needs to store somewhere (like in the campaigns directory) a list of campaigns downloaded, the versions of each, and what the previous vote is (or 0 if never voted for it). Thus voter registration is performed by the existence of preference/cfg files. It's possible for players to mess with these files to commit voter fraud, but if we're not serious enough about this to make login accounts, then it doesn't matter. You could make it binary to prevent basic tampering.
I think 5 stars is sufficient. If a player has more finely tuned feedback he needs to give, he should just contact the writer personally. Give players a rating key to help ease the loss of half ratings:
0 = erase my vote for this campaign
1 = hated it
2 = didn't like it
3 = liked it
4 = really liked it
5 = loved it
Players should be told not to take into account the state of completion of a campaign when rating it. Players are going to want to give specific feedback on dialogue, plot, map design, WML usage, balance, etc. I don't have any specific ideas on how to handle this other than give them a link to a form post webpage that collects specific feedback on each area and e-mails it to the user.
We should get a mockup going.
Store the following:
(current version)
(current version release date)
a = current # of votes
b = current avg score
c = previous # of votes
d = previous average score
weight = exp(-x/x0)
x = (today - current_version_release_date) (in # of days)
For 180 days, x0 = 259.9
For 90 days, x0=129.8
score = (a*b + weight*c*d) / (weight*c + a)
When a new release is posted,
c = c + a
d = score
a = 0
b = 0
The current average is calculated every time someone votes
b = (a*b + vote) / (a+1)
a = a+1
The value of 'vote' is a delta. If I previously scored the campaign as a 5 and now I think it's a 4, vote = 4-5=-1. If I never voted, vote = 4-0 =4.
The client needs to store somewhere (like in the campaigns directory) a list of campaigns downloaded, the versions of each, and what the previous vote is (or 0 if never voted for it). Thus voter registration is performed by the existence of preference/cfg files. It's possible for players to mess with these files to commit voter fraud, but if we're not serious enough about this to make login accounts, then it doesn't matter. You could make it binary to prevent basic tampering.
I think 5 stars is sufficient. If a player has more finely tuned feedback he needs to give, he should just contact the writer personally. Give players a rating key to help ease the loss of half ratings:
0 = erase my vote for this campaign
1 = hated it
2 = didn't like it
3 = liked it
4 = really liked it
5 = loved it
Players should be told not to take into account the state of completion of a campaign when rating it. Players are going to want to give specific feedback on dialogue, plot, map design, WML usage, balance, etc. I don't have any specific ideas on how to handle this other than give them a link to a form post webpage that collects specific feedback on each area and e-mails it to the user.
We should get a mockup going.
Hope springs eternal.
Wesnoth acronym guide.
Wesnoth acronym guide.