Rating campaigns

Brainstorm ideas of possible additions to the game. Read this before posting!

Moderator: Forum Moderators

Forum rules
Before posting a new idea, you must read the following:
Clint Eastwood
Posts: 13
Joined: December 27th, 2005, 5:22 pm

Rating campaigns

Post by Clint Eastwood »

The campaign server is chaotic, there is many different campaigns, some good, some bad, some unfinished, some are multiplayer eras, other are not.
Campaigns, maps, and multiplayer eras should be separated, and there should be a way of rating campaigns, like putting a campaign on the top of the campaign server called "most popular campaigns", that links to a website were users can vote from 1 to 10 on each campaign using their forum account. This way new players could know earlier wich campaign they are most likelly to enjoy.
"I tried being reasonable, I didn't like it. "
Clint Eastwood
freim
Retired Terrain Art Director
Posts: 1113
Joined: November 29th, 2003, 11:40 pm
Location: Norway

Post by freim »

There will be when someone gets around to code it.
User avatar
Casual User
Posts: 475
Joined: March 11th, 2005, 5:05 pm

Post by Casual User »

But campaigns still in developpement would be unfairly penalized...

I think only the campaigns marked as complete should be rated, the version number should give an idea how finished they are for the others.
User avatar
Eleazar
Retired Terrain Art Director
Posts: 2481
Joined: July 16th, 2004, 1:47 am
Location: US Midwest
Contact:

Post by Eleazar »

It would seem a shame for a campaign to get low marks because it was placed on the server earlier than another campaign, and thus piled up some low marks. On the other hand one of the criterion that player will look for is completeness. On the other, other hand, i only counted 14 campaigns that claimed 1.0 status or above.

Since feedback is important to getting a campaign polished and finished, i think some sort of rating for unfished campaigns would be helpful to the community.
The question deserves some though, but even a flawed rating system is more useful than nothing...
Feel free to PM me if you start a new terrain oriented thread. It's easy for me to miss them among all the other art threads.
-> What i might be working on
Attempting Lucidity
User avatar
turin
Lord of the East
Posts: 11662
Joined: January 11th, 2004, 7:17 pm
Location: Texas
Contact:

Post by turin »

Eleazar wrote:but even a flawed rating system is more useful than nothing...
I'm sorry, but I strongly disagree with this... sure, right now, there are many bad campaigns mixed in with the good. But if we have a flawed rating system, it will mean that some good campaigns get marked as bad ones. That is, IMHO, much worse than just not sorting the campaigns at all.

IMHO, it is necessary to be able to change your rating of a campaign. It is almost certain the quality will go up, so votes from 0.1 shouldn't apply to 1.0 of a campaign. That, or having ratings "expire"... only the last 30 or so ratings are counted for the average.
For I am Turin Turambar - Master of Doom, by doom mastered. On permanent Wesbreak. Will not respond to private messages. Sorry!
And I hate stupid people.
The World of Orbivm
romnajin
Posts: 1067
Joined: February 26th, 2005, 7:26 pm
Contact:

Post by romnajin »

Wow...I actually agree with Turin...that's really weird...
Sorry for the meaningless post
scott
Posts: 5242
Joined: May 12th, 2004, 12:35 am
Location: Alexandria, VA

Post by scott »

How about this?

All ratings persist. Any rating for the current version never expires. Ratings for all previous versions are weighted less based on how old they are.

I'm pretty sure you can degrade it exponentially and not have to keep track of the age of each group of votes.
Hope springs eternal.
Wesnoth acronym guide.
User avatar
Aisikle
Posts: 46
Joined: June 17th, 2005, 4:11 am

Post by Aisikle »

I don't know it all seems good and bad (more on the bad though) in a sense.

I mean sure it'd be good to get someone else's opinion but...

How do u know that you agree with the rating until you download and play the campaign, your own idea could be completely different 2 the next person.

Plus as much as this is probably going 2 sound stupid or someone will find a flaw, i do believe that some people who have tried, tested, painted and further more wasted their own time 2 create a campaign would probably be most offended if their campaign got the lowest rating :( .
romnajin
Posts: 1067
Joined: February 26th, 2005, 7:26 pm
Contact:

Post by romnajin »

True, but would it not also encourage them to do better?
Sorry for the meaningless post
User avatar
Zhukov
Art Contributor
Posts: 1685
Joined: November 9th, 2005, 5:48 am
Location: Australia

Post by Zhukov »

WolfyX wrote:I don't know it all seems good and bad (more on the bad though) in a sense.

I mean sure it'd be good to get someone else's opinion but...

How do u know that you agree with the rating until you download and play the campaign, your own idea could be completely different 2 the next person.

Plus as much as this is probably going 2 sound stupid or someone will find a flaw, i do believe that some people who have tried, tested, painted and further more wasted their own time 2 create a campaign would probably be most offended if their campaign got the lowest rating.
This is off-topic, but really...

Is it such an effort to write the word "to"?
Putting "2" in everywhere does not make you appear any more intelligent, original or innovative. If anything, quite the opposite.
And anyway, it's not as if you are saving time. "To" involves only one more keystroke then "2".
And not everyone here speaks English as their first language. We have quite a few people from France and northen Europe. Filling your posts with downright senseless numbers just because you can't seem to type a two-letter word, will not make it any easier for anyone who has a less-then-perfect graps on English.

[/righteous speech]
Duke Guillermo
Posts: 127
Joined: October 25th, 2005, 9:32 pm

Post by Duke Guillermo »

I propose that a voting system be implemented where only votes for the current version of a campaign count.
In summation, you're wrong.
User avatar
Eleazar
Retired Terrain Art Director
Posts: 2481
Joined: July 16th, 2004, 1:47 am
Location: US Midwest
Contact:

Post by Eleazar »

Duke Guillermo wrote:I propose that a voting system be implemented where only votes for the current version of a campaign count.
The problem with that is, as soon as a designer updates a campaign, his rating is gone. (what other way is there "not to count?") A campaing once ranked at the top would drop to the bottom. That would serously penalize good designers who frequently update their campaigns.
Feel free to PM me if you start a new terrain oriented thread. It's easy for me to miss them among all the other art threads.
-> What i might be working on
Attempting Lucidity
Duke Guillermo
Posts: 127
Joined: October 25th, 2005, 9:32 pm

Post by Duke Guillermo »

Good point.

A campaign designer should have the ability to label a campaign as 'in progress'. The campaign could still be rated like any other campaign. When the campaign is labeled 'complete' the campaign editor should have the ability to choose whether the ratings he/she previously received still apply. Even if the designer chooses for these to count they count for only 1/2 a vote. Once a campaign is labeled 'complete' all votes count. However, votes for past versions count as less with the release of each new version. (Votes for the current version count as 1 full vote, those for the version before it count for 2/3 of a vote, those for the version before it count for 1/3 of a vote, versions before this count for nothing) Once two versions of a complete campaign are released all votes from when the campaign was 'in progress' do not count at all.

For Example:

I release a campaign and label it as 'in progress'. It receives modest attention and receives an average of 8 from 10 voters. I then modify it and label it 'complete'. I feel that the 8 is a relatively accurate depiction of how good my campaign is so I decide for those votes to count. My campaign receives a 10 from 10 people. These 10s are counted as more than the 8s, twice as more. So, my campaign now has a rating of 9.33. I release a new version of the campaign. The new version isn't received well (the music was annoying). I receive an average of 6 from 8 people. These votes count as 1 vote each and the 10s count as 2/3 votes each and the 8s count not at all. This leaves me with a rating of 7.82. Etc.

This system allows campaigns to retain their past votes while making new versions count as more.
In summation, you're wrong.
User avatar
turin
Lord of the East
Posts: 11662
Joined: January 11th, 2004, 7:17 pm
Location: Texas
Contact:

Post by turin »

I like the exponential curve better.

Oh, I forgot to mention... another thing I really dislike about rating systems out is that any half-decent campaign can easily get at least a 5... there's no way to fix this, but it seems to me like if you're never even going to use the bottom half of the scale (except for truly horrendous campaigns) why have it?

That's why I prefer to have ratings be out of 5, not out of 10, and allow half stars. It comes out to the same, but psychologically, I believe it will make ratings be more diverse...
For I am Turin Turambar - Master of Doom, by doom mastered. On permanent Wesbreak. Will not respond to private messages. Sorry!
And I hate stupid people.
The World of Orbivm
scott
Posts: 5242
Joined: May 12th, 2004, 12:35 am
Location: Alexandria, VA

Post by scott »

Here's the algorithm. I think 6 months may be too long of a half-life.

Store the following:

(current version)
(current version release date)
a = current # of votes
b = current avg score
c = previous # of votes
d = previous average score

weight = exp(-x/x0)
x = (today - current_version_release_date) (in # of days)
For 180 days, x0 = 259.9
For 90 days, x0=129.8

score = (a*b + weight*c*d) / (weight*c + a)

When a new release is posted,
c = c + a
d = score
a = 0
b = 0

The current average is calculated every time someone votes
b = (a*b + vote) / (a+1)
a = a+1

The value of 'vote' is a delta. If I previously scored the campaign as a 5 and now I think it's a 4, vote = 4-5=-1. If I never voted, vote = 4-0 =4.

The client needs to store somewhere (like in the campaigns directory) a list of campaigns downloaded, the versions of each, and what the previous vote is (or 0 if never voted for it). Thus voter registration is performed by the existence of preference/cfg files. It's possible for players to mess with these files to commit voter fraud, but if we're not serious enough about this to make login accounts, then it doesn't matter. You could make it binary to prevent basic tampering.

I think 5 stars is sufficient. If a player has more finely tuned feedback he needs to give, he should just contact the writer personally. Give players a rating key to help ease the loss of half ratings:
0 = erase my vote for this campaign
1 = hated it
2 = didn't like it
3 = liked it
4 = really liked it
5 = loved it

Players should be told not to take into account the state of completion of a campaign when rating it. Players are going to want to give specific feedback on dialogue, plot, map design, WML usage, balance, etc. I don't have any specific ideas on how to handle this other than give them a link to a form post webpage that collects specific feedback on each area and e-mails it to the user.

We should get a mockup going.
Hope springs eternal.
Wesnoth acronym guide.
Post Reply