Duelboard Rating System

[kitemaster]

On 3/7/2009 and 6/23/2010, two duelboarders, booger417 and Annypst, asked about the Riskee Rating system.  The answer they received was basically, "not meaningful and it will be removed soon."

I would like to discuss that topic ... in depth.

I am the formulator of the Riskee Rating.  I introduced it to Matlu when I noticed that the rating system he was using on Duelboard was in effective..."Meaningless".

The system used to give players ratings on Duelboard is the Elo System.  The Elo System is a method for calculating the relative skill levels of players in two-player games such as chess.  "Such as chess" meaning games based on pure skill.  Rook is not a game of pure skill...it is a game of skill and luck.  Each chess game begins with each player having almost exactly the same chance of winning because the resources for each player are identical.  Of course in Rook the resources are the cards which are randomly dealt each hand.  There is no guarantee that the hands will be equal in value...conversely, they are almost never of equal value.

The best chess players in the world will win practically ever game against an inferior player.  In Rook, the best players often lose to inferior players because of the inpredictabilty of what cards each player receives.

So...the Elo system is not an appropriate system for rating Rook players.  That is why you will see wild swings in Duelboard ratings.  You will see players with high skill rated low (mostly blue, but sometimes even green)...and you will see lesser skilled player with red and sometimes even pink ratings.

Matlu's philosophy seems to be "let's have fun and not worry about the ratings", and he wants the better players to play the weaker players.

I would like to point out that Rook is a competitive game.  In competitive games, you keep score and you strive to win....and it is nice to be rewarded to the degree that others can readily see that you are a good player of the game.  That's the purpose of rating systems...and also the reason they keep score in any competitive game.

The rating system on Duelboard takes away from the meaning of a competitive game site.  I'd like to see it changed.  I'd like to see the Riskee Rating used full time and the current one scrapped.

A few words about the Riskee Rating.  It is based on 3000 points with an average player at 1600 points.  It is calculated based on the number of wins and losses the player has accumulated.  The formula subtracts losses from wins and multiplies the resultant number by the bonus pool of points (2000) and adds the result to the median score of 1600.   When wins are more than losses the bonus points make the players rating higher.  When losses are more than wins the bonus points are subtracted (in effect) making the players rating lower.  A factor is that losing scores are divided in 1/2 before being subtracted to the median score (1600) in order to keep players ratings from getting too low (embarassment).

Since it is impossible to win a high percentage of rook games, you can expect ratings to top out at around 2000 points for very skilled players.  A 2000 point rating is accomplished by winning 60% of your games. 

I have tried to convince Matlu for about 2 years to implement this rating system...but, to put it kindly, does not seem to value the need of a meaningful rating system.

So...for me...I will not be participating on Duelboard anymore as of today...unless a meaningful system is put in place.

Thanks

[hammike]

The way the rating system is now, you lose approximately twice as much for a loss as you gain for a win against the same opponents, even if they are rated higher than you.  That means you have to win over 2/3 of your games in order for your rating to increase.  If you win 55% of your games against equal or higher rated players, your rating should slowly increase.  That is not the case now.  The current rating system is completely useless, the win/loss record tells you more about a persons skill than the rating.

[rookmeister1]

You are absolutely right about the current system not adequately addressing the "luck" factor of rook.  However, unless I missed something, the Riskee Rating also has a major flaw:  it does not adequately address the difference of skill factor between players.

The current system penalizes one too heavily for competing against people of a lower skill level because it does not factor in the "luck of the deal".  The Riskee system, however, does not have adequate protection in place to prevent people from padding their record and their Riskee Rating by playing consistently against inferior competition.  Under the current system, it is hard to pad your rating by playing less skilled players because the 1 in three or 4 times that they do win will cost you significantly.  In the RR system, I could sit and play new players and green players who are afraid to bid over 80 or 125 all day and run my W-L record and my rating up.

It seems to me that the current system need to be tweaked more than replaced.  Lower the penalty for playing against lower-rated players to address the luck factor, but leave it in place enough to provide incentive not to try to pad.  Or, tweak your system by weighting the value of wins based on the quality of opponent.  Both systems, individually though, have inherent weaknesses that can and will be exploited as currently described.

[kitemaster]

rookmeister...I appreciate your input.

You are correct to a degree...but even weak players can win....and I for one am not interested in playing every game (or even many games) against weak players.



I cannot express how much I appreciate your considering this subject and then making an addition to the topic.   Duelboard has not touched it's potential...but changes must be made!

[RGVBaptist]

No rating system will be perfect.  Unfortunately, this one is too far FROM perfect to even be considered good.

If I am "nice" and play a lower rated player I HAVE to beat them or I am severely penalized.  In many instances I could lose the first game due to "bad luck" and then win the next 3 and still wind up down in points.

So, something has to be done to address that.

BUT, more reasonably the games should be between relatively closely matched players, in terms of score.  That's the whole principle.  And to do that you have to create a system where players are allowed to better match themselves.  First, you need a BOOT feature to keep lower players from "blocking" a higher rated table.  Second, you need a MINIMUM RATING feature to be set for players to be able to join a table.  Third, it would REALLY be good to have the ROOMS to be by rating, so that when you have 3 rooms going the players that are more likely to play each other are all in the same room.

[Ms_Deal]

Why not just add 5 points if u win and 5 points if u lose.   Keep win/loss record in place.  As far as figuring in 'luck of the deal'..that's the case in any card game, plain and simple.

[kitemaster]

Thanks for your input RGV and MsDeal!

A comment to MsDeal...and others who have mentioned the idea that giving a set number of points for a win and taking a set number of points for a loss...

Let's say we have 2 players who have 60% of their games...i.e.  player 1 has played 1000 games and player 2 has played 2000 games.

Calculate their ratings per  +5 pts for a win   -5 pts for a loss

Player 1    60% * 1000 games = 600 wins and 400 losses
    This player has 200 more wins than losses so...200 * +5 = rating of   1000

Player 2   60% * 2000 games = 1200 wins and 800 losses
    This player has 400 more wins than losses so...400 * +5 = rating of 2000

You can see that players of equal strengths (as these 2 are) will have ratings that do not reflect their strength because of the number of games played.

With a rating system we are hopefully going to end up with numbers that reflect playeing strength.   See part 2 please.

[kitemaster]

Part 2

Now let's take those same two players and see how their ratings compare when calculated with the Riskee Rating System.

Player 1   600 wins (60%) out of 1000 games played means this player will receive 20% of the bonus rating pool..why because the difference in the players win/loss percentages is 20%...so 20% * 2000 (the bonus pool) = 400 points.
Final step add bonus points to the standard startup rating:  400 + 1600 = 2000
Player 1's rating is 2000.

Player 2  1200 wins (60%) out of 2000 games played means this player will receive 20% of the bonus rating pool...why because the difference in the players win/loss percentages is 20%...so 20% * 2000 (the bonus pool) = 400 points.
Final step add bonus points to the standart startup rating:  400 + 1600 = 2000
Player 2's rating is 2000.

So as you can see both players have performed the same, their rating is the same.  Can we ss the strength of these two players in their rating...YES.

[Ms_Deal]

Dear Kitemaster,

I understand exactly what you are saying...and you are right...But you have to keep in mind that this is online rook, therefore it's not going to work to do as u want.  Unless you have on set lobby for just that type of competition.

Sometimes you get online  to play and there aren't those particular players online at the same time to play with.  So, you are forced to play with just anyone.  If you were playing 'offline' with players as you are referring to then your system works great.   On here it's not going to happen.  \

I enjoyed it when we had the riskee rating system.   Maybe Matlu would  consider putting it back on  here along with a system that more conforming to online playing.

You come back to rook and I'll play with you or against you anytime.  LOL

[matlu]

I believe that how the rating works now (ELO) *would* be an ideal way to compute meaningful rating, but only under two conditions:
- people would play with anybody and would not refuse any partner or opponent
- people would not cheat
These conditions make sure that it works e.g. in chess, but it's impossible to guarantee it in online card game. Another thing is that Rook is very high variance game (the "luck" factor is much higher compared to e.g chess), which causes extreme swings. It could perhaps somehow work for big websites with very high number of players, where you don't meet same people very often, and you can easily find many people with similar rating, but Duelboard is not one of these big websites

So after all, yes, I agree with you that how it works now is absolutely wrong and meaningless.
And the best thing I can think of is to simply make people gain/lose a fixed number of points for each won/lost game. I'm not saying that it will become meaningful, but it will probably have more pros and less cons than how it works now.

I don't think that computing a win/lose ratio and trying to pass it for "rating" makes any sense. It's major drawback is that it simply stops changing after playing enough games, which would make people think that there is no progress, another drawback is that it brings no new information, you can already see the number of wins and losses. I don't mind showing win/lose percentage somewhere in the profile, but it don't think it makes any sense to make further operations with this number and pretend it's a "rating".

Anyway, I will not make any changes until February, I hope you understand. Then I will do my best to change it.
I think that the idea for fixed points for win/lose could be improved by awarding people extra points for some achievements, e.g. you could get extra points for each hand when you collected all 120 points, or something like that. You can make suggestions.

[hammike]

My current win/loss shows me winning close to 60% of my games (almost 1900 played).  If I were to start playing against those with green ratings, I would lose so much on a loss that I would soon be green too, even if I continued to win 60% of the games.  You should not lose 3 times as many points in a loss to someone with a green rating as you gain with a win.  It makes no sense.  If you are winning above 50% of your games, your rating should be increasing.

[kitemaster]

I would like to respond to matlu's message regarding the rating systems.

Regarding "Elo would be ideal if:
- people would play with anybody and would not refuse any partner or opponent"

In chess Grandmasters play players rated less than master about once in a blue moon...AND...even if they did THEY WOULD ALWAYS WIN....ALWAYS WIN.....Unlike Rook where even the strongest players can be beaten because of the inequalities of random cards dealt to the players.  THIS IS WHY STRONG ROOK PLAYERS DO NOT WISH TO PLAY GREENS...NOT ONLY ARE THEY SUBJECT TO LOSING, BUT THEIR RATINGS TAKE A TERRIBLE BEATING WHEN THEY LOSE.

ELO IS NOT AND CAN NEVER BE APPROPRIATE FOR RATING THE GAME OF ROOK!

- people would not cheat...I do not cheat because there is nothing to be gained for me personally by cheating.  I believe at one time cheating was rampant on duelboard, and tho I'm sure it still happens, I really don't think that it happens as much.

A rating system based on win/loss % is DEFINITELY the only valid BASIS for a rating system for Rook!

Matlu, you seemed concerned that after a certain number of games the rating of players will not noticeably change.  THAT'S WHAT WE ARE AIMING FOR!  The purpose of a rating system is to "find out just how good a player is."  The more games that are played, the truer the rating becomes.   THE LACK OF MOVEMENT IN A RATING IS A SIGN THAT WE HAVE IDENTIFIED THE STRENGTH OF THE PLAYER!  THE LACK OF MOVEMENT IS A DESIRED CHARACTERISTIC OF RATING...NOT SOMETHING TO BE AVOIDED!

I have said this to you many times in private conversation...now let me say it publicly... It's a fact that Duelboard is your site, to do with as you please....but, your rating system does not work  and it is a detriment to the popularity of the site.  Change it to what I have suggested and you will see the benefits in short order...I would bet money on it!

Duelboard is our only option for playing Rook.  I'm sure you will have a hard time seeing this, but, if properly handled it could be what the USCF is to chess.  There is a need for expanded features (i.e. the ability to play tourneys on the site...without having to go to another site to sign up for instance).  BUT UNTIL YOU UNDERSTAND THAT THE CURRENT RATING SYSTEM IS IMPORTANT ENOUGH TO CHANGE...WE ARE STUCK IN FIRST GEAR.  THE RATING SYSTEM IS THE HEART OF THE SITE...AND CURRENTLY THE HEART ISN'T BEATING!

I have played the game of rook with my extented family for over 50 years.  We have some wonderful rook players in my family.  I love the game and would like to have a site that is serious about it!  Please try the Riskee System for 2 months....just try it...if you don't see positive results...I will promise to shut my mouth on the subject!

I would like for those players who understand this situation to SPEAK UP!  If you don't understand the situation...PLEASE DON'T SPEAK UP!

By the way Matlu, thanks for fixing the scroll bars.

[kitemaster]

Yes...I am trying to find an acceptable solution to the rating system, other than the Riskee Rating System which you do not accept as viable.

I recently visited  kaggle.com, a site that proposes projects of various kinds where people (experts in most cases) are asked to find a "better solution to a problem."   The latest challenge is entitled "Elo versus the Rest of the World." 

In this project they are looking for rating systems for chess that outperform the currently used Elo system.

I posted a question in the forum, and although I feel there is a better solution, I did get a suggestion.  The replier said the Elo system could be used to rate Rook players, but the k factor would have to be low for it to work.

I beleive, Matlu, that you are probably using a k of about 30...that is why the ratings swing so violently up or down when players of high and low ratings play and the lower rated player wins.

I would suggest that you try a k of 3, 4 or 5.

Just try it...ratings should then not move so much and hopefully one day stablize.

Just think, you'd still get to use Elo (the chess rating system), and the duelboard ratings would be (at least a little more) meaningful.

ty

[matlu]

I believe I use standard 32 as a parameter. I think that when I make it lower then everything is going to stay basically same, just on "smaller scale". But I can give it a try anyway.

Talking with that people who search for better way to compute rating it should be clearly explained what game is it for, because if they are mainly interested in chess then it may not be suitable for rook.
[edit] It should also be noted that it's for a partnership game, which further complicated things. E.g. currently I average rating of opponents and use this average as "opponent rating".

[kitemaster]

Thanks for the explanation...I thought you were probably averaging the partners ratings, which would seem appropriate.

If you wish to "see" the exact conversation go to  http://www.kaggle.com   click on the 'Elo versus Rest of the World'  then choose 'Forum' and the topic which mentions 'Rook'

I hope you will act on the k value...I do believe it will make a difference, and in a month or so we should see more realistic ratings.

TY