Five Tribes Five Tribes

Forums

Search
Forums » T2R Competitive Play - English » ELO score distribution
Show: Today's Posts 
  
AuthorTopic
SOJA maunaloa
Member

User Pages
Posts: 69
Registered:
March 2009
ELO score distribution Sun, 24 May 2009 20:30
Hi all,

This is my first post. I'm very new, registered on this March. Also, please go easy on my poor expressions in this post since I'm not an English native speaker.

By the way, I made a distribution of ELO score as follows.

Linear scale version:
http://www.daysofwonder.com/uploads/userpages/528043/_2227_web.jpg

Log scale version:
http://www.daysofwonder.com/uploads/userpages/528043/_2228_web.jpg

In those fiures, X axis represents overall ELO score and Y axis represents number of players in each 10-point bin as of 2009/05/24. As expected, it is a Gaussian distribution with a tail towards higher score. I put some statistics in the figure for those who love statistics (Gaussian mean, sigma, median and average).

From the previous threads, I know quite a few people love statistics in TTR community although that's not common in the rest of the world.
      
Zeno
Senior Member
Cadet

User Pages
Posts: 582
Registered:
February 2006
Re:ELO score distribution Mon, 25 May 2009 05:03
First of all, welcome to the community, and thanks for the graphs, There has been a nice history of people applying their talents to the game, and I'm glad you joined in. In some ways it looks a lot like a graph I did a long time ago, but now I find that I put the pictures up on an old account, and they are no longer there. Never mind though, yours is better. It shows the nice central core, in which the ELO formulas enforce a normal distribution, along with strange features on the left and right, both of which are going to get stranger.

On the left, you have the people who play a game using the same account more than once. If you log in on your account, and give your friend the account info, having them log in on the same account, but a different computer, then when the game is over you automatically lose four points. Do this a couple of hundred times, and you can get a very low ELO indeed. With the guest accounts becoming good for only four games, expect more of this activity. Of course, if the games were played unrated, I assume there would be no problem, but some people just don't care about that, or don't know the effect their self-play is having.

On the right there have been several explanations. a) The top players protect their rankings (since they are more aware of their rankings, and since they cherish them, they do more to protect them). b) The top players tend to stick around longer because the top players get more reinforcement (aside from the flame wars that erupt from time to time, most are cordial, friendly, and while fiercely competitive, they share the game with humor and grace). c) ELO doesn't work. Perhaps all three are true to an extent, but I favor b. The data that you have appears to be more striking than mine, however, and I suspect that it too comes from the recent DoW changes. Though it is controversial, many top players have second accounts. It used to be common to find different varieties of guests. Some were clueless, most were common, but every once in a while you would go "wow". Either an average player had, by mistake, played brilliantly, or you played against someone's second account. It was rare for players to pay to keep two accounts going, but many had old accounts that had been player accounts and that recently converted. I know that I have a second account which was once a player account. I don't play that account (dewey, userid 221893), though now that I look at it, it is 13 pts higher than my own account Embarassed . Still, I suspect that many of the top players have multiple accounts, all of which appear on the right hand side of your graph. So that gives us at least four reasons for the skewing.

I hope this is the first of many interesting posts, but be careful. I know that I went through a period where I cared more about my posts than I did about the game. Have fun, learn things, make friends, enjoy the community,

Z
      
CIA Truckerteller
Senior Member

User Pages
Posts: 676
Registered:
October 2007
Re:ELO score distribution Mon, 25 May 2009 11:07
For some reason the mean/median seems to be somewhere close to 1200, yet when you start an account and play only friends who also just started an account you will all have an average of 1500, or maybe I'm mistaken here. It could be 1400, as that is the number to which accounts get reset for dubious behaviour.

On the other hand, when you're new and play random multi's with anyone who jumps in and you have little experience with strategy, your starting elo after 20 games will probably be close to the 1200 rank which seems to be the default rank for casual ticket drawing, make your route in a direct line and see what happens style of play.

My theory for the left skewness would be that there are clear distinctions in playing level between 1300, 1400, 1500, 1600, 1700 and 1800 (and more 2P games are played the higher up you go) yet between 1000 and 1300 distinctions are harder to make as people leave mid-game, draw 6 times or not and play is far more casual than competitive. To have a more normal distribution and equal tails on both sides would require a whole bunch of people and levels of self-destructiveness on the downside of 1000. You would need 100+ players @ <800 who make a sport out of it to finish below their <800 competitors.
      
SOJA maunaloa
Member

User Pages
Posts: 69
Registered:
March 2009
Re:ELO score distribution Mon, 25 May 2009 12:53
Hi Zeno,

Zeno wrote on Sun, 24 May 2009 17:03

First of all, welcome to the community, and thanks for the graphs, There has been a nice history of people applying their talents to the game, and I'm glad you joined in. In some ways it looks a lot like a graph I did a long time ago, but now I find that I put the pictures up on an old account, and they are no longer there. Never mind though, yours is better. It shows the nice central core, in which the ELO formulas enforce a normal distribution, along with strange features on the left and right, both of which are going to get stranger.



Thank you very much for your comments and observation.
Now I reminded that I wanted to make these plots because I could not see your graph in the following thread started by Wildfire2009 (Topic:TTR Ranking Distribution):

http://www.daysofwonder.com/en/msg/?goto=36450#msg_36450

I think I must make a link to this intensive long-term work when I started this thread. But after struggling to extract the score information, I forgot about it. I'm sorry. And I want to add one more link in which you showed the temporal change of the same ranking:

http://www.daysofwonder.com/en/msg/?goto=134199#msg_134199

As to the tail in the right side (higher ELO), I think this is exactly the ELO system intended to. If there is no skill component, you cannot go further to the right because of ELO's feed back system. But because of skill, higher ELO player can win against lower ELO player more often. There should be a slight discrepancy between the actual winning percentage and that assumed in the ELO system (and this is very interesting topic, too), but generally it follows very well. And that's why TTR online community grows rapidly, I think. Thus, how much #1 palyer (or top players) can go further in ELO score shows how much strategy components are in TTR. I think this is the same reason that explained in Truckerteller's response. I agree with him.

On the other hand, I should mention that I included all the bots in my plot. Maybe that's the main reason for the skew on the left. There might be bot's Gaussian distribution around 800.
      
CIA Truckerteller
Senior Member

User Pages
Posts: 676
Registered:
October 2007
Re:ELO score distribution Mon, 25 May 2009 14:10
I find the bottom part of the graph Zeno put up a year ago quite puzzling:

http://www.daysofwonder.com/en/msg/?goto=134199#msg_134199

What's happening to the section below the top 500, which seems to be gradually moving lower in ELO ? Are less people playing the game / active ? Are points leaking from the pool ?
      
kolmo
Senior Member

User Pages
Posts: 556
Registered:
November 2006
Re:ELO score distribution Mon, 25 May 2009 19:55
I like the chart ! I have only one minor qualm : it does not mean anything. An ELO that calculates your overall performances in so many different game variants and settings can only lead to nonsense.

Statistical work is already walking on a thin ice when comes time to show something meaningful. Let's not break it, shall we ? We should stick to outward behaviour of the community : performance and competence should be kept for more discriminate tools than the overall ELO.

Disclosure. Of course I am saying that because my overall ELO is plunging right now...

[Updated on: Mon, 25 May 2009 23:38]

      
Wildfire2099
Senior Member

Posts: 654
Registered:
June 2005
Re:ELO score distribution Wed, 27 May 2009 14:47
Nice to see another stat head.. and nice graphs.

I see you found my stuff, that's good.. your presentation is way better though.. I was never that interested in putting in the extra time to make pretty graphs.

Like Zeno, there have been times where the stats for the game have exceed my interest for the game itself, which is kinda ironic.

the think I find most interesting is with the elimination of guests, there are nearly twice as many ranked players... which has normalized the curve some. If you look at my stats, you'll see there was a strange gap from 1400-1500, that band was far lower that it should have been.. it seems that's where the guests were... which was actually not my favorite theory about the phenomenon, but definately explains it.

How to see you both playing and stating more in the future Smile
      
Yann
-= Crew =-
KS Backer - Skeleton

User Pages
Posts: 2355
Registered:
October 2002
Re:ELO score distribution Wed, 27 May 2009 19:04
Dear players:

The charts are cool, but one question: from which data were they built from? Extracted from the ranking section of the HTML page in your account?

Because you will see only the scores of people who played at least once for the past 15 days.

If you really want to measure the gaussian factor, you should take the ENTIRE list of players - since normally it is a zero-sum (energy conservative) formula. Well, at least for people with more than 20 games.


One quick study I made last week was to measure the correlation between the ELO score and the winning success of players. In other words, if you are better ranked than the other player by, say, 200 points, what are your chances to beat him? Basing my research on the actual data of the past 15 days, I was astonished to see how well the correlation worked! For example, if you have between 200 and 300 more points than your opponent, you have 75% chance to beat him. And if you have 400 to 499 more points, you have 86% to do it!

It is the statistical proof of two things:

  • Ticket to Ride is not a random game - but all good T2R players already knew this Smile
  • The ELO system that we are using does work - the curve is nicely progressive and continuous, slowing down as it moves up Smile


Yann

[Updated on: Wed, 27 May 2009 19:05]

      
SOJA maunaloa
Member

User Pages
Posts: 69
Registered:
March 2009
Re:ELO score distribution Sat, 30 May 2009 00:46
Hi Truckerteller,

Truckerteller wrote on Sun, 24 May 2009 23:07


My theory for the left skewness would be that there are clear distinctions in playing level between 1300, 1400, 1500, 1600, 1700 and 1800 (and more 2P games are played the higher up you go) yet between 1000 and 1300 distinctions are harder to make as people leave mid-game, draw 6 times or not and play is far more casual than competitive. To have a more normal distribution and equal tails on both sides would require a whole bunch of people and levels of self-destructiveness on the downside of 1000. You would need 100+ players @ <800 who make a sport out of it to finish below their <800 competitors.


I agree with you that the higher tail is due to player's skill.
On the other hand, how about the main Gaussian component centered at 1201? I thought the best way to check this is to obtain the distribution wighout any skill factor involved. I ran a simulation in the following condition:


  1. 10000 players in this simulation
  2. every player has exactly the same skill regardless of his/her current ELO point (i.e. 50%win, 50%loss; neglected tie for simplicity)
  3. all players have ELO score of 1201 at the begining
  4. players fight against each other and on average 20000 games are played for each player
  5. ELO points gain/loss are determined by the ELO formula


The resultant ELO score distribution for simulated players are plotted as blue histogram in the following figures (left: liner, right: log). It is superimposed in the previous figure to see the difference between them.

http://www.daysofwonder.com/uploads/userpages/528043/_2296_web.jpg

The distribution shows a clean Gaussian with a sigma = 26 points. This is about 1/3 of the main component in the real distribution. It means that there is a clear skill differences in the main component, too.

Also, the fact that the simulated distribution has a sigma of 26 points means it is probable that you get 26 points higher (or lower) than your actual skill level although it is not usual to get 78 points above your actual skill level (3-sigma effect; 3/1000 probability).

kolmo,

I understand your point. Actually, I only play USA 2-player game recently although I played other variants and multi player game when I start playing. That's the problem of overall ELO, I think. But currently I enjoy it very much.

Wildfire,

I agree with you. I spent too much time to make those graphs. But I'm curious about the system and it's fun, too. Also, I am quite impressed by World TTR league in which you guys aims to build a new ranking system.

And Yann,

Yann wrote on Wed, 27 May 2009 07:04

Dear players:

The charts are cool, but one question: from which data were they built from? Extracted from the ranking section of the HTML page in your account?



Of cource I took the data from ranking section of the HTML page in my account. I want to obtain the whole data, but it is simply not possible. Like Wildfire, I'm curious about the difference between active and inactive accounts.
      
Wildfire2099
Senior Member

Posts: 654
Registered:
June 2005
Re:ELO score distribution Sun, 31 May 2009 07:30
Yeah, when I've done stats I've taken them off the ranking page as well... it's not great, but it's what we have Smile

It's kinda neat to see statistical proof that there is skill involved in the game.

I can't believe the average is down to 1201.. it was over 1300 at one point... prolly not having guests anymore (at least the permanent kind we used to have) has a big impact.
      
Zeno
Senior Member
Cadet

User Pages
Posts: 582
Registered:
February 2006
Re:ELO score distribution Mon, 01 June 2009 01:23
Some interesting stuff here.

First, with respect to the peak, and Wild's question about the average, we have to be careful with mean, median and mode. In order to compute downward drift, i used the median score, as wildfire had. Maunaloa is giving us the peak of the curve, which is the mode. The median goes in the direction of skewing, so if the mode is at 1201, I would expect the median should be significantly above that. If wildfire is correct that the missing 1400's were the guests, and that is a tempting thought, then the downward drift should have appeard to have slowed. IIRC, it was about 1226 last I saw. Of course, I still expect downward drift, for the same reasons I gave in another thread somewhere or other.

Wrt the simulation, that is interesting indeed. When I ran the genius list I used the standard deviations to make inferences about skill. As expected, 2-player had a higher standard deviation than multi (119.43 to 89.37), and US was over EU (112.18 to 96.13). Some of that variation is background variation, and it is nice to know the amount. The claim that we stick inside a band 26 above or below of our score about 2/3rds of the time is correct to the extent that we give the same effort each time. I know that I play harder to protect a good rating, however, and sometimes stubbornly play on when I am sick/fatigued/inebriated despite my rapidly falling score. Additionally, it is indexed to games as opposed to time. Many people play less when they are near their best score, in order to stretch out their time at the top. As a result, I always found the band to be larger. On the top 200 list I used to do, i routinely found people rising or falling by over 100 points in a week.

wrt kolmo, of course he has a point, and he is correct, but it is usually best to ignore him and move on. Otherwise the work gets bogged down by a long list of qualifications and hedges.

wrt Yann, it is nice to hear that they are still checking for consistency. I imagine that this is an interesting period, as old players return. I presume that many people got an e-mail offering to convert an account that they had stopped playing long ago. We see the ex-guests joining in again, and the community is much less contentious (on a separate note, I hope the cruise is a long one). At times, i would love to have the power to look at all of the stats that are available to Yann. But on balance, I am happy to muck around in the lists or erps stats tool.

Z
      
kolmo
Senior Member

User Pages
Posts: 556
Registered:
November 2006
Re:ELO score distribution Mon, 01 June 2009 01:31
Zeno écrit le Sun, 31 May 2009 19:23

wrt kolmo, of course he has a point, and he is correct, but it is usually best to ignore him and move on.


I already feel less ignored.
      
SOJA maunaloa
Member

User Pages
Posts: 69
Registered:
March 2009
Re:ELO score distribution Sat, 06 February 2010 15:53
Hi Yann and all,

Yann wrote on Wed, 27 May 2009 07:04

One quick study I made last week was to measure the correlation between the ELO
score and the winning success of players. In other words, if you are better
ranked than the other player by, say, 200 points, what are your chances to beat
him? Basing my research on the actual data of the past 15 days, I was astonished
to see how well the correlation worked! For example, if you have between 200 and
300 more points than your opponent, you have 75% chance to beat him. And if you
have 400 to 499 more points, you have 86% to do it!



I have checked this with my own record. Before plotting, I am in doubt about
what he stated (sorry), but the result is as follows:

http://www.daysofwonder.com/uploads/userpages/528043/_3156_web.jpg

I must say that my record is in a very good agreement with the ELO curve.
It might be due to fast feedback of the ELO system, and/or I only play against the
players within ELO difference of about 200.

Here's an intensive discussion about ELO in the past (hard to follow, but very interesting):
http://www.daysofwonder.com/en/msg/?goto=64065
      
    
Previous Topic:Champions League: an open PM to blubes
Next Topic:next cheater
Goto Forum: