Verfasser  Thema 
ntaatgen
Nachrichten: 52
Registriert: February 2005

Are adventures balanced? Ask statistics!

Sat, 30 December 2006 06:31


Updated 1/1/2007
I posted the following balance analysis on the boardgamegeek, but thought it might be useful to post here as well:
Using the reported wins and loses on the Days of Wonder site, it is possible to calculate whether scenarios are likely to be balanced. What I did was use the binominal function to calculate the 95% confidence interval. That is, for each adventure I will give a range of percentages, and it there is a 95% probability that the true win percentage is in that range.
For example:
1. Agincourt: 85 wins out of 135 for the standard banners means that the 95% confidence interval is 55%71%. This means the true probability of winning for the French is somewhere in between 55%71%, with 5% probability that is is outside this range. This means that Agincourt is probably not fair, because 50% is not within the confidence interval.
Here are the others. The confidence interval is always the probability that the standard bearers win, and the numbers are based on the information on the DOW site on 1 January. I will make an update when the numbers increase at some point to improve the reliability (assuming people are interested).
2. First Chevauchee 39 out of 72, 43%65%
3. Burgos 25 out of 37, 51%80%
4. Deeper in Castille 16 out of 21, 55%89%
5. Wizards and Lore 14 out of 33, 27%59%
6. A complex web 22 out of 32, 51%82%
7. Crisis in Avignon 6 out 13, 23%71%
8. A Burgundian Chevauchee 4 out of 16, 10%49%
9. Free companies on war footing 9 out of 19, 27%68%
10. Assaulting the Tourelles 9 out of 14, 39%84%
As you can see, there are quite some adventures that are not balanced given the current reports. It is of course unlikely that the probability of winning is exact 50%, so hopefully the intervals can inform you how reasonable the odds are (and they will become more precise with more data).
[Aktualisiert am: Tue, 02 January 2007 02:15]



DarkPadawan
Nachrichten: 599
Registriert: November 2006

Re:Are adventures balanced? Ask statistics!

Sat, 30 December 2006 12:38


Wow, that's a very mathematical approach to the topic.
But when I think about it, there are at least three things that let me take your figures very easy (and this is not meant an offense):
 When you play each scenario twice (swapping sides) and calculate the total result, the odds are averaged.
 The cards and dice rolls should bring enough "variability" to give both sides a chance in every battle (with the exception of extremely unbalanced scenarios of 7030 odds and worse)
 Some consider it a challenge to try to win, even if the odds are saying something else.
Nevertheless, a well done statistic.
Dark.



casbarian
Nachrichten: 10
Registriert: December 2006

Re:Are adventures balanced? Ask statistics!

Sat, 30 December 2006 16:49


I like the idea that M'44 had in that the scenarios were lopsided based upon history. The way you countered this idea was that you played both sides and then totalled your points to see who truely won. I think the same idea can and should be applied to some of the scenarios of BattleLore and I wouldn't mind seeing more scenarios created with that idea.
Goblins vs. Dwarves...no human mercenaries. A slaughterhouse, but can you get slaughtered less than your opponent when the table is turned? I like it.



Gilberreke
Nachrichten: 34
Registriert: December 2006

Re:Are adventures balanced? Ask statistics!

Sat, 30 December 2006 18:10


Has anyone read the battle reports? Half of them states winning with one side, yet the result is stated the other way round. Really dumb. As such, the statistics are totally flawed anyway...



ColtsFan76
Nachrichten: 3326
Registriert: February 2006

Re:Are adventures balanced? Ask statistics!

Sat, 30 December 2006 19:57


When the report system first came out  their was a bug. The lanels where you placed your banner count were backwards. DOW was aware of the problem ans supposed ot fix it. I double check my log and it is correct.



SHwoKing
Nachrichten: 402
Registriert: August 2006

Re:Are adventures balanced? Ask statistics!

Sat, 30 December 2006 23:28


There's still people doing mistake on which side they are playing.
Check Agincourt report for this. I was surprised thta this scenario wasn't that much on the french side. But then i saw there was a lot of mistake on the other side. This scenario must be something like 7580 % for the french IMO while playing with followon actions.



ntaatgen
Nachrichten: 52
Registriert: February 2005

Re:Are adventures balanced? Ask statistics!

Sat, 30 December 2006 23:48


I'll run the analysis again once there are more reports, so the effect of potential erroneous reports will diminish.



Gilberreke
Nachrichten: 34
Registriert: December 2006

Re:Are adventures balanced? Ask statistics!

Sun, 31 December 2006 04:18


The effect won't diminish, it'll keep on going this way. Read the reports. Every report where the English won is actually a French victory if you read the comments...



SHwoKing
Nachrichten: 402
Registriert: August 2006

Re:Are adventures balanced? Ask statistics!

Sun, 31 December 2006 10:50


I have calculated winning percentage for Aguincourt taking only into account reports with comments teeling me which side won.
Results is : 65% for French and 35 % for english. Pretty self explanatory. But most of English wins are really close.



tastic
Nachrichten: 52
Registriert: July 2006

Re:Are adventures balanced? Ask statistics!

Tue, 02 January 2007 01:00


That is why you play both sides.. And I am reading rumors of possible misprinted scenarios? If this is true, someone please PM me to let me know. Then I can decide whether or not to go on a tirade about production accuracy. Sure I'm going to spend more time with homegrown scenarios, but still..



ntaatgen
Nachrichten: 52
Registriert: February 2005

Re:Are adventures balanced? Ask statistics!

Tue, 02 January 2007 01:14


tastic wrote on Mon, 01 January 2007 19:00  That is why you play both sides.. And I am reading rumors of possible misprinted scenarios? If this is true, someone please PM me to let me know. Then I can decide whether or not to go on a tirade about production accuracy. Sure I'm going to spend more time with homegrown scenarios, but still..

But now you know when you don't have to play both sides! There is one misprint in adventure 9, where there are two goblin archers units. The archer unit not in the woods should be an infantry unit instead. Otherwise I haven heard of no misprints. It also appears (apart from Agincourt) that balance issues can mainly be attributed to goblins. There is a rumour that the gobs will get some enhancement in an expansion though



tastic
Nachrichten: 52
Registriert: July 2006

Re:Are adventures balanced? Ask statistics!

Tue, 02 January 2007 01:20


quote
But now you know when you don't have to play both sides! There is one misprint in adventure 9, where there are two goblin archers units. The archer unit not in the woods should be an infantry unit instead. Otherwise I haven heard of no misprints. It also appears (apart from Agincourt) that balance issues can mainly be attributed to goblins. There is a rumour that the gobs will get some enhancement in an expansion though [/quote]
ahh "enhancement". I thought those pills only work if something is already there



ColtsFan76
Nachrichten: 3326
Registriert: February 2006

Re:Are adventures balanced? Ask statistics!

Tue, 02 January 2007 01:21


tastic wrote on Mon, 01 January 2007 18:00  That is why you play both sides.. And I am reading rumors of possible misprinted scenarios? If this is true, someone please PM me to let me know. Then I can decide whether or not to go on a tirade about production accuracy. Sure I'm going to spend more time with homegrown scenarios, but still..

It is not misprinted scenarios that we are talking about. The main web page has a place for you to keep a record of your battles. You enter in when you played the game, your opponent, and the side you were playing (from a drop down). You then enter in the number of banners each side captured. Either their filters wer wrong or they mislabelled where you enter in the banners. If I entered in the data to show that I was the standard bearer and won 64; it was showing the pennant actually won the game.
We caught this early enough and reported it to DOW. The fixed it realatively quickly before the vast amount of data was entered. I for one held off on entereing my data (i keep a seperate spreadsheet on my C&C games) until the problem was fixed.
So we either have people that entered in the data before the fix and DOW did not change it, or people are just confused as to which side they play.



GhostWolf69
Nachrichten: 148
Registriert: April 2004

Re:Are adventures balanced? Ask statistics!

Tue, 02 January 2007 07:43


ntaatgen wrote on Sat, 30 December 2006 06:31  Updated 1/1/2007
I posted the following balance analysis on the boardgamegeek, but thought it might be useful to post here as well:
Using the reported wins and loses on the Days of Wonder site, it is possible to calculate whether scenarios are likely to be balanced. What I did was use the binominal function to calculate the 95% confidence interval. That is, for each adventure I will give a range of percentages, and it there is a 95% probability that the true win percentage is in that range.
For example:
1. Agincourt: 85 wins out of 135 for the standard banners means that the 95% confidence interval is 55%71%. This means the true probability of winning for the French is somewhere in between 55%71%, with 5% probability that is is outside this range. This means that Agincourt is probably not fair, because 50% is not within the confidence interval.
Here are the others. The confidence interval is always the probability that the standard bearers win, and the numbers are based on the information on the DOW site on 1 January. I will make an update when the numbers increase at some point to improve the reliability (assuming people are interested).
2. First Chevauchee 39 out of 72, 43%65%
3. Burgos 25 out of 37, 51%80%
4. Deeper in Castille 16 out of 21, 55%89%
5. Wizards and Lore 14 out of 33, 27%59%
6. A complex web 22 out of 32, 51%82%
7. Crisis in Avignon 6 out 13, 23%71%
8. A Burgundian Chevauchee 4 out of 16, 10%49%
9. Free companies on war footing 9 out of 19, 27%68%
10. Assaulting the Tourelles 9 out of 14, 39%84%
As you can see, there are quite some adventures that are not balanced given the current reports. It is of course unlikely that the probability of winning is exact 50%, so hopefully the intervals can inform you how reasonable the odds are (and they will become more precise with more data).

I like your aproach here, it will be even more interesting to see how it unfolds when we have a couple of hundred games to base the stats on. Right now we have quite a spread... 23%71% for example... doesn't really tell us much except that we need to play more!
/wolf


