Clan Ladder Version 2 - For Public Comments

[jpcloet]

viewtopic.php?f=438&t=90907

[jpcloet]

Forgot to mention this is more similar to college sports RPI.

[waseemalim]

With some tweaking to the reliability factor:

Rank Clan Name Grade
1 Legends of War 100.000
2 The Spanking Monkeys 99.726
3 THOTA 99.490
4 The Untouchables 97.736
5 EMPIRE 96.432
6 Imperial Dragoons 93.615
7 Immortal Assassins 93.450
8 ++The Legion++ 92.459
9 Sky Force 92.023
10 De Veroveraars der Lage Landen 91.893
11 Black Sheep Squadron 90.864
12 Legion of The Damned 90.707
13 Imperial Britain 89.612
14 Generation One: The Clan 89.333
15 The Regulators 88.646
16 Mythology 88.517
17 Warlords of the Wort 87.670
18 Marvel/DC Heroes 80.078
19 Eternal Empire 75.972
20 Bullet Proof Bandits 74.342
21 The Last Warriors 73.934
22 Knights of the Empire 72.012
23 Order of Odin 71.900
24 Freemium Forces 70.183
25 Nemesis 69.487
26 Water's Fury 66.636
27 The Bushwhakers 65.567
28 Left4Dead 65.445
29 Soldiers of War 61.778
30 Project Enigma 61.190
31 Bounty Hunters 60.055
32 Dark Defenders 59.738
33 Agents Of Chaos 57.320
34 Death By Comity 54.066

[waseemalim]

And some scenario analysis at the top:

Thota beats LoW 31-29
1 The Spanking Monkeys 100.000
2 THOTA 99.712
3 Legends of War 99.419
4 The Untouchables 98.055
5 EMPIRE 96.287
6 Imperial Dragoons 93.814
7 Immortal Assassins 93.248
8 ++The Legion++ 92.295


Thota beats LoW 33-27
1 THOTA 100.000
2 The Spanking Monkeys 99.410
3 Legends of War 98.481
4 The Untouchables 97.669
5 EMPIRE 96.012
6 Imperial Dragoons 93.628
7 Immortal Assassins 92.952

[waseemalim]

And , the table w/o any "reliability factor":

Rank Clan Name Grade
1 Legends of War 100.00
2 The Spanking Monkeys 99.73
3 THOTA 99.49
4 The Untouchables 97.74
5 EMPIRE 96.43
6 Dark Defenders 95.77
7 Imperial Dragoons 93.61
8 Immortal Assassins 93.45
9 Bullet Proof Bandits 92.96
10 ++The Legion++ 92.46
11 Sky Force 92.02
12 De Veroveraars der Lage Landen 91.89
13 Nemesis 91.22
14 Black Sheep Squadron 90.86
15 Legion of The Damned 90.71
16 Left4Dead 90.11
17 Order of Odin 90.04
18 Eternal Empire 89.95
19 Imperial Britain 89.61
20 Generation One: The Clan 89.33
21 Knights of the Empire 89.11
22 The Regulators 88.65
23 Mythology 88.52
24 Warlords of the Wort 87.67
25 Death By Comity 86.67
26 Agents Of Chaos 84.99
27 Marvel/DC Heroes 83.38
28 Soldiers of War 81.47
29 The Last Warriors 80.79
30 The Bushwhakers 80.62
31 Bounty Hunters 80.26
32 Project Enigma 78.33
33 Freemium Forces 77.78

[waseemalim]

In case you are wondering why THOTA is not number 1. It is because of the quality of opponents.

Thota's opponents have a win % of 44. Whereas, LoW's opponents have a win % of 50.

As Jp mentioned, this would be a pretty good ladder once we put in the clan league results.

[waseemalim]

The important thing here is not only your results matter, but also your opponents results matter.

A clan would see its points change even without playing in any wars -- because their scores are retroactively affected by their opponents current wins.

And that an undefeated result means nothing -- its all about the overall win %.

[jpcloet]

That's why OOWP is also important, of course THOTA opponents are going to have a low win % as they keep winning.

[Scott-Land]

The important thing here is not only your results matter, but also your opponents results matter.

A clan would see its points change even without playing in any wars -- because their scores are retroactively affected by their opponents current wins.

And that an undefeated result means nothing -- its all about the overall win %.

I agree with most of what you say Wassee but the one factor in the old ladder that drove me nuts is the overall clan record- it was non-existent. I think the ladder needs to incorporate that into the system. Granted win percentage is a good indicator but a loss should be formulated as well and it be more devastating.

[jpcloet]

Overall record could be added to some degree, another good suggestion. It is more difficult to win challenges than games in some instances.

I hope someone sigs Scott-Land for admitting he's gone nuts.

[Incandenza]

That's why OOWP is also important, of course THOTA opponents are going to have a low win % as they keep winning.

But OOWP can also be misleading. Back then THOTA took on the bushwackers, the wackers were one of the best clans on the site. Then THOTA won and the wackers never played another challenge to my knowledge. But now it's basically counted like a win against some scrub clan, and probably hurt us in the standings. In fact, I wager that if that challenge were removed, our rating would probably increase in certain formulae.

And then you have a situation like the Untouchables: strong clan, strong players (many of whom escaped from the thota-induced flaming wreckage of koe), but by the time we could have challenged them, the clan was basically dead. Call me crazy, but I seem to recall them basically deadbeating out of the last phases of their challenge with LoW.

So it seems like a substantial part of the ladder equation comes down to timing and perception, where you can beat the 2003 Detroit Tigers and get credit like you beat the 1968 Tigers, or beat the 1927 Yankees yet get credit for beating the 1992 Yankees. (quick guide for the baseball neophyte: 1927 Yankees and 1968 Tigers all-time great, 1992 and 2003 iterations very very bad)

Let me put it a different way: this whole endeavor has way too many variables to be considered anything more than a very informal guide, rather than its current quasi-official placeholder status. Plus, given the current tools available to the average user, clan challenges take so much goddamn time to set up and organize that expecting a clan to play more than, say 4-5 60-game challenges per year is insane (and even that is a quick pace). So you're invariably dealing with pretty small sample sizes, thus there's so little depth to the numbers that you can tweak them to say whatever you want.

And before anyone says it, of course I'm biased. I want to see my clan at #1, like anyone in any of the top clans. But this is starting to feel like stumbling around in the dark with a broken flashlight.

[jpcloet]

You should add KOE to that as well. That clan dismantled to eventually create TUC and LOD which then led to some of the best clans (IA, LOW). I hear what you are saying and one of the suggestions could be to limit the history. Eg. Only keep a rolling 2 years worth of data. I'm inclined for these reasons to put all of the league data in. More data and just as valid. Some of those were late 2007 which would soon drop off.

[waseemalim]

There is only 1 recorded challenge for Bushwackers. Which is vs. Thota.

So the OOW% credit element is overrated, rather than underrated, in Thotas point calculation. Infact, I just tried removing BW from the list -- and lo and behold, Thota drops.

I agree that this algo does not take into account qualitative factors such as clans breaking up. The arguments for a time limit is definitely valid. But that brings up the question of where do we draw the line. I frankly feel that a 2 year horizon (which is what it currently is) is pretty safe. I understand that the burn-out rate for clans is pretty high, but I dont think we have to worry about it yet.

The sample size is an issue, but it will get better as we add the clan league stuff. That's actually the main reason that I feel a game win% offers a much better picture.

[waseemalim]

The important thing here is not only your results matter, but also your opponents results matter.

A clan would see its points change even without playing in any wars -- because their scores are retroactively affected by their opponents current wins.

And that an undefeated result means nothing -- its all about the overall win %.

I agree with most of what you say Wassee but the one factor in the old ladder that drove me nuts is the overall clan record- it was non-existent. I think the ladder needs to incorporate that into the system. Granted win percentage is a good indicator but a loss should be formulated as well and it be more devastating.

It would be great if we could do it. But the stumbling block here is that clan challenges are not standardized. If we did add that element it would have to come with limits so that a middle tier clan cannot climb up the ladder by beating plenty of new clans.

[Incandenza]

So the OOW% credit element is overrated, rather than underrated, in Thotas point calculation. Infact, I just tried removing BW from the list -- and lo and behold, Thota drops.

That can swing both ways, tho, because if we're getting credit for beating the wackers, then you guys are getting as much or more credit for beating project enigma. Since there's no way to objectively rate a given clan absent challenges (save for maybe taking an average score of its members), then we're talking about a powerful disincentive for any top clan to face, say, the knights of the round table, who are still very new but would give anyone a tough fight, while creating a powerful incentive to beat up on the lowest-ranked clans and especially new clans comprised of low-ranked players. After all, it was the fact that you guys jumped 2 spots by creaming a brand new clan (that, with all due respect to its members, will probably never make it out of the bottom third of the ladder) that motivated this effort to tweak the ladder calculation in the first place.

Plus it seems that we're not making allowances for what a clan's strength was at the time of the challenge. The analogy here is that if I had happened to beat king herpes in a 1v1 on his first day on the site, my point gain shouldn't have accrued upward as he rose to conqueror. Similarly, if he beat me as conqueror, then I shouldn't lose more points should he decide to medal-hunt and derank.

Maybe we do need some sort of objective baseline score for a new clan, where the knights of the round table and project enigma aren't basically considered the same thing.

[AndrewB]

One comment on the calculations side:

you need to round up all the numbers to the same significant number; lets use 5 significant numbers through-out. (Currently WP/OWP/OOWP and reliability has just 2 significant numbers, RPI has 4).

And yes, that change in rounding will drop the THOTA in the rating, but it is more correct one. And If you would use 2 throughout then we would be tied for the first place

[AndrewB]

And , the table w/o any "reliability factor":

Rank Clan Name Grade
1 Legends of War 100.00
2 The Spanking Monkeys 99.73
3 THOTA 99.49
4 The Untouchables 97.74
5 EMPIRE 96.43
6 Dark Defenders 95.77
7 Imperial Dragoons 93.61
8 Immortal Assassins 93.45
9 Bullet Proof Bandits 92.96
10 ++The Legion++ 92.46
11 Sky Force 92.02
12 De Veroveraars der Lage Landen 91.89
13 Nemesis 91.22
14 Black Sheep Squadron 90.86
15 Legion of The Damned 90.71
16 Left4Dead 90.11
17 Order of Odin 90.04
18 Eternal Empire 89.95
19 Imperial Britain 89.61
20 Generation One: The Clan 89.33
21 Knights of the Empire 89.11
22 The Regulators 88.65
23 Mythology 88.52
24 Warlords of the Wort 87.67
25 Death By Comity 86.67
26 Agents Of Chaos 84.99
27 Marvel/DC Heroes 83.38
28 Soldiers of War 81.47
29 The Last Warriors 80.79
30 The Bushwhakers 80.62
31 Bounty Hunters 80.26
32 Project Enigma 78.33
33 Freemium Forces 77.78

And comment on the TSM ranking. I certainly appreciate that Monkeys is one of the top three clan around for sure. But they need to get more challenges on the way. They have only 2 completed. It is very difficult to come up with the proper rating based on just 2 challenges. As result the current reliability factor is low.

I actually was very surprised to find out that they have only 2 at all...

[waseemalim]

So the OOW% credit element is overrated, rather than underrated, in Thotas point calculation. Infact, I just tried removing BW from the list -- and lo and behold, Thota drops.

That can swing both ways, tho, because if we're getting credit for beating the wackers, then you guys are getting as much or more credit for beating project enigma. Since there's no way to objectively rate a given clan absent challenges (save for maybe taking an average score of its members), then we're talking about a powerful disincentive for any top clan to face, say, the knights of the round table, who are still very new but would give anyone a tough fight, while creating a powerful incentive to beat up on the lowest-ranked clans and especially new clans comprised of low-ranked players. After all, it was the fact that you guys jumped 2 spots by creaming a brand new clan (that, with all due respect to its members, will probably never make it out of the bottom third of the ladder) that motivated this effort to tweak the ladder calculation in the first place.

Plus it seems that we're not making allowances for what a clan's strength was at the time of the challenge. The analogy here is that if I had happened to beat king herpes in a 1v1 on his first day on the site, my point gain shouldn't have accrued upward as he rose to conqueror. Similarly, if he beat me as conqueror, then I shouldn't lose more points should he decide to medal-hunt and derank.

Maybe we do need some sort of objective baseline score for a new clan, where the knights of the round table and project enigma aren't basically considered the same thing.

you are spot on there. The new system is actually better than ELO in terms of that. But the new system creates a liability for us in terms of Project Enigma -- if they continue to perform worse than expected, our ratings would drop, if they do better then ours would increase. On the other hand beating Knights of the Round table would be quite the opposite -- as they improve their standings in the league so does the clan who had beaten them in the 1st place.

Plus it seems that we're not making allowances for what a clan's strength was at the time of the challenge. The analogy here is that if I had happened to beat king herpes in a 1v1 on his first day on the site, my point gain shouldn't have accrued upward as he rose to conqueror. Similarly, if he beat me as conqueror, then I shouldn't lose more points should he decide to medal-hunt and derank.

Agreed with you on that. But the alternative is to put new entrants at a standard score. You can suggest that we do it by ranks, but there are countless examples of clans which are better/worse than the sum of their parts. Infact, I am willing to bet that if you did it with PE, we probably would have scored more points from winning the challenge.

Personally, I think clan rank is a much less volatile thing than personal rank. In absense of any ladders, how does a clan prove that it has become better? By defeating other clans who had defeated them in the past or defeating clans who have defeated their nemeses. The system suggested here works exactly the same way. If you defeated herpes and then herpes defeated countless other players better than you -- you wouldnt enjoy the benefits of having defeated him. That sort of takes care of it on the upside. On the downside, apart from the fact that clans dont go medal hunting, it works the same way. Obviously we need some scenario analysis here.

Btw, I ran some tests on Project Enigma's impact. Here are the results:


No PE:
Rank Clan Name Grade
1 THOTA 100
2 The Spanking Monkeys 99.24415296
3 Legends of War 98.82948627
4 The Untouchables 97.84837365
5 EMPIRE 97.06300722
6 Imperial Dragoons 94.22685363
7 Immortal Assassins 93.82852181
8 ++The Legion++ 92.83774324


If we went 23-7 against PE
Rank Clan Name Grade
1 THOTA 100
2 The Spanking Monkeys 99.94872966
3 Legends of War 99.92650567
4 The Untouchables 98.15773532
5 EMPIRE 96.96316208
6 Imperial Dragoons 94.12992595
7 Immortal Assassins 93.91278828
8 ++The Legion++ 92.91990035
9 Sky Force 92.44963664



As you can see, what we required to get to number 1 was the margin against PE. Had we won 2 fewer games, there would have been no change in our ranking -- and it falls if we lose more games. Honestly, while playing PE, I dont think we had that in mind.

Actually, the more scenarios I run on this, the more I start to like this. The No Project Enigma scenario is probably a very good representation of the reality back then.

[waseemalim]

One comment on the calculations side:

you need to round up all the numbers to the same significant number; lets use 5 significant numbers through-out. (Currently WP/OWP/OOWP and reliability has just 2 significant numbers, RPI has 4).

And yes, that change in rounding will drop the THOTA in the rating, but it is more correct one. And If you would use 2 throughout then we would be tied for the first place

Actually all of these are on an microsoft excel sheet so the decimal places, in theory, are infinite (ofcourse theres the excel limit).

SO the way the reliability factor works under the algorithm I suggested is that clans need to get a certain number of games against established clans. Playing not-established clans help a bit, but not significantly.

However, the impact of reliability factor is not infinite -- i.e. you cant get more and more points by playing more challenges. It plateaus off at 100%. Once the clans reach that level there is no impact of the reliability factor. The idea is to tone down results of clans who dont have enough experience -- such as BpB who are penalized 20% for not playing enough games.

Obviously it could use some tweaking here and there (right now the plateau is ~100 games and OOW total of ~1300 -- its a function of stuff but those numbers are fairly good estimates). The monkis have 100% reliability and that I can totally see an argument for toning it down -- although personally, I feel that its alright.

Btw, the formula right now is

(W%+OW%+OOW%)*Reliability

[e_i_pi]

Um wow. Third order equations. This is about as good as you'll get. I like it, I think we should run with it, it seems really good, much better than ELO.

[Bruceswar]

This ladder will always be a joke. We need a human Element added to it. As much as I do not like the BCS system, we need to put into place something like it. A computer cannot tell you everything. As Incadenza said, at times certain clans were considered top tier clans, now which are treated as scrub clans. TUC, BW, etc. It also should not matter how other clans fair vs other clans as to your ranking. Only match that should matter is your match vs them. This ladder means nothing in most people's eyes anyhow. We all know who the best clans on CC are regardless of what ladder ranking they are. The CLA doubles league is a pretty good measuring stick. All the top clans are where they should be, and the rest well they are getting blown out. A clan such as Empire who picks some really nasty settings(In my mind) for clan challenges is getting trashed in the doubles league. Granted they may be 5-0 in challenges but that is skewed if you ask me. This all boils down to we need some human element to this ladder. Surely there are 10-20 people who know the clans well enough to give an opinion on them.

[jpcloet]

Our BCS poll experiment proved otherwise.....

[Bruceswar]

Our BCS poll experiment proved otherwise.....

Nobody took it serious. If it was used for something people would have. It is like a fire drill. Most people just sit at there desk working while it is going on. If it is the real thing they run out ASAP.

[Scott-Land]

And comment on the TSM ranking. I certainly appreciate that Monkeys is one of the top three clan around for sure. But they need to get more challenges on the way. They have only 2 completed. It is very difficult to come up with the proper rating based on just 2 challenges. As result the current reliability factor is low.

I actually was very surprised to find out that they have only 2 at all...

I agree that 2 challenges isn't a good indicator of a clan's strength. Within those 2 challenges we chose real opponents. At the time of the first challenge with LoW, I don't think the ladder was in place but IA was 7th I believe. Challenges take great effort to plan and play-- and we don't take it lightly. Shall we beat up 3 noob clans and jump straight to the top and be 5-0? Where's the fun in that?!

My concern was never about where TSM should be but where certain clans shouldn't be.

[Bruceswar]

I agree that 2 challenges isn't a good indicator of a clan's strength. Within those 2 challenges we chose real opponents. At the time of the first challenge with LoW, I don't think the ladder was in place but IA was 7th I believe. Challenges take great effort to plan and play-- and we don't take it lightly. Shall we beat up 3 noob clans and jump straight to the top and be 5-0? Where's the fun in that?!

My concern was never about where TSM should be but where certain clans shouldn't be.

This brings us to why clan matches are played. Should it not be to prove who is the better clan? Well at least for this match. I guess some clans would rather play 10 crappy clans over 5 quality wins. Farming clans for points sucks. I guess the same could be said about the CC ladder.