ELO implementation, half baked?

Chat about anything and everything Frozen Synapse

Moderator: Admins

Eckie
Posts: 15
Joined: Fri Jun 24, 2011 5:29 pm

Re: ELO implementation, half baked?

Post by Eckie » Fri Sep 09, 2011 4:14 pm

Before deciding on a score-reset, please make sure the new scoring implementation is robust and tested thoroughly. That's all I'm asking for really.

I'd be sad to see my score go since I have been working on it for months and am finally seeing some results getting higher on the list. Also it would be harder to judge your opponent when their scores are gone: you could see whether they had played hundreds of games or just a few. So a score reset would be a big deal to me.

But, there have been complaints about the score system so I would be understanding towards a score reset if this would mean a big step towards a better and robust scoring system. (That wouldn't need to be changed again 6 months later!)

@TheRaven42 (and others): Maybe this would require a better match-making algorithm as well. I'm not looking for unranked games, I am looking for good matches. So maybe an option to constrain matched players to within a certain percentage of your own score?
Hdfisise
Posts: 45
Joined: Mon May 09, 2011 6:09 am

Re: ELO implementation, half baked?

Post by Hdfisise » Fri Sep 09, 2011 4:15 pm

Rakshasa wrote:On another note, what just happened to the leaderboard? I just looked and some players are missing and all the scores changed and I'm #13 and some guy named Jefis is #1 now? It can't just be the Elo implementation, because I checked as soon as that was updated and TheBeefiest was still up there with like 7800 something.
Initial implementation was broken and Omroth fixed it!
Rakshasa
Posts: 24
Joined: Fri Jul 22, 2011 8:48 pm

Re: ELO implementation, half baked?

Post by Rakshasa » Fri Sep 09, 2011 4:19 pm

Hdfisise wrote:
Rakshasa wrote:On another note, what just happened to the leaderboard? I just looked and some players are missing and all the scores changed and I'm #13 and some guy named Jefis is #1 now? It can't just be the Elo implementation, because I checked as soon as that was updated and TheBeefiest was still up there with like 7800 something.
Initial implementation was broken and Omroth fixed it!
Oh, well then, now I really don't want a score reset :P
Image
User avatar
icebrain
Posts: 561
Joined: Sun Feb 27, 2011 1:39 pm
Location: France
Contact:

Re: ELO implementation, half baked?

Post by icebrain » Fri Sep 09, 2011 4:20 pm

It must be pointed out that if TheBeefiest and Bin have exaggerated Elo ratings it is because of the formula, and a reset won't fix the formula, therefore in N month we'll have exaggerated ratings again.

reset = no fix
regular reset = no fix ever

and of course:
fix = fix = no reset

After you read the Official Manual, check Frozen Synapse's Tactics & Strategy Guide, go further with the Advanced Mechanics for Multiplayer and beat all SP missions with the Campaign Walkthrough • Read the latest News

Rakshasa
Posts: 24
Joined: Fri Jul 22, 2011 8:48 pm

Re: ELO implementation, half baked?

Post by Rakshasa » Fri Sep 09, 2011 4:20 pm

icebrain wrote:It must be pointed out that if TheBeefiest and Bin have exaggerated Elo ratings it is because of the formula, and a reset won't fix the formula, therefore in N month we'll have exaggerated ratings again.

reset = no fix
regular reset = no fix ever

and of course:
fix = fix = no reset
Check it IceBrain, Beefiest is down to #87 with the new Elo implementation.
Image
User avatar
Sid
Posts: 60
Joined: Wed May 11, 2011 12:07 pm

Re: ELO implementation, half baked?

Post by Sid » Fri Sep 09, 2011 4:22 pm

Rakshasa wrote:On another note, what just happened to the leaderboard? I just looked and some players are missing and all the scores changed and I'm #13 and some guy named Jefis is #1 now? It can't just be the Elo implementation, because I checked as soon as that was updated and TheBeefiest was still up there with like 7800 something.
There was a lot of talk on the IRC and bugs were found in the Elo implementation. The maths was reworked and what you see now is the correct Elo rankings. The blog post was also updated.

You can use http://slashsrv.com/projects/fs-elo-rank-changes/ to see how your rank changed, it's not 100% accurate but it's in the right ballpark.

Edit: I didn't notice there was a page 2 of this thread where this was already answered.
Image
User avatar
Lu-Tze
Posts: 42
Joined: Mon Apr 19, 2010 1:12 pm

Re: ELO implementation, half baked?

Post by Lu-Tze » Fri Sep 09, 2011 4:29 pm

Eckie wrote:Maybe this would require a better match-making algorithm as well. I'm not looking for unranked games, I am looking for good matches. So maybe an option to constrain matched players to within a certain percentage of your own score?
Seconding this (although giving people a percentage to later is horrifically nasty UI, just make it be Ranking "Narrow" "Wide" or "Open")

Personally i'm against a reset, because right now you really can ascribe an experience level to a player. This adds to the player experience, because if I rock up against someone who is twice my level if I win i'm going to feel like KING OF THE WORLD and if I lose I can write it off because that dude was WAAYYYYY higher than me. It even feeds into the decisions you make, you might ascribe more basic moves and go for riskier strategies against a low-level player.

Removing that metric makes it something that holds no information or value for a long time until it re-establishes itself. And I don't think it's a piece of data that can be underestimated.

EDIT: This is also the first time i've seen what ELO did to my rank... from 23 to 81.
Image
User avatar
Omroth
Site Admin
Posts: 1825
Joined: Sun Nov 07, 2004 7:07 pm

Re: ELO implementation, half baked?

Post by Omroth » Fri Sep 09, 2011 4:41 pm

I'd like to point out that your LEVEL would not be reset.
User avatar
Paul
Posts: 685
Joined: Fri Mar 24, 2006 8:33 pm
Location: Oxford

Re: ELO implementation, half baked?

Post by Paul » Fri Sep 09, 2011 4:46 pm

It was an error which is now fixed - the current rankings are correct with the mysterious Jefris in first place!
Paul Kilduff-Taylor

Mode 7 Games
Toeofdoom
Posts: 25
Joined: Sat May 08, 2010 5:15 am

Re: ELO implementation, half baked?

Post by Toeofdoom » Fri Sep 09, 2011 4:46 pm

The scores I see don't seem to vary especially widely so I don't think a score reset is necessary. Going back the starcraft 2 "seasons" that were brought up, they're mainly to stop inactive players clogging up the various divisions. They keep the ELO-like "MMR" they use behind the scenes anyway because it's a system that shouldn't need to be reset! I don't really mind if it's reset once (I'd prefer it wasn't), but resetting it repeatedly definitely wouldn't make sense.

An activity test for people to be ranked would have a hugely noticeable effect in the long run, so you can't just get to 1600 then sit on your hands for a year and stay in the top 20 (ELO allows that pretty easily). I would consider having an activity requirement actually be random challenge based (for example, 10 random matches in the last 3 months before being removed) so the system can't be gamed* so easily.

Also, with the code posted it's a bit hard to tell, but as long as you win (even by a tiny bit) you never lose points, right? But the more points, the more you gain/lose?

Oh and it seems I jumped from outside the top 100 to #35 :D

*EG: fulfilling activity test by challenging dummy account that auto-loses 10 times
TheRaven42
Posts: 19
Joined: Tue May 31, 2011 11:06 pm

Re: ELO implementation, half baked?

Post by TheRaven42 » Fri Sep 09, 2011 4:47 pm

ok, roller coaster morning. first ELO sent me from 30 to 110 and now I'm at 26!

And I'm still for removing the pre-release games. Beta games when the balance and maps and everything was buggy, just does not make sense.

Honestly if its hard to come to a decision that works for everyone I'll jump on the 'remove scoring' option. The community can always create its own ladder and tournament rankings.

@Rakshasa: Jefis should probably be #1

Hey, I know we should decide rank with a committee just like College Football in the US, cause it make so much sense..... I kid... I kid (probably missed all you UK folks).
User avatar
Omroth
Site Admin
Posts: 1825
Joined: Sun Nov 07, 2004 7:07 pm

Re: ELO implementation, half baked?

Post by Omroth » Fri Sep 09, 2011 4:49 pm

TheRaven42, Obama is in favour of FS playoffs to determine ranking.
Diomyr
Posts: 15
Joined: Tue Jun 07, 2011 2:12 am

Re: ELO implementation, half baked?

Post by Diomyr » Fri Sep 09, 2011 4:51 pm

Jefis has been consistently annihilating the competition at the DGL Tournaments:
http://devgaming.net/tournaments.php?c=12

While I've never seen TheBeefiest or Raeynn (or some others in the top 10 really) play as of recently, and therefore can't say with certainty the #1 spot is rightly attributed, but he certainly at least is a very strong contender for it.
Diomyr
Posts: 15
Joined: Tue Jun 07, 2011 2:12 am

Re: ELO implementation, half baked?

Post by Diomyr » Fri Sep 09, 2011 4:55 pm

TheRaven42 wrote:And I'm still for removing the pre-release games. Beta games when the balance and maps and everything was buggy, just does not make sense.
I could be wrong, but I believe Elo at the moment only takes into account games played on UK1. Source: the red header at http://slashsrv.com/projects/fs-elo-rank-changes/
User avatar
Sid
Posts: 60
Joined: Wed May 11, 2011 12:07 pm

Re: ELO implementation, half baked?

Post by Sid » Fri Sep 09, 2011 5:01 pm

Diomyr wrote:I could be wrong, but I believe Elo at the moment only takes into account games played on UK1. Source: the red header at http://slashsrv.com/projects/fs-elo-rank-changes/
And my source for that is:
IRC wrote: 15:01:55 @Omroth | this is currently only calced based on games on UK1
15:02:14 @Omroth | (your record is still all games though)
Image
Post Reply