Correction: "I don't really have anything to contribute to this, but...." Tss my shit is too deep for you. This ain't mathematics, this is art! Imma give you a hint:

_________________

Add a smiley, don't add a smiley! I added the smiley this time!! I get in trouble either way.

Adding logs distort people's ability to think linearly. The leveling system is setup like that, with each step being a larger step. Every level you get, you breathe a sigh of relief and then exasperation because you need to do 1.5x the amount of work to get to next level.

This is designed NOT to discourage the user like that. Steady improvement shows steady results. Even if you have one (or a few) whacked out scores, the law of averages always wins...

Currently, all the STDDEVs are averaged out (with a preset (read: arbitrary) minimum). What I SHOULD do, is count the total number of tracks each person has, and STDDEV that as well. This way, as more people run more tracks, the minimum moves as well. If you have less than (AVG#Tracks - (1 STDDEV)) tracks, you are not calculated or is weighted towards 0 in some fashion.

Too bad this shitty PHPBB doesn't have a like button or an AwesomeScore(tm). :p Checkout out some sample scoreboards on ozmonet, you'll see how the distribution spreads out.

Trademarked.... You bastard. But I can borrow it right?

I was just as surprised as you that it was available worldwide, but.. haters gonna hate.

I was just as surprised as you that it was available worldwide, but.. haters gonna hate. gondee has worked out a deal with me that he's allowed to use it only on MerqCity.com

My first thought was... just re-balance the weighting to take into account that SSX 2012 has far more tracks than the other games. Don't count "popularity" according to straight number of scores per game, but by number of scores per track per game. That would be a much "fairer" way of determining the popularity of each game.

After reading the other suggestions, which I like as well, I'm not sure whether my first thought is still applicable or workable alongside those methods.

On a side note, would it help anything if I posted more scores to compete with some of Rocklee's? If I'm understanding things right, it won't make any difference under the current methodology, but could help balance things under some of the new proposals. I have some scores which take first for some of the tracks. I know Rocklee will just take any of them he wants right back, but it would close the gap some between his score and second place. I've held off posting them since the recent "spat" that we had kind of soured me on even wanting to deal with any of it. But it looks like serious consideration is being put into the standard deviation methods, which posting some scores to make Rocklee's not quite such huge outliers should help with.

I suppose I'll go ahead and post some of the scores I have. but I'd like to know if it would help for me to post more. I'm willing to put in some effort to close some of the gaps between Rocklee and everyone else, but not so much if it isn't going to help anyway.

No, it wouldn't change anything under the current system. Post them if you want to though.

I'm glad everyone is putting forth proposals - and I want to let people know I'll be accepting input on the current ideas until next week.

_________________

This wasn't brought up on here, but on another thread Tayste brought up a point I remember recently showing the AwesomeScore(TM) on some tracks to be way out of whack. If you look at Serenity Race It there is probably a good 300 times on there, but only two people rank above a SD of 1.0. I'm assuming this has something to do with the increased number of scores, but I just wanted to bring it up in case there is a fix or explanation or whatever.

I'm sorry, I'm not trying to just bring up things and not take the time and think about it myself, but my critical thinking skills do not work like that too well.

_________________
"You came to prepare for the future, I live for right now"
Xbox Gamercards
PSN Gamercards

This is a case of the math WORKING. There's nothing to fix. There is a steady stream of times from 2m:30s to 4m:10s, there's no distribution; it's relatively linear.

If you had 90 friends graded on a test, and 89 people scored each number between 1 and 89, and one person scored 95. The 95 isn't "so great". It's just higher than everyone else. If MOST people scored around a 50, a few scored 75, and 1 person scored 95, THEN the 95 is "more awesome". It's about distribution, the farther away you are from the group, not just the highest.

I just wanted to post to say this is still on my radar. I'm talking with the scoreboard programmer and we'll come to a decision shortly. I may have a few more questions Ozmo, but that'll probably be next week at some point.

_________________

Whatever you guys decide, even if it is to do nothing, I'd be interested in hearing about it. So please, whatever the final outcome is, let us know about it.

Thanks.

Don't worry, I just need to wrap my head around what Ozmo and the others are proposing, talk with my programmer, ally any concerns I have about changing things, and then I'll post - either to ask questions or ask for your opinion on what I'm proposing.

_________________

Well, the scoreboards on Winonanet have incorporated this math. Not to take it away from you, but to allow you for a real-world viewing.

What I've found is that when you take into considering everyone else's score (like the stdev function), each type of run has to be calculated differently. Races have a hard ceiling and a tight range. No matter HOW much you try, you WILL NOT get less than 1:30m on Serenity, the game does not let you go that fast. Therefore, the STDEVs max out at +2.

Tricks have a medium ceiling and a rather wide range. Fro with the Fro ranges from 215m to 3m, the STDEV runs from near +5 to -2.

Survives have the softest ceiling and widest range. Hush has STDEV ranges from +8.5(!) to down under 0.

So, whatever calculation you make, HAS to take into account the difference between awesomeness(tm) of each type. I tried averaging them together and it just muddles the greatness.

No offense taken. If nothing else, this is good testing of your excellent idea.


That's good to know. So I suppose my question after this is, can you think of a reasonable way to average them out to determine the "Best of" categories? We have other methods of evening out tracks that are way out of the standard range, but I believe that since Survival has no determined ceiling, we might have to go a little deeper than the options used before.

_________________

I do not think you should. It is three separate skills. Apples, oranges and watermelon (snow peas?). How do you compare Xedec, Hero Complexity and Rocklee? 2 seconds diff on Serenity is a LOT different than 2 seconds on Kids corner. 10 million difference on Serenity is less than 10 million diff on Kids Corner. So how can you compare 2seconds to 10 million? You can't, and shouldn't.

This message sponsored by Riders for Separate leader boards. I am ticklemeozmo and I approve this message.

_________________
WinonaNET is a RiderNet experiment with a Cross-Platform Scoreboard: http://ssx.ozmonet.com

Sorry about this Ozmo - this dropped off my radar for a few weeks what with real life and then Borderlands 2 took up a lot of my time.


It should. If it can't, then the formula is no good. There is no point to a ranking system that can't equate the most skilled between different categories, and especially between the different games. The system as it stands does this (though it's got it's problems admittedly). If your formula can't do this, then we'll have to find something else, or tweak what we already have to equate things a little differently.

I will talk with our programmer and see what can be done, because the problem still stands and I would love to come up with something elegant to solve it.

_________________

Just counterweigh them by providing a smaller bonus based on the percentage of scores overall? So for instance, if Blur only has 35% of the scores that 2012 has, then multiply scores in that game by 4.
Argument of course is that this favors players of the more obscure games over the guys working so hard to be top of the [massive] heap that is SSX2012 or any of the more popular games.

I agree with ozmo: if you take a #1 trick score and a #1 race time, the measure of skill is entirely subjective. Like movies, we all have our favorites. Someone can say that the formula is wrong because of their opinion. Fact is there's no direct comparison between mode a and b, there's no direct comparison between game a and b - in contrast a maths question always has a right or wrong answer. It would be really nice if you could find something reliable in between, though with the way survival messes with things I think this might be what TrickyEmu suggested with exceptions. I don't know, I'm way out of my depth. Question is how do you rate skill not only across different game modes but across different games too? It's got to be volume and scale of success in their respective mode/game. 2012 is more popular, so there's more competition and with more content naturally SSX 2012 players are expected to be higher up in any SSX overall list. You're not #1 tricks when you're setting #5 on every map and you're not the "best player of SSX overall" if you don't even have SSX 2012! This IS 2012 afterall, SSX 2012 is the game The "best player of SSX overall" achievment is for the one who has more high scores across more modes/games than anyone else. Can this can be automated with a formula?

_________________
When should I open my eyes?!

No need to apologize, I'm only here as a voice, it's your board.

In the Olympics, the best sprinter isn't the highest jumper, and the highest jumper isn't the the fastest marathoner. They are three different skills. I feel it would be a disservice by saying that "there can be only one(tm)".

It doesn't do this, that's why it has it's problems. I can't claim I'm a good driver with hundreds of accidents.

If you are trying to combine three different skills, you have to find some way to marginalize and sanitize two of them. I think the fans would want to know where they are in three different skills, not "one overall".

That's why there are three different stats: Speed, Boost, and Trick. Not one stat.

#AllForThreeAndThreeForAll

_________________
WinonaNET is a RiderNet experiment with a Cross-Platform Scoreboard: http://ssx.ozmonet.com

If we're going with this argument, there really isn't any point to the rankings mod as is. No, we have a system in place now that does this, rather nicely prior to SSX 2012, except one game is drastically larger in scope than the previous ones and thus the formula needs to be changed. SSX 2012 is extremely similar to past games in almost every aspect except scope. It's just much larger than past games. But throwing out the baby with the bathwater - finding out objectively who is the best overall SSX player, up to and including the best overall out of all the games - isn't an option on the table as it can be done and shouldn't require a revamp of the whole feature to do so. Ozmo's formula is nice, I agree, and I would love to use it, but if it can't be changed to equate different games, or even different modes for that matter, then we'll have to use something else as it's simply too limited in scope for the task at hand.


It does, the only issue you're arguing over is how well it does it. But it does do it, as yours doesn't seem to (or you would prefer not to change it to do so based on your feelings on the matter.) Both instances are fine - we'll simply have to find another solution to rectify the instances that 2012 brought to light.

If anyone has an additional input on the formula used for the scoreboard rankings mod, feel free to list their ideas. I may try other sites to see if I can find something that has been used in other places. I found the formula used for races on a mathematics board, so that might be a place to mine for interesting approaches to our dilemma.

_________________

It can do different games, I'm against combining the modes. I feel you are trying to mush something into your current paradigm which is a single "score". You are trying to sell records in a mp3 world. Respectfully, I think it needs a change.

It's a preference. The more you sanitize a score and normalize a difference the less important the number becomes. The more exponents and functions you apply to a score, the less the user understands what it actually means. Right now, with the stddev, a user knows and understands (if they graduated high school), how to move and why they are average, good, or great.

I'm interested in seeing it work, I just don't want the work you are going to do for some minor cryptic update that users aren't going to understand. If it puts Hero, Xedec and Rock fighting for 1st, 2nd and 3rd, a user has to understand WHY a tricker beats a racer beats a survivor. If you are going for a change, make it count, don't try to pidgeon-hole a three score system into the phpbb (one) scoring model.

Use this smiley more , you'll feel better.

I personally feel that mushing them together is small part of what the scoreboard is all about, so I personally support trying to maintain the mode mushing rankings.

I agree with what you're saying here, but I couldn't help but wonder how many high school graduates truly understand standard deviation. I suspect it's not that high.

_________________
[/url]
XBL GT: iTofu | Always looking for rivals.
Ozmo/WinonaNET | MerqCity Scoreboard

It's been done that way for a long time, and this isn't about changing the system. It's about fixing it so that the rankings are reflective of reality. And on the single "score" idea, of course! That's the ultimate goal. And we did it pretty well I think, in a way that was satisfactory to the community.

The two issues here haven't changed: 1) the glut of SSX 2012 scores is throwing off the weighted scores for the "best overall" categories, and 2) the limitless ceiling on the survival scores are doing much the same thing except in a more limited sense. I think if we can tackle those problems separately, we can find solutions here that are satisfactory to the site as a whole, in a way that doesn't require a complete revamp of the whole system.


Sure, I realize that. Honestly your formula is over my head, and you're the one holding the cards on it, so it's a package deal. It's one of the reasons I'm crowdsourcing this. I'm taking new methodology to maintain the current set of categories, and in this case your formula would be excellent, except I wouldn't really know how to change it to do what it needs to do, so that means I have to discard it. Not much debate here so I won't harp on it. I'd love for you to really go to town on this but I totally get why you wouldn't want to.


Sure, transparency is important, that's why I spent all this time publishing the formulas we're using now. We're not into it to confuse people, that's one of the reasons I really like the way we're doing it now, as it's easy to understand. Problem is, that's no longer adequate for all the ways we tabulate the rankings, so we'll have to tweak it further. SSX 2012 is not that different from the regular history of the games, and we've managed to create a good system for the previous 5 games. This is more about your personal preference how the scoreboard should be done, not a problem with the system as a whole (which is a statement I'm sure you'll disagree with in your much more broad mathematical view, but the community's been happy with it since it was instituted.)


It's absolutely about that. Why limit it to just per mode, or per game? Too limited in scope. No point in spending money to create something so limited. Anyone can look a cross section of the user scores and see what ranking they are and who is, in general, the best racer/tricker. That's not much of an addition. The whole point is to equate trick and race times into something that can be understood, tabulated, and then rank people across modes, across games, to the ultimate point of declaring the best player through the series. Until SSX 2012, we did do that, and rather well I thought. Sure, the system wasn't perfect, but it's understandable and a good estimate of skill.

It served well until the snowball of SSX 2012 scores and the lack of a ceiling for survival scores. Those are the two issues here, and that remains unsolved. I will think about that and see what might be done about both.


I'm sure it's easy for Ozmo or any programmer worth his salt (as Ozmo undoubtedly is) for that matter, but while I get the idea behind it it's beyond my understanding without a bunch of studying. I'll have to enlist the aid of some math-heads I think.

_________________

I'm reviewing everything on this thread and also reacquainting myself with Standard Deviation, so I'll have a little better input on everything to get this back on track. I took several classes on statistics when I was in college, and I realized I have studied this before. But over the course of real life I just didn't use it, and so I need to refresh my math-addled brain on the steps involved.

With that said, I wanted to correct a small error on Ozmo's part from his first post -


You are using the wrong equation for the showoff/survival scoreboards here. The numbers above are using the RACING equation (10000 * 0.99^(mean(catscore - cattopscore))) for showoff numbers. In survival or showoff, the equation used is this one: (mean(catscore/cattopscore) * 10000) - we DO use this for the showoff and survival scores, as you suggested below:


Yes, it is basic, which is why we've been using it the whole time.


Example of Showoff/Survival math:


(100,000/300,000) = .33
90/150 = .6
150/150 = 1.0
210/250 = .84

(.33 + .6 + 1.0 +.84)/4 = 0.6925*10,000 = 6925/10,000

The problem is you can't use this equation with racing (times in seconds) because the dynamic is different - you're trying to achieve the lowest score, not the highest, which is why we used the equation mentioned before: (10000 * 0.99^(mean(catscore - cattopscore))). This works for the most part for racing times, but as you mentioned, it definitely does mean that any track that has a wide deviation of a top score can inflict penalties on other users who aren't in the ballpark of the few tracks with wacky scores. We realized this whenever we first came up with the rankings mod. The Peak Races in SSX 3 were the problem here. Over the course of time they had been glitched out, resulting in a super low top time that meant a wide difference in time that penalized users who didn't glitch out the Peak Races. You can read below:



Track Divisors
The solution we came up for this was to institute a divisor on these tracks that have such wide differences between average and top score.


The divisor lowers the difference between first and last place to a more palatable figure that was based upon the average in each track in SSX 3. It minimized the penalties users were given on these tracks. It's possible we could institute a divisor on the Survival tracks to equalize things a bit, what with RockLee being so dominant - but Survival doesn't use the same equation, so I'm not sure this is even necessary. He's dominating, and fairly so. No problem there.

The other option we had instituted before we used track divisors is a score ceiling. A maximum score a user could achieve on a certain track that would limit the difference between first and last place. We could institute a ceiling on Survival tracks, but again, since it's not really penalizing the other players because it doesn't use the Racing formula, but rather the Showoff formula. So I'm not sure anything really needs to be done about the Survival Scores - what we need to work on more is a way to better weight the "Overall" scores so that SSX 2012 is not that dominant in the standings.

Now that I've done some studying up on this, I'm going to write out the math on the formulas you proposed and see what comes of it.

_________________