Posted by James McCormack on May 15, 2011 at 9:55 AM | comments (1) |
I have tried and failed to post a detailed response to this post by Tom Barlow
http://community.advancednflstats.com/2011/05/fixing-elo-ratings-system.html
here is the full post it kept refusing to publish maybe because of formatting.
Tom
I am the creator of the pointshare rating system which I use to rank college football and NFL teams and also other sports (see wwwdotpointsharedotwebsdotcom particularly the FAQ and my older now discontinued blog at dontsaveapitcher.blogspot.com where I first outlined the system see the google docs on the top left hand side of the blog) .
When I developed my Pointshare rating system last year I had to wrestle with the same type of issue as you describe here. As like you I have a “game outcome function” to translate a football score into a value between 0 and 1 so have others such as Ken Massey (by the way his comparison site of >100 college football ranking systems is an essential read for you to see what other approaches people are using.)
But I think there is a flaw in your logic. If your football model says that if team A and B are equally good then team A will win by 5 or more points 40% of the time it is incorrect to then credit them with 60% skill. By definition you have said the two teams are equal so how can you credit team A with anything. This is akin to saying in statistics that because your null hypothesis has a p value of 0.05 you are 95% sure that your alternative hypothesis is correct. Which is not what a p value of 0.05 means.
The purpose of ELO (or any other rating system I guess) is to discover the underlying skill differences between teams. Knowing how likely a result is if the teams are equally skilled tells you nothing about how much better or worse team A is relative to team B.
I approached the problem from a conditional probability perspective and my game outcome function attempts to say that, given the score in the game that was played, what is the probability a team would win a rematch at the same venue.
Or in other words
if Team A defeated team B by 5 points. How likely is that result if team A would beat team B 1% of the time if they played an infinite number of games.
How likely is this result if team A would win 2% of the time,
How likely is this result if team A would win 3% of the time
etc etc up to 99%.
This gives me a distribution of what a 5 point win tells me about the relative merits of the two teams. It looks like an inverted U, I simply then look at the area under the curve greater than 50% and compare it to the area under the curve less than 50% to determine the probability that A would win a rematch.
I don’t know if your drive model allows you to vary the skill of each team to determine how likely a 5 point win is if team A are much better than team B. My really simple model of how football works allows me to do this. If it doesn’t you could use Las Vegas to help you and set your elo coefficients based on a pointspread to moneyline conversion charts which is not the same thing as what I have discussed above but as 3pt favourites win straight up 58% of the time giving a 3 point win 58% of the credit is one way of tackling this issue.
If you like I can send you my excel spreadsheet where I run my system although it is quite big and I will need some time to tidy it up to be reader friendly.
When you finalise your system I encourage you to adapt it to college football (as there may be no more NFL!!) and submit your rankings to the Massey site as it is a wonderful resource.
James