Verbatim: Stats guru Jeff Sagarin (Part 2)
Jeff Sagarin is one of the Godfathers of data-based sports projections, having started earning money for his work in the early 1970s after graduating from M.I.T. The New Rochelle, N.Y., native has provided rankings to USA Today since the mid-1980s and also has provided advisory services to multiple professional sports franchises. His Sagarin Ratings were one of the rankings used in college football's BCS calculations and the NCAA tournament committee also has access to his work.
SI.com was able to catch up with Sagarin last Friday for a captivating conversation. Part 1 of the discussion, in which Sagarin provided depth and color to how he got started in his profession, is here. Today in Part 2, Sagarin talks more about current-day applications and the understanding of projections in sports.
SI: I’ve spoken to (current NCAA tournament overseer) David Worlock and said repeatedly “What is the issue with having some margin of victory cap or diminishing returns in the formula, so there’s a sportsmanship aspect?” You can’t sit here and look at one team beating someone by two points and say [that’s] as good as another team that beat them by 21. It’s not true.
Jeff Sagarin: You can show in the RPI where a bad team is going to play a good team -- they’d have to be pretty far apart in rating -- and then the crappy team, by the act of playing the game, will gain rating even if they lose, and the better team, will lose in the rating, even if they win the game.
People would say ‘Well, gee, Jeff, that could happen in your system.” Yeah, but I’m using scores. So hypothetically, let’s say Duke were to beat Grambling by one point at Cameron Indoor, yeah, Duke should go down in rating and Grambling should go up. But in a system that’s measuring metric is winning and losing, that doesn’t even know the score, it’s wrong that Duke should go down and Grambling should go up by the virtue of them playing and Duke winning. Because, remember, it doesn’t know the score, and even if it did, it can’t use it.
SI: Do you care about game results as they apply to your ratings, even though we understand they’re one-game samples?
JS: Let’s say with the NCAA championship game in basketball, it was Louisville and Michigan. I had Louisville to win by four or whatever the betting line was, but let’s say hypothetically I had Louisville to win by 27 and they won a real close game, yeah, that would make me feel “Darn, I wish they had won by closer to 27.” So, yeah, basically I find myself rather than having a personal attachment to something, I find as my friends say, “What a surprise! Jeff Sagarin is rooting for himself.”
SI: What did you personally learn working for professional teams? I know you were working for the Dallas Mavericks for awhile. And what did you take away as far as where pro teams are with data acceptance?
JS: It’s always interesting. The last season we were with Dallas, I believe, was the 2010-11 season I guess when they won everything. We have not been with them the last two seasons. Me and my friend Wayne Winston, we’ve been with the New York Knicks.
SI: Well, that explains everything. I’m a Knicks fan.
JS: Well, the Knicks got better this past year. We helped them!
SI: Without prying into proprietary information, do you ever read coverage of a team you’re advising? People saying “Well, they can’t play this way or Carmelo Anthony can’t go so at much as he does?”
JS: I’ll give you something even better than that. One of my friends from my home neighborhood in New Rochelle, he’s a huge Knicks fans, so I’ll tentatively tell him some of the secret stuff, we got so-and-so, and he says, “That’s bull---! This is totally wrong!”
SI: I say to my readers all the time: If I could pick games with 60 percent accuracy, I’d be living in a big mansion in Nevada.
JS: People don’t understand. When I first started and people started reading what I was doing in the fall of 1972, I was only 24, and I was so cocky, I said “I’ll put those guys in Vegas on their backs. I’ll show them who’s the boss.” Well, I found out that guys with the diamond pinky rings put out pretty good odds. Damn good odds. When I started challenging them week to week, sort of using a pure math, thinking-it-through strategy, the cliche I arrived at was “Real life is under no obligation to perfectly conform to a mathematical model created by a human being.”
SI: That’s what I was getting at. People believe if you’re a quant guy, you only believe in numbers, and I see this trend and this team should win by six. And I say, I start with stats but there’s no substitute at the end of the day for watching games or watching tape to either support what your initial belief is or counter it.
JS: You end up recalibrating your model to reflect what’s actually happening. It’s the model’s obligation to reflect real life. Not real life’s obligation to reflect your model.
SI: I guess that goes back to when you go to tweak your models. You said you’re comfortable where you are with your models…
JS: Oh, I tweak them. I have the results of all the games for any given time segment and I’ll test to see what would have worked the best. Then my goal is to find what would have worked the best for this season, what would the parameter be for last season. I like to see parameters that are similar, for an optimal parameter that works from year to year. If one year, the parameter is 1.1 and then the next year, it’s 99.7 and then the next, it’s 61.3, that doesn’t give you much confidence.
Luckily, the way I have been analyzing lately, I have been getting similar parameters, so I can live with this. You’ve put in as much effort as you can, and at a certain point, you can’t get any more blood out of a stone. If you’re trying to look at average points error in these games, even if the season is totally over and you’re just retro-fitting it, there’s a limit as to how accurate you can be. Could I go back retroactively and get the right score in every game? No.
SI: I think it’s a philosophical thing. You can have a ratings system that’s as good as anything, but at the end of the day can’t account for human interaction or account for where you have a bunch of 80-20 situations that sometimes break the 20 percent way.
JS: Here's an example. I was rooting for San Antonio to beat the Miami Heat because those guys never shoot their mouths off, they come to work and don’t have their fingers pointing at their own shoulders, like look at me! To have them lose with 25 seconds left, up by five, and they’re going to be at the free throw line two more times [San Antonio had two free throws on their next possession]. If you simulate that situation, up five, other team has the ball, but you’re going to be at the free throw line two more independent times, it’s almost impossible to lose.
They’re going to have two threes they’re going to have to make, you’re going to have free throws you have to miss. It was incredible that they lost that game. Miami had to get a rebound on the three that tied the game because they actually missed the first three and got the rebound as time was expiring on the second three.
Yet, there’s going to be some pompous talking head say, “Obviously, San Antonio didn’t have the killer instinct. Miami showed the heart of a champion.” They got lucky, and San Antonio was the victim of random fate.
SI: Sports are not like chess, which is one on one with a finite number of defined moves, but I would argue that any sport is to some extent solvable. As a mathematician, how close do you think we can get to “perfect play” in any kind of sporting event?
JS: I think it’s laughable. It’s impossible. Put it this way. Let’s say you have a ping pong table in your rec room and you have two of your kids play each other, or you’re one of the people playing and you play your next door neighbor every night, two out of three because you like to and it keeps your reflexes going. Is the score of every game the same? No. Some nights, he might skunk you, beat you 21-7. The following night, you might beat him 21-14. You’re still the same human beings. What happened? There’s just a randomness to life. That’s called being on a random curve.
Everyone talks about the bell curve. There’s a million curves in math, all similar to the bell curve. There’s nothing magic about the bell curve, OK? But the point is that whatever the distribution around the mean is, there’s always going to be a randomness. After the season is over, I’ll tell the program “OK, now with the ratings you have, plug them back in,” [and you see] how many you got right and how many you have wrong and what’s the margin of error.
The average margin of error after the season is over, plugging it back in -- it depends on the sport, what the margin is -- but let’s say 10 points a game error. It’s randomness. You can’t penetrate that. The National Security Agency could not come up with an algorithm [better than that]. You can’t get lower than a certain number based on the specific scores. A lot of times they call that the “least squares fit.” You can’t do better than that. And that number is always, in basketball or football, 10, 11, 12, 13 points. And that’s just the way it is.