Should we put stock into ESPN’s NBARank?
Over the past few days, we’ve begun a lovely yearly tradition where the major sports outlets generate lists designed to rank NBA players. We’ve also begun the necessary follow-up tradition in which some player gets annoyed because their ranking was too low. C.J. McCollum, DeMar DeRozan, Victor Oladipo, Daryl Morey, and Carmelo Anthony have all voiced heavy negative opinions about the varying lists this year. It’s even boiled over on Twitter, with several other role players mocking the journalists who assembled the lists.
But there’s a general philosophical problem in these lists that really causes most of these issues to appear in the first place: one-to-one player comparisons virtually never work in the NBA. These kinds of lists explicitly depend on them — ESPN’s list literally is generated by the panelists choosing between a series of player pairs in direct one-to-one competition. The lists also implicitly depend on them; SI’s list assumes that if you put player 53 in direct competition with player 54, player 53 would always be more valuable. This logic, unfortunately, simply can’t work.
One of my personal favorite player comparisons to use, for example, is Steven Adams and Cody Zeller. There are, of course, the basic box score stats, which line up as follows per possession:
[table id=7 /]
There’s also just the general characteristics of them as players. Adams and Zeller both have limited range offensively, taking the majority of their shots within three to five feet of the basket. Both developed a little push shot to cope with this weakness against rim protectors. They predominantly work from the pick and roll in order to get their shots, and they’re excellent screeners in order to make that work as well.
On defense, Adams and Zeller are somewhat limited as rim protectors, but they’re consistently within one-percent of the other in terms of what percentages they allow. To make up for that, they’re both excellent to elite at stepping out to the perimeter and containing perimeter players. Heck, even in rebounding, which very often has more to do with the scheme than with the player, they gather and defer comparable numbers of rebounds.
In other words, Steven Adams and Cody Zeller are two of the most directly comparable players in the league.
And yet, while they play very similar styles, for their team, they might as well be worlds different. When you watch the Thunder, the majority of their screens are set at the three-point line. This minimizes the defense’s time to react to Westbrook and allows him to get to the rim most effectively. As a result, Steven Adams tends to catch the ball deeper on the pick and roll and use his strength to go up through contact. Cody Zeller, on the contrary, does not go up through contact as well as Adams.
Similarly, the Hornets often set their screens two to three feet beyond the three-point line, utilizing Kemba Walker’s ability to pull up from three with a defender trailing him. As a result, Cody Zeller tends to catch the ball off of the screen and roll closer to the elbow, where he uses his quickness, relative to most centers, in order to beat his man off the dribble. This is something Zeller does with a degree of success, whereas Steven Adams does not.
For their respective teams, swapping Adams and Zeller wouldn’t make much sense given their specific roles. For two players that are as closely comparable as you’re going to find in the NBA, it doesn’t make much sense to rank them against one another because for one team it’s Cody, and for the other it’s Adams.
Now imagine that the players are Rudy Gobert and Damian Lillard. The only thing that they both do well is rebound. Lillard ranked eighth on a per-possession basis among starting point guards and Gobert ranked seventh among starting centers. Even that doesn’t look remotely similar since, for obvious reasons, they gather different quantities at different areas on the floor. Gobert took 99.1% of his shots from within ten feet of the basket last year, whereas Lillard is a threat to score from anywhere on the court. Meanwhile, on defense, Gobert is probably one of the league’s best rim protectors and individual defensive centers. Dame is, optimistically, able to check his own matchup and that’s about it. And if you felt really optimistic, you could keep breaking them down further, but all that would accomplish is demonstrating the painfully obvious: Gobert and Lillard have almost nothing in common as players.
At some point, though, Sports Illustrated, who ranked them fifteenth and seventeenth respectively, had to compare them. Their entire list is filled with comparisons like this one that simply can’t have any bearing in reality; there’s simply no good basis by which to compare such drastically disparate players.
Of course, there are aggregate advanced stats. Those do theoretically provide a basis to weigh individual advantages against each other from players who differ heavily. But using those to rank players has a whole slew of problems:
1. Individual stats tend to have some individual flaw that biases them in a subset of the data.
ESPN’s Real Plus-Minus, which is the strongest of the commonly accessed metrics, for example, biases against quality defenders who are shorter and occasionally incorrectly distributes their credit. Further, players in strong lineups that play large minutes together often see their impact disproportionately spread out. As a result, for those player pairs, the statistic isn’t reliable to resolve the difference.
2. There’s no good way to resolve the dichotomy between rate and volume.
Is it fair to assume that Joel Embiid could do exactly what he did over a full 82 game season? Not at all, but then you have to start to figure out how much he actually loses by having a full season. But then how much do you reward a guy like Trevor Ariza, who played a massive amount of minutes at a so-so efficiency rate? There are some techniques that have been established to deal with this like adjusted True Shooting; however, those are typically insufficient and also rarely used. So for those player pairs, the statistic isn’t reliable to resolve the difference.
3. The stat itself isn’t designed to be anything more than an estimator
With most of the modern stats, the results are regression determined at some step along the way. The problem is that most regressions give you a band within which the result probably actually falls. So when you’re comparing two players that are close, the best you can say is that it’s slightly more likely that the player with the better performance stat is also better. So again, for those player pairs, the statistic isn’t reliable to resolve the difference.
As a result, there simply isn’t ever going to be any winning with a list like these. Not only is there no good way to actually discriminate between players, most of the time it doesn’t even make sense to try to do so because players never serve the exact same function and there isn’t a choice to be made between them. And yeah, that may not be the best for fan entertainment, as these lists wouldn’t keep running if people didn’t lap up every drop of the controversy. However, the best way to see these lists is that they are ultimately meaningless and of no consequence whatsoever.