Meh.
I wanted to avoid getting involved in what is turning out to be a pretty telling philosophical disconnect between some at PPP and MLHS. However, I just can’t let this post slide.
Steve Burtch is quite a smart guy. He’s full of ideas, loves to write, and the passion in his work is both amusing in its audacity and engaging with its enormous depth of information. His SDI research raised some good discussion points that perhaps we, as both hardcore and voracious hockey fans, could stand to know more about the evolving role of a shutdown defenseman and perhaps take these discussion points as a mean of contextual evidence when applied at the individual level.
And to be honest, that’s all advanced stats should be – context and evidence, but not a truth.
But Burtch’s most recent post irked me. Instead of approaching Declan’s post point by point, he created one red herring after another and essentially missed the meat of what Declan was trying to say. All Declan said was that the Leafs, as a team, had many holes – particularly down the middle, and it was bound to reflect in the columns of CORSI Rel and Fenwick Close.
While raising the completely fair point that Reimer faces more shots than what’s statistically normal, Burtch attempted to conflate shot quantity with shot quality. What also bugged me most is how Burtch inferred that there was a higher degree of shots taken from a certain area, when it’s obvious to anyone that watches hockey that more shots are taken the further away from the net.
Burtch’s argument is poor on many levels. For one, it’s impossible to apply a quantitative measure to the state of quality. Quality is a transcendental act of an individual deferring to instincts and making a choice. It’s an amalgam of both experience and conscious level of decision making. It is muscle memory gleaned from hours and hours of practice. Quality is a lot of things. But quality is neither ascribable to numbers nor applicable of dimensions (see: home-plate scoring chance area).
In addition, the Leafs played a system similar to a box plus one system in their zone. While it was the coaching staff’s decision to protect their defensemen by using a speedy forward to help the defense, it was going to lead to shots against. The entire point of the system was to remove the man to man coverage and try and funnel shots into the goaltender, leading to a helpful millisecond of predictability. However, the overwhelming total of the shots per game was likely not in the coaching staff’s plans.
Lastly, Reimer faced a comparable number of <20’ shots to the likes of Bobrovsky, Fleury, and Rask to name a few. From that distance, Reimer had a better save percentage than all of them. Over an entire season, the Leafs would have lopped off almost 100 shots against from <20’ compared to previous years under Wilson. However, the Leafs would have given up the most shots from outside the 20 feet area of the net since the first record of shot location was made available. Of note, Reimer would have led the league in shots against from OUTSIDE 20’. Of course, that CONTEXT would be misplaced if it wasn’t for the fact that Declan specifically mentioned it in his original post:
Reimer had some easy nights of lots of routine, positional saves where he was giving up big-time rebounds. He had lots of support and benefited from plenty of shots from non-critical areas of the ice.
That would be CONTEXT by the way.
Over the last few years, CORSI and Fenwick has risen in prominence as a metric capable of a certain degree of predictability. Its usefulness resonates nicely with a quote from Gretzky: “You miss 100% of the shots you don’t take”. Okay, that makes a lot of sense.
However, as we know for the most part, that predictability fails considerably when applied at the individual level. See this quality comment from JeffGM of PPP:
The other problem I see with the advanced stats is that most of the studies have been completed on the team level. I haven’t seen work to show that advanced stats on an individual level offer similar correlation, usefulness or are even accurate. We know many “good” players look bad when using advanced stats (Phaneuf, McClement, Kulemin etc). There are other players that have both a better CORSI and are far worse than these players. And then there are players that have a good CORSI but poor goal production (Zherdev and Gomez).
When applied to individuals, advanced stats are a lot of like +/- that you mention which is more useful as a team metric and less useful when looking at a player (or at least more context is needed when applying them to the individual). That said, shots differentials are “better” then +/- at the team level (and probably at the individual level) but people may have too much faith in them when they are unproven at the individual level. And what level correlation to winning do advanced stats have at an individual level when they only correlate to winning at the team level with 35%.
The point here is that we need to be very suspicious of what advanced stats may portend and apply a lot more context then we do at the team level. That said, speaking as conspiracy theorist I’m not sure how often that happens.
Putting it simply, all Jeff is saying that as a rule, it’s not favourable to any argument to apply CORSI at the individual level because it tends to ignore the concept of roles – but it does give us an idea of how the player is performing in that role. That said, I think JeffGM is still very much in favour of using advanced stats as a rule of thumb when applied at the team level.
So where does that leave the advanced stats argument regarding predictability at the team level?
Here, at MLHS, we spend a lot of time discussing the smallest details about the Toronto Maple Leafs. We write, we post, we discuss. These are core tenets of any website that purports itself as a content rich place to go for anyone who shares a similar interest. We have many posters who believe in the accuracy of advanced stats and we have just as many who dislike it.
None of us at MLHS purport to be experts in the field of advanced stats, but we do try to incorporate it when looking for context to add to a post. Mike Stephens is a willing participant in this case, as well as Aaron Chan. I am as well as I have indicated many times that I like reading them because they share some roots with baseball’s Sabermetrics.
But I have never seen an argument or group dissent break out within the commenters on MLHS. Nor have I seen anyone other than myself treat the concept with derision. That doesn’t mean I find advanced stats to be flaky or disagreeable to my tastes. I find that the root of my disagreement with advanced stats comes from the people who use them as an overarching argument – or putting it more bluntly, foist it on anyone as a centric theme of their vitriol.
Among people I follow on twitter, David Johnson has made a name for himself as someone who eagerly embraces advanced stats and takes the time to study the results. In addition, Johnson is someone who doesn’t hesitate to question the viability of CORSI or Fenwick.
Not too long ago, Johnson wrote up an interesting take on the decreasing value of Fenwick and CORSI with an increased sample size. I won’t dig too deeply, but Johnson took one column of a team specific stat and measured it against winning percentage to find an r-square value between 0 and 1. The takeaway is that the defensive side of the game, which includes goaltending, favours winning more than scoring.
This is important for several reasons:
a) It reaffirms that at the pro-level, coaches are insistent about making defense the primary team focus;
b) It reaffirms that goaltending does win games – see GA60 and save percentage in the second table;
c) It reaffirms that blocked shots may have more value than previously suggested as a shot suppression via shot funneling or shots blocked (FF% — CORSI minus shots blocked). This doesn’t mean all blocked shots are good, but it requires player judgment – a quality. Conversely, I also recommend reading up on the value of blocked shots from Cam Charron. Keep in mind that blocked shots aren’t reliably recorded;
d) Shooting percentage correlates better at the individual level rather than the team level – see on-ice shooting percentage. This would suggest to some degree that the Leafs, regardless of their many flaws, focused their shot suppression by matching up line against line rather than at a team level.
While all those effects are well-reported within hockey circles and the fans that follow the sport, I find it encouraging that Johnson is willing to explore the repeatability of CORSI and Fenwick to understand what’s wrong with the stats and how to best adjust for deviations when they arise. That kind of effort and research is important if advanced stats are to take off as a repeatable and predictable measure of team success at the professional level. Johnson encourages discussion rather than suppresses it.
So rather than disagree entirely with the concept of advanced stats, I would much rather explore the nuances than altogether dismiss an opinion like Burtch did with flimsy off-handed logic that attempted to qualify a quantity that’s looking less and less reliable.
Then again, on any other site, confirmation bias doesn’t seem to rear its ugly head as often.
