I wanted to avoid getting involved in what is turning out to be a pretty telling philosophical disconnect between some at PPP and MLHS. However, I just can’t let this post slide.
Steve Burtch is quite a smart guy. He’s full of ideas, loves to write, and the passion in his work is both amusing in its audacity and engaging with its enormous depth of information. His SDI research raised some good discussion points that perhaps we, as both hardcore and voracious hockey fans, could stand to know more about the evolving role of a shutdown defenseman and perhaps take these discussion points as a mean of contextual evidence when applied at the individual level.
And to be honest, that’s all advanced stats should be – context and evidence, but not a truth.
But Burtch’s most recent post irked me. Instead of approaching Declan’s post point by point, he created one red herring after another and essentially missed the meat of what Declan was trying to say. All Declan said was that the Leafs, as a team, had many holes – particularly down the middle, and it was bound to reflect in the columns of CORSI Rel and Fenwick Close.
While raising the completely fair point that Reimer faces more shots than what’s statistically normal, Burtch attempted to conflate shot quantity with shot quality. What also bugged me most is how Burtch inferred that there was a higher degree of shots taken from a certain area, when it’s obvious to anyone that watches hockey that more shots are taken the further away from the net.
Burtch’s argument is poor on many levels. For one, it’s impossible to apply a quantitative measure to the state of quality. Quality is a transcendental act of an individual deferring to instincts and making a choice. It’s an amalgam of both experience and conscious level of decision making. It is muscle memory gleaned from hours and hours of practice. Quality is a lot of things. But quality is neither ascribable to numbers nor applicable of dimensions (see: home-plate scoring chance area).
In addition, the Leafs played a system similar to a box plus one system in their zone. While it was the coaching staff’s decision to protect their defensemen by using a speedy forward to help the defense, it was going to lead to shots against. The entire point of the system was to remove the man to man coverage and try and funnel shots into the goaltender, leading to a helpful millisecond of predictability. However, the overwhelming total of the shots per game was likely not in the coaching staff’s plans.
Lastly, Reimer faced a comparable number of <20’ shots to the likes of Bobrovsky, Fleury, and Rask to name a few. From that distance, Reimer had a better save percentage than all of them. Over an entire season, the Leafs would have lopped off almost 100 shots against from <20’ compared to previous years under Wilson. However, the Leafs would have given up the most shots from outside the 20 feet area of the net since the first record of shot location was made available. Of note, Reimer would have led the league in shots against from OUTSIDE 20’. Of course, that CONTEXT would be misplaced if it wasn’t for the fact that Declan specifically mentioned it in his original post:
Reimer had some easy nights of lots of routine, positional saves where he was giving up big-time rebounds. He had lots of support and benefited from plenty of shots from non-critical areas of the ice.
That would be CONTEXT by the way.
Over the last few years, CORSI and Fenwick has risen in prominence as a metric capable of a certain degree of predictability. Its usefulness resonates nicely with a quote from Gretzky: “You miss 100% of the shots you don’t take”. Okay, that makes a lot of sense.
However, as we know for the most part, that predictability fails considerably when applied at the individual level. See this quality comment from JeffGM of PPP:
The other problem I see with the advanced stats is that most of the studies have been completed on the team level. I haven’t seen work to show that advanced stats on an individual level offer similar correlation, usefulness or are even accurate. We know many “good” players look bad when using advanced stats (Phaneuf, McClement, Kulemin etc). There are other players that have both a better CORSI and are far worse than these players. And then there are players that have a good CORSI but poor goal production (Zherdev and Gomez).
When applied to individuals, advanced stats are a lot of like +/- that you mention which is more useful as a team metric and less useful when looking at a player (or at least more context is needed when applying them to the individual). That said, shots differentials are “better” then +/- at the team level (and probably at the individual level) but people may have too much faith in them when they are unproven at the individual level. And what level correlation to winning do advanced stats have at an individual level when they only correlate to winning at the team level with 35%.
The point here is that we need to be very suspicious of what advanced stats may portend and apply a lot more context then we do at the team level. That said, speaking as conspiracy theorist I’m not sure how often that happens.
Putting it simply, all Jeff is saying that as a rule, it’s not favourable to any argument to apply CORSI at the individual level because it tends to ignore the concept of roles – but it does give us an idea of how the player is performing in that role. That said, I think JeffGM is still very much in favour of using advanced stats as a rule of thumb when applied at the team level.
So where does that leave the advanced stats argument regarding predictability at the team level?
Here, at MLHS, we spend a lot of time discussing the smallest details about the Toronto Maple Leafs. We write, we post, we discuss. These are core tenets of any website that purports itself as a content rich place to go for anyone who shares a similar interest. We have many posters who believe in the accuracy of advanced stats and we have just as many who dislike it.
None of us at MLHS purport to be experts in the field of advanced stats, but we do try to incorporate it when looking for context to add to a post. Mike Stephens is a willing participant in this case, as well as Aaron Chan. I am as well as I have indicated many times that I like reading them because they share some roots with baseball’s Sabermetrics.
But I have never seen an argument or group dissent break out within the commenters on MLHS. Nor have I seen anyone other than myself treat the concept with derision. That doesn’t mean I find advanced stats to be flaky or disagreeable to my tastes. I find that the root of my disagreement with advanced stats comes from the people who use them as an overarching argument – or putting it more bluntly, foist it on anyone as a centric theme of their vitriol.
Among people I follow on twitter, David Johnson has made a name for himself as someone who eagerly embraces advanced stats and takes the time to study the results. In addition, Johnson is someone who doesn’t hesitate to question the viability of CORSI or Fenwick.
Not too long ago, Johnson wrote up an interesting take on the decreasing value of Fenwick and CORSI with an increased sample size. I won’t dig too deeply, but Johnson took one column of a team specific stat and measured it against winning percentage to find an r-square value between 0 and 1. The takeaway is that the defensive side of the game, which includes goaltending, favours winning more than scoring.
This is important for several reasons:
a) It reaffirms that at the pro-level, coaches are insistent about making defense the primary team focus;
b) It reaffirms that goaltending does win games – see GA60 and save percentage in the second table;
c) It reaffirms that blocked shots may have more value than previously suggested as a shot suppression via shot funneling or shots blocked (FF% — CORSI minus shots blocked). This doesn’t mean all blocked shots are good, but it requires player judgment – a quality. Conversely, I also recommend reading up on the value of blocked shots from Cam Charron. Keep in mind that blocked shots aren’t reliably recorded;
d) Shooting percentage correlates better at the individual level rather than the team level – see on-ice shooting percentage. This would suggest to some degree that the Leafs, regardless of their many flaws, focused their shot suppression by matching up line against line rather than at a team level.
While all those effects are well-reported within hockey circles and the fans that follow the sport, I find it encouraging that Johnson is willing to explore the repeatability of CORSI and Fenwick to understand what’s wrong with the stats and how to best adjust for deviations when they arise. That kind of effort and research is important if advanced stats are to take off as a repeatable and predictable measure of team success at the professional level. Johnson encourages discussion rather than suppresses it.
So rather than disagree entirely with the concept of advanced stats, I would much rather explore the nuances than altogether dismiss an opinion like Burtch did with flimsy off-handed logic that attempted to qualify a quantity that’s looking less and less reliable.
Then again, on any other site, confirmation bias doesn’t seem to rear its ugly head as often.
I cant even post on my home or work computers, this has been going on for months now..fix this site, its my fav-TOLEAFS.
There's a Cody Franson review posted but for some reason its not showing on the home page
Ok guys, everybody will probably laugh at this...but what about signing Grant Clitsome? I mean, the name is sick, and lookin' at his stats he doesn't look too bad!
System is inconsistent on my Blackberry when replying... Very slow and usually don't get the previous poster's name defaulting in the comment box... Typing new comments is OK...
The Bruins are taking a page out of the Flyers playbook from last yrs playoffs . Its not difficult to get Crosby and the Pens rattled and the Bruins played them like a fiddle last night .
Great post, I especially like that last bit about Corsi/Fenwick. The Bruins/Pens series should be interesting - scoring vs. defense. Anyway....
I just recently rented "Moneyball", and I took away three lessons:
1. The Oakland A's proved that their metrics like on base batting average were better predictors of individual success.
2. Collecting enough "Runs Scored" and "Runs Prevented" via getting the players which your advanced stats tell you will get or prevent Runs will, inevitably, get you enough wins and into the postseason.
3. The A's repeatedly failed in the postseason. Even in baseball, is still a crapshoot and there are no guarantees there.
What the advanced stats crowd is missing in hockey is all three of these salient points. We know Phaneuf is a good defenseman, but his advanced stats suck. So we can't rely on advanced stats to predict which player will be effective like Phaneuf.
Even if point 1 were true, and hockey's advanced stats did reliably measure individual success, then there's still point 2, which is what the Oakland A's demonstrated. They demonstrated that collecting enough players with good scores in these sabermetrics will lead to regular season success. To date we haven't had a demonstration like the 2002 A's in the hockey world.
But then we still have point 3 above - that using advanced stats effectively still won't necessarily lead to a championship, which is what all of us Leafs fans want.
Just want to give major props to morrganreilly on this. Credit to PPP too, the back and forth aside I am continually amazed at the talent of contributors to both sites. Just reading the oringal article, Steve's response and then this it's astounding how good the content is. Keep up the good work everybody.
And the wheels on the bus fall right on off, right on off, right on of..... the wheel on the bus................... wait, that doesn't sound right
NHL has a fricken hardon for boston!!!!!!!! Does it take a terrorist bombing in your city to get specialized treatment????????/
Frick OFF !!!!!!!!!!!! Show the replay of Lucic's hit on Gunnarsson !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! What a fricken JOKE !!!!!!!!!!!!!!!!!!
The Leafs are a young team that are becoming fast and dangerous... They're going to eat lightning and crap thunder...
Leafs coaching staff should record this game and play for the Leafs at training camp - the Hawks are remarkably efficient and quick and decisive when it comes to clearing the puck from their zone.
Hey guys, I think the leafs also need to get bigger upfront or sign some guys with size and grit...I'd like to sign 2 of these ufas:
Bickell (hopefully chic can't afford him, prob 3mil X3yrs
Clarkson (4mil X 3yrs)
Penner(2mil X 2yrs) 3rd liner who is not afraid to stay infront of net
Torres (2mil X 2yrs) solid 4th liner who hits like a mac truck.
Handzus (2.5mil 2yrs) solid 3rd line centre with size.
I wonder what we'd have to pay to get Hamhuis out of Vancouver. He's a guy I'd like to see the Leafs go after. Our D would look so much better with a guy like him.
Hamhuis - Phaneuf
Gunnarsson - Gardiner
Fraser - Franson
Hamhuis - Phaneuf
Gardiner - Franson
Gunnarsson - Liles
Something the leafs should consider is making a deal with Clb to aquire that 14th overall pick? (They hold) since Clb has 3 1st rd picks I'd like to see Nonis send them our 1st + scrivens for #14...........thi way we have a shot at Horvat, Domi,Gauthier or lazar....
Nice post on the limitations of advanced stats as they currently are.Not totally confident hockey as a sport lends itself well to them. We may always need to keep them in context and it's a shame there are those who don't, and worse yet look down their noses at those that do like the writing staff here at MLHS
My general system for tracking if teams are good: Did they win?
Y | N
If there are at least 5 more checks in the Y section at the end of the season then they're a good team