Anyone that follows me on twitter or that has read any of my pieces here at MLHS knows that I enjoy using possession statistics alongside production statistics to examine and evaluate players. After recent events, like Lupul’s tweets and Alec’s interview with Greg Cronin, that have stirred up the tension between those that use these statistics and those that don’t, I thought I’d dig into why the use of statistics should be embraced.
A few common responses that I’m seeing to the use of statistics in general, but especially possession statistics, are “watch the game, statistics don’t tell you everything,” or “watching the game tells me everything I need to know” or even “statistics are basically magic.”
The first response isn’t incorrect. In fact, there are very few (if any) bloggers that would tell you statistics are all-encompassing measures of a player. Many bloggers (and fans) use a combination of video evidence and collected statistics to evaluate and compare players, teams and coaches. Disgraced journalist Jonah Lehrer wrote an article about the problems with sabermetrics in baseball (mainly that people are fixating on these numbers as the be-all and end-all) and Bill Petti wrote a critical response that has a great (and relevant) quote:
[pull_quote_center]… Lehrer’s main argument shouldn’t be that teams are assembling bad teams because of a narrow-minded focus on things they can quantify. The argument should be that teams that don’t think deeply about what are the right metrics and how much variance they account for in player achievement will fail just as much as those teams that used to generally ignore analytical approaches to the game. Data and statistics are not to blame for bad decisions–their misapplication is.[/pull_quote_center]
What I love about this quote is that it echoes something that I’ve said in response to Cronin’s interview with Alec: that just because an NHL team is using certain statistics to make decisions, doesn’t mean that method is the right way to proceed. In other words, I’m warning people about confusing what is and what should be.
The third response seems pretty silly to me and since there really aren’t that many people forwarding that view I’ll leave it at that.
The second response isn’t necessarily incorrect, but it’s highly unlikely and that’s really what I want to focus on in this piece.
Why is it highly unlikely that watching the game tells you everything you need to know? It’s a great question and there are multiple challenges.
Let’s start with the old adage “practice makes perfect,” and apply it to responses about how much hockey you’ve watched. Well, as it turns out, that adage should really go “proper/perfect practice makes perfect.” The idea that I’m trying to convey here is that just watching the games isn’t enough to have you excel in understanding hockey. You’ll need to approach watching hockey in a systematic way that pushes you to your limits and that aims to improve your ability to understand hockey. This, in part, brings us back to the idea of needing know what we need to be focusing on and what events are important to understanding how to evaluate players, coaches and teams. Cue the next challenge.
Let’s say that you have a systematic approach to watching hockey and now you want to know what events you need to focus on. People are influenced by conspicuity (how eye catching something is), and so when deciding what to focus on, that’s likely where you’ll start. This is where you’ll run into the challenge of inattentional blindness. Inattentional blindness occurs when you don’t notice an unexpected event in your field of vision because you’re focusing on another event.
Why does this matter when watching hockey? If you’re focusing on the conspicuous events (possibilities: goals, hits, big saves) then you’re less likely to notice less conspicuous events (possibilities: set plays, zone exits/entries, pre-shot positioning) and you’re more likely to see what you expect to see (perhaps why Cronin thought he saw Grabovski turn the puck over in game 7). Inattentional blindness shows that even if the event is conspicuous, like a player entering the attacking zone, if you don’t know to look for that event and you’re focused on something else, like a hit that just happened, you’d likely miss the zone entry. Good news though: once you know to look for certain things they become easier to spot. The problem is that you can’t focus on everything.
Now, let’s assume that you have a systematic approach to watching hockey and you know what you want to focus on. After a game, or a season, is over and you go to evaluate a player (or coach or team), you first have to recall everything you’ve seen and despite your system and focus, your mind will play tricks on you. Most people think the brain works like a video camera that stores your memories perfectly, but the reality is that your memory is malleable, stored in pieces and events reconstructed every time you call on it. When you listen to the commentators describe the game, the way you remember the events they are talking about is influenced by the terminology used. For example, stronger phrasing, like “bulldozed” over “hit”, can lead to altered memories to support the phrasing (meaning you’ll remember the hit as bigger than it was).
On top of that, negative memories are more accurate than positive memories and studies lead to the suggestion that “individuals in a negative mood process information in an analytical and detailed fashion, whereas people in a positive mood rely on broader schematic or thematic information and ignore the details.” (Note: This is probably an important piece of information to keep in mind for Leafs fans as we head into next season after making the playoffs for the first time since 2004.)
Despite your system and your focus, not only is your memory unreliable and malleable but your mind is also subject to cognitive biases and heuristics (mental short-cuts that make problem solving simpler) that impact your judgment and decision-making. There are many heuristics and cognitive biases but I’m going to narrow the focus to just a few big ones (though I strongly encourage you to look at as many as you can).
Let’s take a look at availability heuristic first. This heuristic occurs when you assign the probability of an event occurring based on the ease with which you can recall a similar event to mind. So, you see a player turn the puck over (and because that may be a negative even you may be more likely to remember it) during the season and it lead to a goal against. When you try to recall how this player is defensively and that image jumps into your head, you’re more likely think that it’s a frequent occurrence even though it may not be. It’s worth noting that the images called to mind don’t have to be memories, imagined scenarios work just as well.
Next on the list is representativeness heuristic. This heuristic states that when slotting objects into multiple classes, the probability with which an object belongs to a class is dependent upon the degree to which that object is representative of the stereotype of that class. Very wordy, but a simple example is the stereotype of defensive-defensemen being big, mean, crease-clearing machines and then asking you whether a 5’10 defenseman that looks honest and plays cleanly is a defensive defenseman.
Confirmation bias is a tendency to favour information that supports your preconceived notion, irrespective of the validity of that information. This leads to the selective use of memory and information gathering skills. For example, you are more likely to remember or notice giveaways by a player that you think (rightly or wrongly) is a turnover machine.
The halo effect is another bias that can have a large impact on decision-making. This bias occurs when the perception of an element of an object spills over into other elements of that object. This can result in the evaluation of a player that exhibits generally likeable aspect to their game like blocking shots (*cough* Tim Brent *cough*) or physical play being more favourable than otherwise would be the case. The opposite bias (less favourable evaluation due to spillovers from a negative aspect) is the horn effect.
So, even if you are observing the game properly and know what to look for, your observation and interpretation of the game and the results are inherently biased and your memory unreliable. Hopefully, you are beginning to see the potential interaction between some of these phenomena and why they complicate observation, recollection and evaluation.
What’s the good news? Using statistics can help overcome some of these challenges (though they also present some different ones) by attempting to provide objective accounts of transpired events that can act as fact checking measures on what we perceive, observe and remember.
I’ll leave this monstrosity of an article (though there is a lot unsaid and I wish I could have touched on Bayesian thinking) with a question: is it statistics that need to be confirmed by observation, or the other way around?
