This day on January 25
Acquired: Glenn Anderson (1996)

Happy Birthday To: Angelus, barnaclehead, sharks eat oilers, RomanianOiler, cinnamon, Oilers since 84, m_fish82, Bubble

F.A.Q. Terms of Use F.A.Q. F.A.Q.
Members Members   Search Search     Register Register   Login Login   Home Home
 Speculation » Advanced stats, causality, and why numbers matter
Switch to flat viewSwitch to tree viewCreate a new topicSubmit Reply
 Advanced stats, causality, and why numbers matter [message #696596]
Tue, 27 June 2017 01:49 Go to next message
ziltoid  is currently offline ziltoid
Messages: 425
Registered: January 2011

No Cups

If this is in the wrong section please move it to wherever you see fit.



This is going to be a bit of an essay, but I think it touches on an important topic that has kicked up in the wake of recent events (e.g., Russell signing, the Eberle trade, and how different posters draw their conclusions re: all these moves). In short, I am going to provide a case for using "advanced statistics," as-well-as what we can reasonably conclude from them.


Descriptive versus Inferential

I think any advanced stats discussion needs to begin by delineating between descriptive and inferential statistics. As you would expect, descriptive statistics are simply metrics that describe some population, with the overwhelming majority of advanced stats falling into this category (e.g., shots, goals, assists, points, cori, fenwick, WOWY, zone entry and exit data, etc.). On the other hand, inferential statistics make predictions based on sample/population data. In the case of hockey, these would include things like ridge regression models prediction wins, or modelling goal-scoring as a semi-Markov process. To distill it down even further, descriptive statistics count and group things, whereas inferential statistics make heavy use of probabilities.

Main point #1 (MP1): most advanced statistics in hockey are simply counting and grouping things we directly observe.

Take corsi-for per 60; we simply count how many shots and shot attempts (blocked and missed) occur over a specific interval (e.g., a game, a season, a career, etc.), then divide by the total TOI, then multiply that result by 60.

60( (shots + shot attempts) / TOI)

There is nothing predictive about CF60; it is just a number that describes what we directly observe (corsi), but converted into a different unit (per 60). Given MP1, it is hypocritical to say these statistics are not valid but metrics such as goals and assists are (more on this later); all are direct observations, some of which simply get converted into different units and/or grouped into different sets, much in the same way kilometers can be converted into miles, or how Statistics Canada calculates things such as X per Y (e.g., debt per household, children per family).

Of course, there are people who recognize the above argument, and instead reject stats on the grounds that they, to paraphrase, "don't tell us anything about why."

The Problem of Causality

Conventional wisdom would state that even though we can measure all off these "advanced statistics," it takes an "expert" (aka, a "hockey guy") to be able to infer anything meaningful from these observed variables. That is, even though we can measure a player's corsi, it takes a trained eye to understand what causes that number to be what it is. In a sense this is true; it is hard to determine causality even under experimental control, let alone with correlational data. Thus, the bigger argument is the extent to which (i) correlational data can inform our understanding of the game, and (ii) we can reasonably infer causality.

The argument in favour of (i) is as follows:

(1) Goals, Assists, and Points all inform our understanding of team and player performance.

(2) If a measure is as valid as Goals, Assists, and Points, then it can inform our understanding of team and player performance.

(3) Advanced statistics are as valid as Goals, Assists, and Points.

Therefore, given (1-3), advanced statistics can inform our understanding of team and player metrics.

Now, the elephant in the room is, "what constitutes a valid observation." For example, we can count how many strides each player takes per shift, but one may be hard-pressed to view that as a valid metric, especially when we consider it along side metrics like goals, assists, and points. On the surface this seems like a complex and difficult question to answer, but in reality it is very simple. A valid observation is anything we directly observe and measure about the game. So yes, number of strides per shift IS as valid as goals, assists, and points, but valid does not entail useful, it only entails informative. That is, if we can directly observe and measure it, then it tells us something about the game, though whether this constitutes a useful piece of information remains to be seen. Thus, the next question with respect to (i) is, "in what way can advanced statistics usefully inform our understanding of the game."

If we take the conservative view that no correlational data can infer causality, then we are forced to view advanced statistics much in the same way we view lots of medical evidence. For example, the link between smoking and cancer is correlational; we can't randomly select 500 kids and have half of them smoke for 20 years and the other half not, then see who gets cancer; we have to correlate cancer rates with smoking while statistically controlling for mediating variables (though there are lots of other statistical ways of assessing the link between smoking and cancer, I don't want this to turn into a dense stats lecture). Even animal studies are correlational as a causal effect in rats does not entail a causal relationship in humans; it correlates with a causal relationship, but it is an imperfect proxy. Thus, from the most conservative viewpoint, advanced statistics in hockey tell us about the strength and direction of relationships between variables. Corsi has a moderate positive correlation with goals, therefore we can conclude that players with high corsi values generally score more goals than those with poor corsi values, but we cannot stake any claims on why this is (as an aside, for the 2012/13 - 2014/15 seasons, corsi-for percentage has a correlation of 0.35 with GF per 60). Clean zone exits correlate with corsi, therefore we can conclude that defensemen who produce clean zone exits generally produce more scoring chances as measured by shots and shot attempts. We can take this a step further when we delve into inferential statistics. For example, the stuff I am currently working on shows, among other things, that offense mediates the relationship between possession and defense. We cannot make any definitive claims as to why this occurs (at least not under this conservative view re: causality), but the relationship clearly exists in the data and should not be easily dismissed.

Main point #2 (MP2): advanced statistics in hockey are useful because they afford us the ability to assess the strength and direction of relationships between increasingly specific aspects of the game.

Let us now turn our attention to causality.

In short, there are 3 main criteria for determining causality such that X causes Y:

(1) X has to be correlated with Y
(2) X has to precede Y, temporally.
(3) Competing explanations of causality (within reason) must be ruled out.

Criteria (1) and (2) can, at times, be satisfied via hockey data (e.g., a goal must be proceeded by a shot), but the biggest problem we face is (3). Here, the problem is that advanced stats in hockey exhibit a high degree of multicollinearity, which is just a fancy way of saying that metrics are often highly correlated with each other. Thus even though criteria for (1) and (2) can be reasonably satisfied when trying to establish a causal relationship between corsi and goals, the fact that corsi is highly correlated with lots of other metrics means that we cannot directly rule out other competing explanations of causality. That said, there are statistical techniques we can use to control for the effects of other variables, but it is still an imperfect system. However, I do believe that if we (i) collect and analyze the data carefully, and (ii) repeatedly find strong convergent evidence for an effect/relationship, then we can say with a high level of confidence that there is very likely a causal relationship between X and Y. This goes back to the old adage that a single study does not mean anything, but 100 of them do. In reality, nothing in hockey is caused by a single factor, but rather by a combination of factors. Thus, we talk about "how much" of a factor X is in causing Y.

Main point #3 (MP3): although we can never be 100% certain with respect to causality, in the face of strong convergent evidence we can become highly confident that there is a causal relationship.

Advanced Stats versus Seen 'em Good

At the end of the day, it seems people's view of stats boils down to what they think is going to be more detrimental: the inherent biases of personal opinions, or our limited ability to determine causality via stats. If a person views our limited ability to determine causality via stats as being more detrimental than the inherent biases we all carry when we watch hockey, then they will likely side more-so with the "seen 'em good camp" (and visa-versa for the stats camp). Personally, I side heavily with the stats camp, but I will be the first to acknowledge that if you do the statistical analyses wrong, then your conclusions are going to be invalid and you are going to make bad decisions. Regardless of which camp a person situates themselves in, it is an objective truth that the stats are significantly less biased than personal opinions (in fact the biggest source of bias in stats comes from rink-to-rink variability in how they count each measure, but that bias largely washes out at the end of the day).



Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #696597 is a reply to message #696596 ]
Tue, 27 June 2017 02:47 Go to previous messageGo to next message
George  is currently offline George
Messages: 204
Registered: October 2009

No Cups

Really interesting post, thank you. I’m still learning the different metrics, but my own view has been that we’re still in the early stages of advanced descriptive stats and are too reliant on corsi, which I always thought would be a bit too influenced by the context you are put in. So I found it interesting that you said that corsi is highly correlated with lots of other metrics, like clear zone exits. Maybe I’m not giving corsi enough credit.

I wonder how much scope there is for grouping player type/usage. For instance, I would think it would be hard to get any meaningful measure that compares all out defensive defensemen (e.g., Russell) with all out offensive defensemen (e.g., Schultz). To get something meaningful you would have to make within group comparison of players (e.g., Schultz vs Barrie). But then this leaves open the question as to whether each player type is necessary; I’ve seen a few bloggers suggest that all six of your defensemen should have puck-carrying ability, presumably based on analysing corsi. So is the role of defensive defensemen now becoming obsolete is today’s game, or is this collateral of our reliance on shot-based metrics? I don’t know, I’m just wondering out loud here, but I find all of this really interesting.



Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #696647 is a reply to message #696597 ]
Tue, 27 June 2017 12:39 Go to previous messageGo to next message
ziltoid  is currently offline ziltoid
Messages: 425
Registered: January 2011

No Cups

George wrote on Tue, 27 June 2017 02:47

Really interesting post, thank you. I’m still learning the different metrics, but my own view has been that we’re still in the early stages of advanced descriptive stats and are too reliant on corsi, which I always thought would be a bit too influenced by the context you are put in. So I found it interesting that you said that corsi is highly correlated with lots of other metrics, like clear zone exits. Maybe I’m not giving corsi enough credit.

I wonder how much scope there is for grouping player type/usage. For instance, I would think it would be hard to get any meaningful measure that compares all out defensive defensemen (e.g., Russell) with all out offensive defensemen (e.g., Schultz). To get something meaningful you would have to make within group comparison of players (e.g., Schultz vs Barrie). But then this leaves open the question as to whether each player type is necessary; I’ve seen a few bloggers suggest that all six of your defensemen should have puck-carrying ability, presumably based on analysing corsi. So is the role of defensive defensemen now becoming obsolete is today’s game, or is this collateral of our reliance on shot-based metrics? I don’t know, I’m just wondering out loud here, but I find all of this really interesting.



There are statistical tools to group "things" (in this case player types). Again, I don't want to turn this into a stats lecture, but things like factor analyses and principle component analyses can be used to group similar "playing styles."

At the end of the day the biggest consideration (from my experience) is not individual skill, but the interactive effects between players. Long story short, no one player's performance can be attributed solely to them as an individual; their performance is a byproduct of their statistical interaction with other players on the ice. Thus you want to find players that interact with each other such that they both perform at an elevated level. So defensive d-men are not necessarily obsolete, it is just that they tend not to interact well with other d-men (though, ironically, Klef and Larsson represents a case where a defensive d-man is clearly beneficial to the pair as a whole; though Russell + anyone is a bad, bad, bad idea).



Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #696603 is a reply to message #696596 ]
Tue, 27 June 2017 08:50 Go to previous messageGo to next message
Adam  is currently offline Adam
Messages: 17053
Registered: August 2005
Location: Edmonton, AB

6 Cups

Thanks for this. Interesting read. I'm always fascinated by how many brilliant people from different backgrounds we get in here.

I was re-reading this piece ( http://edmontonjournal.com/sports/hockey/nhl/cult-of-hockey/ analytics-expert-says-kris-russell-of-edmonton-oilers-is-und errated-and-one-of-nhls-top-defensive-d-men) this weekend and I've been puzzling over it.

Werenka's made a believer out of Chiarelli and David Staples, but this part has bothered me:

Quote:

This kind of work comes down to dozens and dozens of decisions and judgements made by TruPerformance game analysts on every player every game. The decisions are subjective, but Werenka says training, guidelines and auditing is in place to make sure that the raters are reliable, that one rater will watch the same game and make almost all the same calls on the same plays and players.

TruPerformance has a set standard of rules analysts apply to judge each play. Auditing is done to make sure the raters are consistent. “Is there a level of variance? Absolutely. We operate on a level of five or eight per cent variance. We would like to keep it within five per cent.”

The key is to get good people to rate, Werenka says. “There are two types of people that are very good. Experienced people that have open minds. They decide through their experience that, you know, being physical or a check is one of the most important things that will happen in a game, and if they’re not open minded to change that then we’re in trouble. So we want experience guys that are open-minded, or guys that have a good basic understanding of the game and have an extreme learning curve, guys that can learn. One of the best compliments we can pay to those guys is, ‘He never asked the same question twice.’ Those are the guys we love.”

When it comes to the issues of rater reliability and subjectivity, Werenka has come to a few conclusions.

“You can say this is subjective, but there’s been a couple phrases we like to use: Consistent subjectivity equals objectivity. And how do you know if that consistent subjectivity is meaningful, if it makes sense, is right? That’s where your correlation to winning comes in.”


There is subjectivity in Corsi analysis as well - what constitutes a shot? And scoring chance metrics are even more so, but I wonder with Werenka's system if there isn't some built in biases. If for example, you were a defensive defenceman when you played and believed that your practice of off the glass and out was a valuable skill - after all, you're relieving pressure and possibly allowing linemates to change - might you rate similar players high?

They're very proud that their analysis has the top players in the game consistently at the top of their charts, but since there's a high level of subjectivity, might part of that be due to marker bias?

I've also been thinking about how this plays in to the hiring of General Managers. I think the mistake that Arizona and Florida made is that the spreadsheet guy is really just a part of your scouting department. Advanced stats is about player analysis, which is only one piece of the puzzle when it comes to being a GM. What does a 26 year old number cruncher know about negotiating contracts or managing the salary cap? I think your best GM has a negotiating background, as he has to handle much of that himself, and is a good leader with great delegation skills and the ability to hire well and trust those he brings in. Having Lamoriello as your GM with Dubas as the assistant is a much better route than say, hiring Tyler Dellow to run your team.

Anyhow, I'm getting off topic, but I appreciated the post on stats, and it will be interesting to see how it evolves in the NHL, and how much we'll actually get to see (since the best analytics guys keep getting hired away).



"Thinking that a bad team's best players are the reason the team is bad is the "Tambellini re-signing Lennart Petrell" of sports opinions." @Woodguy55


#FireLowe #FireChiarelli #FireBobbyNicks #FireKeithGretzky #FireKenHolland #FireTippett

Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #696649 is a reply to message #696603 ]
Tue, 27 June 2017 12:42 Go to previous messageGo to next message
George  is currently offline George
Messages: 204
Registered: October 2009

No Cups

Adam wrote on Tue, 27 June 2017 08:50

Thanks for this. Interesting read. I'm always fascinated by how many brilliant people from different backgrounds we get in here.

I was re-reading this piece ( http://edmontonjournal.com/sports/hockey/nhl/cult-of-hockey/ analytics-expert-says-kris-russell-of-edmonton-oilers-is-und errated-and-one-of-nhls-top-defensive-d-men) this weekend and I've been puzzling over it.


I really like that piece, though I’m probably biased as it speaks to my own concerns about corsi. It does raise a good point, corsi is embraced so much partially due to the fact that it is public data. It is a pretty objective metric that can be scraped from an NHL website. Conversely weighting each play in a game for a pre-determined value requires teams of people and an agreed upon standard.


Quote:

Werenka's made a believer out of Chiarelli and David Staples, but this part has bothered me:

Quote:

This kind of work comes down to dozens and dozens of decisions and judgements made by TruPerformance game analysts on every player every game. The decisions are subjective, but Werenka says training, guidelines and auditing is in place to make sure that the raters are reliable, that one rater will watch the same game and make almost all the same calls on the same plays and players.

TruPerformance has a set standard of rules analysts apply to judge each play. Auditing is done to make sure the raters are consistent. “Is there a level of variance? Absolutely. We operate on a level of five or eight per cent variance. We would like to keep it within five per cent.”

The key is to get good people to rate, Werenka says. “There are two types of people that are very good. Experienced people that have open minds. They decide through their experience that, you know, being physical or a check is one of the most important things that will happen in a game, and if they’re not open minded to change that then we’re in trouble. So we want experience guys that are open-minded, or guys that have a good basic understanding of the game and have an extreme learning curve, guys that can learn. One of the best compliments we can pay to those guys is, ‘He never asked the same question twice.’ Those are the guys we love.”

When it comes to the issues of rater reliability and subjectivity, Werenka has come to a few conclusions.

“You can say this is subjective, but there’s been a couple phrases we like to use: Consistent subjectivity equals objectivity. And how do you know if that consistent subjectivity is meaningful, if it makes sense, is right? That’s where your correlation to winning comes in.”


There is subjectivity in Corsi analysis as well - what constitutes a shot? And scoring chance metrics are even more so, but I wonder with Werenka's system if there isn't some built in biases. If for example, you were a defensive defenceman when you played and believed that your practice of off the glass and out was a valuable skill - after all, you're relieving pressure and possibly allowing linemates to change - might you rate similar players high?


Presumably they have some safe guards in place. They talk about variance levels, I'm sure they rotate guys so it's not the same one or two people marking the same teams, and they probably have an inter-rater reliability measures that are regularly updated. With all of that they /should/ get reliable data. If they're carrying out the analyses properly, they can really partial out a lot of subjectivity.

Quote:

They're very proud that their analysis has the top players in the game consistently at the top of their charts, but since there's a high level of subjectivity, might part of that be due to marker bias?


Of course, these players actually are the best in the league so it seems they're doing something right ;) I actually liked their argument here: the corsi players are obviously way off suggesting that it is a not a strong a predictor as some may believe. Again, if TruPerformance are collecting data using proper protocol, they should be able to hone in on objective values.

For those who haven't read the article, Top 5 Centres as rated by each metric:

TruPerformance: Sidney Crosby, 93%, Evgeni Malkin, 92%, Anze Kopitar, 91%, Patrice Bergeron, 89%, Tyler Seguin/Connor McDavid, 88%.

Corsi: Nick Shore, 61%, Pavel Datsyuk, 58%, Mike Ribeiro, 58%, Mathieu Perreault, 58%, Eric Staal, 57%,




Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #696657 is a reply to message #696649 ]
Tue, 27 June 2017 12:50 Go to previous messageGo to next message
Adam  is currently offline Adam
Messages: 17053
Registered: August 2005
Location: Edmonton, AB

6 Cups

George wrote on Tue, 27 June 2017 12:42

Adam wrote on Tue, 27 June 2017 08:50

Thanks for this. Interesting read. I'm always fascinated by how many brilliant people from different backgrounds we get in here.

I was re-reading this piece ( http://edmontonjournal.com/sports/hockey/nhl/cult-of-hockey/ analytics-expert-says-kris-russell-of-edmonton-oilers-is-und errated-and-one-of-nhls-top-defensive-d-men) this weekend and I've been puzzling over it.


I really like that piece, though I’m probably biased as it speaks to my own concerns about corsi. It does raise a good point, corsi is embraced so much partially due to the fact that it is public data. It is a pretty objective metric that can be scraped from an NHL website. Conversely weighting each play in a game for a pre-determined value requires teams of people and an agreed upon standard.


Quote:

Werenka's made a believer out of Chiarelli and David Staples, but this part has bothered me:

Quote:

This kind of work comes down to dozens and dozens of decisions and judgements made by TruPerformance game analysts on every player every game. The decisions are subjective, but Werenka says training, guidelines and auditing is in place to make sure that the raters are reliable, that one rater will watch the same game and make almost all the same calls on the same plays and players.

TruPerformance has a set standard of rules analysts apply to judge each play. Auditing is done to make sure the raters are consistent. “Is there a level of variance? Absolutely. We operate on a level of five or eight per cent variance. We would like to keep it within five per cent.”

The key is to get good people to rate, Werenka says. “There are two types of people that are very good. Experienced people that have open minds. They decide through their experience that, you know, being physical or a check is one of the most important things that will happen in a game, and if they’re not open minded to change that then we’re in trouble. So we want experience guys that are open-minded, or guys that have a good basic understanding of the game and have an extreme learning curve, guys that can learn. One of the best compliments we can pay to those guys is, ‘He never asked the same question twice.’ Those are the guys we love.”

When it comes to the issues of rater reliability and subjectivity, Werenka has come to a few conclusions.

“You can say this is subjective, but there’s been a couple phrases we like to use: Consistent subjectivity equals objectivity. And how do you know if that consistent subjectivity is meaningful, if it makes sense, is right? That’s where your correlation to winning comes in.”


There is subjectivity in Corsi analysis as well - what constitutes a shot? And scoring chance metrics are even more so, but I wonder with Werenka's system if there isn't some built in biases. If for example, you were a defensive defenceman when you played and believed that your practice of off the glass and out was a valuable skill - after all, you're relieving pressure and possibly allowing linemates to change - might you rate similar players high?


Presumably they have some safe guards in place. They talk about variance levels, I'm sure they rotate guys so it's not the same one or two people marking the same teams, and they probably have an inter-rater reliability measures that are regularly updated. With all of that they /should/ get reliable data. If they're carrying out the analyses properly, they can really partial out a lot of subjectivity.

Quote:

They're very proud that their analysis has the top players in the game consistently at the top of their charts, but since there's a high level of subjectivity, might part of that be due to marker bias?


Of course, these players actually are the best in the league so it seems they're doing something right ;) I actually liked their argument here: the corsi players are obviously way off suggesting that it is a not a strong a predictor as some may believe. Again, if TruPerformance are collecting data using proper protocol, they should be able to hone in on objective values.

For those who haven't read the article, Top 5 Centres as rated by each metric:

TruPerformance: Sidney Crosby, 93%, Evgeni Malkin, 92%, Anze Kopitar, 91%, Patrice Bergeron, 89%, Tyler Seguin/Connor McDavid, 88%.

Corsi: Nick Shore, 61%, Pavel Datsyuk, 58%, Mike Ribeiro, 58%, Mathieu Perreault, 58%, Eric Staal, 57%,




One of my issues with Staples though is that he seems to group all shot-related data in with Corsi. I don't think it's a perfect metric by any means. We saw with Eakins what happens when a coach tries to run a system designed to improve shot metrics without regard to shot quality. It was a horrendous failure because correlation does not equal causation. Good teams shoot more, but just increasing the number of shots doesn't necessarily equate to more goals and more wins...Smytty clappers from the half-boards just surrender possession for a low percentage shot.

Corsi is just the beginning though and most of the stats guys are diving a lot deeper than Corsi now.

I think the work being done by Steve Valiquette has more relevance than the Werenka stuff (in part because it doesn't have a super secret metric that says Russell is a top-ten d-man in the league on a level with Marc-Edouard Vlasic).

http://www.thehockeynews.com/news/article/whats-the-next-dim ension-of-analytics-former-nhl-goalie-steve-valiquette-knows

He's spent a lot of time on what happens just before a shot, and how scoring chances are developed.




"Thinking that a bad team's best players are the reason the team is bad is the "Tambellini re-signing Lennart Petrell" of sports opinions." @Woodguy55


#FireLowe #FireChiarelli #FireBobbyNicks #FireKeithGretzky #FireKenHolland #FireTippett

Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #696679 is a reply to message #696657 ]
Tue, 27 June 2017 14:21 Go to previous messageGo to next message
George  is currently offline George
Messages: 204
Registered: October 2009

No Cups

Quote:

I think the work being done by Steve Valiquette has more relevance than the Werenka stuff (in part because it doesn't have a super secret metric that says Russell is a top-ten d-man in the league on a level with Marc-Edouard Vlasic).


Yeah, that part made me cringe a bit.


Quote:

http://www.thehockeynews.com/news/article/whats-the-next-dim ension-of-analytics-former-nhl-goalie-steve-valiquette-knows

He's spent a lot of time on what happens just before a shot, and how scoring chances are developed.




The Valiquette analysis is really interesting and would seem to have a more direct application to coaching than the other measures used. The Royal Road idea also seems to sum up all of Letestu’s goals this year! (not that Valiquette’s analysis contains 5-on-4 data, but it essentially explains the sequence involved). It also summed up about 50% of the goals scored against the Oilers during the days of Eakins' swarm defense….

I’d love to see a more detailed analysis like this, but focusing on defensive plays. So much of corsi promotes maintaining puck possession, but inhibiting the other team’s possession and re-gaining possession are often over-looked. Can we identify unheralded forwards or defenders that excel at keeping the puck out of their green zone or from crossing the Royal Road. And even better, do they gain possession or at least get the puck out of their zone and relieve pressure on a consistent basis.



Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #696680 is a reply to message #696657 ]
Tue, 27 June 2017 14:35 Go to previous messageGo to next message
Burgeoboy  is currently offline Burgeoboy
Messages: 47
Registered: July 2006
Location: Burgeo, Newfoundland

No Cups

Adam wrote on Tue, 27 June 2017 16:20

George wrote on Tue, 27 June 2017 12:42

Adam wrote on Tue, 27 June 2017 08:50

Thanks for this. Interesting read. I'm always fascinated by how many brilliant people from different backgrounds we get in here.

I was re-reading this piece ( http://edmontonjournal.com/sports/hockey/nhl/cult-of-hockey/ analytics-expert-says-kris-russell-of-edmonton-oilers-is-und errated-and-one-of-nhls-top-defensive-d-men) this weekend and I've been puzzling over it.


I really like that piece, though I’m probably biased as it speaks to my own concerns about corsi. It does raise a good point, corsi is embraced so much partially due to the fact that it is public data. It is a pretty objective metric that can be scraped from an NHL website. Conversely weighting each play in a game for a pre-determined value requires teams of people and an agreed upon standard.


Quote:

Werenka's made a believer out of Chiarelli and David Staples, but this part has bothered me:

Quote:

This kind of work comes down to dozens and dozens of decisions and judgements made by TruPerformance game analysts on every player every game. The decisions are subjective, but Werenka says training, guidelines and auditing is in place to make sure that the raters are reliable, that one rater will watch the same game and make almost all the same calls on the same plays and players.

TruPerformance has a set standard of rules analysts apply to judge each play. Auditing is done to make sure the raters are consistent. “Is there a level of variance? Absolutely. We operate on a level of five or eight per cent variance. We would like to keep it within five per cent.”

The key is to get good people to rate, Werenka says. “There are two types of people that are very good. Experienced people that have open minds. They decide through their experience that, you know, being physical or a check is one of the most important things that will happen in a game, and if they’re not open minded to change that then we’re in trouble. So we want experience guys that are open-minded, or guys that have a good basic understanding of the game and have an extreme learning curve, guys that can learn. One of the best compliments we can pay to those guys is, ‘He never asked the same question twice.’ Those are the guys we love.”

When it comes to the issues of rater reliability and subjectivity, Werenka has come to a few conclusions.

“You can say this is subjective, but there’s been a couple phrases we like to use: Consistent subjectivity equals objectivity. And how do you know if that consistent subjectivity is meaningful, if it makes sense, is right? That’s where your correlation to winning comes in.”


There is subjectivity in Corsi analysis as well - what constitutes a shot? And scoring chance metrics are even more so, but I wonder with Werenka's system if there isn't some built in biases. If for example, you were a defensive defenceman when you played and believed that your practice of off the glass and out was a valuable skill - after all, you're relieving pressure and possibly allowing linemates to change - might you rate similar players high?


Presumably they have some safe guards in place. They talk about variance levels, I'm sure they rotate guys so it's not the same one or two people marking the same teams, and they probably have an inter-rater reliability measures that are regularly updated. With all of that they /should/ get reliable data. If they're carrying out the analyses properly, they can really partial out a lot of subjectivity.

Quote:

They're very proud that their analysis has the top players in the game consistently at the top of their charts, but since there's a high level of subjectivity, might part of that be due to marker bias?


Of course, these players actually are the best in the league so it seems they're doing something right ;) I actually liked their argument here: the corsi players are obviously way off suggesting that it is a not a strong a predictor as some may believe. Again, if TruPerformance are collecting data using proper protocol, they should be able to hone in on objective values.

For those who haven't read the article, Top 5 Centres as rated by each metric:

TruPerformance: Sidney Crosby, 93%, Evgeni Malkin, 92%, Anze Kopitar, 91%, Patrice Bergeron, 89%, Tyler Seguin/Connor McDavid, 88%.

Corsi: Nick Shore, 61%, Pavel Datsyuk, 58%, Mike Ribeiro, 58%, Mathieu Perreault, 58%, Eric Staal, 57%,




One of my issues with Staples though is that he seems to group all shot-related data in with Corsi. I don't think it's a perfect metric by any means. We saw with Eakins what happens when a coach tries to run a system designed to improve shot metrics without regard to shot quality. It was a horrendous failure because correlation does not equal causation. Good teams shoot more, but just increasing the number of shots doesn't necessarily equate to more goals and more wins...Smytty clappers from the half-boards just surrender possession for a low percentage shot.

Corsi is just the beginning though and most of the stats guys are diving a lot deeper than Corsi now.

I think the work being done by Steve Valiquette has more relevance than the Werenka stuff (in part because it doesn't have a super secret metric that says Russell is a top-ten d-man in the league on a level with Marc-Edouard Vlasic).

http://www.thehockeynews.com/news/article/whats-the-next-dim ension-of-analytics-former-nhl-goalie-steve-valiquette-knows

He's spent a lot of time on what happens just before a shot, and how scoring chances are developed.





That article does not say that Kris Russell is a top ten d man, it clearly says he's not even a top pairing guy ( which puts him outside the top 62 at least) it says he's in the next level, with guys like Vlasic , who I was suspirsed to see rated that low. I am by no means an advance stats guy, this is the first time I even heard of top performance, but at first look, it seems to do a much better job then Corsi , based on who it list as the top centers and d men in the League .

One says Kris Russell sucks and shouldn't be in the nhl , it also doesn't rank Crosby or Mcdavid in the top 5 centers . The second says Kris Russell is a second pairing d man and that Crosby and Malkin are the top 2 centers....clearly the first one is better , because there's no way Kriss Russell is a second pairing guy ?

There is no prefect stat/metric yet, but if I had to pick between those two based on thier top rated guys , it's not even close . I am sure top performance as it issues , it sounds subjective, but let's not dismiss it cause it's doesn't say Kriss Russell is the worest hockey player alive. Is it possible these guys that apparently break down each player , play by play and assign a score to each play , might be on to something , rather then just looking at a stat sheet and counting shots for and against ?



Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #696686 is a reply to message #696680 ]
Tue, 27 June 2017 15:01 Go to previous messageGo to next message
Adam  is currently offline Adam
Messages: 17053
Registered: August 2005
Location: Edmonton, AB

6 Cups

Burgeoboy wrote on Tue, 27 June 2017 14:35


That article does not say that Kris Russell is a top ten d man, it clearly says he's not even a top pairing guy ( which puts him outside the top 62 at least) it says he's in the next level, with guys like Vlasic , who I was suspirsed to see rated that low. I am by no means an advance stats guy, this is the first time I even heard of top performance, but at first look, it seems to do a much better job then Corsi , based on who it list as the top centers and d men in the League .

One says Kris Russell sucks and shouldn't be in the nhl , it also doesn't rank Crosby or Mcdavid in the top 5 centers . The second says Kris Russell is a second pairing d man and that Crosby and Malkin are the top 2 centers....clearly the first one is better , because there's no way Kriss Russell is a second pairing guy ?

There is no prefect stat/metric yet, but if I had to pick between those two based on thier top rated guys , it's not even close . I am sure top performance as it issues , it sounds subjective, but let's not dismiss it cause it's doesn't say Kriss Russell is the worest hockey player alive. Is it possible these guys that apparently break down each player , play by play and assign a score to each play , might be on to something , rather then just looking at a stat sheet and counting shots for and against ?


It's Werenka's stats that Chiarelli's touting and he's said a couple of times that in some super secret stat category, Russell is among the very best in the league.

You'll note that I've not defended Corsi as the be-all-end-all of stats. Like most, it tells you something, and there's a value in knowing it. I'm more of a believer in deeper numbers, because I think Corsi doesn't tell enough of the story.

My issues with Russell started when he was with Calgary. He was regularly watching as McDavid blew around him. We've seen why since he came here - his gap control is an issue. He backs off attacking forwards so as to better position himself for shot blocks. If they don't just shoot, that can be an issue. McDavid was extremely fast so giving him extra space is suicide. He just went around Russell again and again.

I believe he gives up the zone too easily and that he doesn't apply adequate pressure on the puck carrier. That's mostly from watching him - not from advanced stats. I don't know what the advanced stat would be that would prove that out.

I also think he's a terrible puck mover. He doesn't connect with his breakout passes, and often he doesn't even look to try...he just fires it up the boards and hopes for the best. If I was going to check some fancy stats based on breakouts, I'd like to know what the number of completed breakout passes are. That's not easily acquired, but I'd be asking for it if I was a GM.

The term 'rigid subjectivity' is what gives me the most pause. They are talking about extraordinary plays, like lunging to break up a two-on-one, but maybe the best players don't have to lunge - maybe they're in the right position and just break it up without leaving their feet (which creates its own issues - one of my problems with the Kris Russell shot-blocks). How much credit are they giving players for diving in front of shots? Is that why Russell scores high?

It's possible that the best players score the best because they're the best, but it's also possible that the model is skewed, and that they've designed it to measure based against the players who they thought were the best.

You are right that there's no perfect system, but I distrust the numbers that Werenka is doing, because Chiarelli talks about controlled zone entries and passing metrics, and that jars with both all the stats I've seen and the eye test.



"Thinking that a bad team's best players are the reason the team is bad is the "Tambellini re-signing Lennart Petrell" of sports opinions." @Woodguy55


#FireLowe #FireChiarelli #FireBobbyNicks #FireKeithGretzky #FireKenHolland #FireTippett

Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #729031 is a reply to message #696686 ]
Fri, 25 January 2019 17:54 Go to previous messageGo to next message
GabbyDugan is currently online GabbyDugan
Messages: 2240
Registered: May 2002
Location: Edmonton, AB

2 Cups

....not sure if we will learn much from player tracking, but this does have a bit of a WOW!!! factor:

https://www.nhl.com/news/nhl-plans-to-deploy-puck-and-player -tracking-technology-in-2019-2020/c-304218820?tid=277549086

"The Puck and Player Tracking system can track pucks at a rate of 2,000 times per second in real-time with inch-level accuracy," Commissioner Bettman said. "We'll instantaneously detect passes, shots, and positioning precisely. It will be equally accurate in tracking players -- their movement, speed, time on ice -- you name it."

What's next? I wonder when a robot will be given a job as an NHL coach.




Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #696658 is a reply to message #696603 ]
Tue, 27 June 2017 12:50 Go to previous messageGo to next message
ziltoid  is currently offline ziltoid
Messages: 425
Registered: January 2011

No Cups

Adam wrote on Tue, 27 June 2017 08:50

Thanks for this. Interesting read. I'm always fascinated by how many brilliant people from different backgrounds we get in here.

I was re-reading this piece ( http://edmontonjournal.com/sports/hockey/nhl/cult-of-hockey/ analytics-expert-says-kris-russell-of-edmonton-oilers-is-und errated-and-one-of-nhls-top-defensive-d-men) this weekend and I've been puzzling over it.

Werenka's made a believer out of Chiarelli and David Staples, but this part has bothered me:

Quote:

This kind of work comes down to dozens and dozens of decisions and judgements made by TruPerformance game analysts on every player every game. The decisions are subjective, but Werenka says training, guidelines and auditing is in place to make sure that the raters are reliable, that one rater will watch the same game and make almost all the same calls on the same plays and players.

TruPerformance has a set standard of rules analysts apply to judge each play. Auditing is done to make sure the raters are consistent. “Is there a level of variance? Absolutely. We operate on a level of five or eight per cent variance. We would like to keep it within five per cent.”

The key is to get good people to rate, Werenka says. “There are two types of people that are very good. Experienced people that have open minds. They decide through their experience that, you know, being physical or a check is one of the most important things that will happen in a game, and if they’re not open minded to change that then we’re in trouble. So we want experience guys that are open-minded, or guys that have a good basic understanding of the game and have an extreme learning curve, guys that can learn. One of the best compliments we can pay to those guys is, ‘He never asked the same question twice.’ Those are the guys we love.”

When it comes to the issues of rater reliability and subjectivity, Werenka has come to a few conclusions.

“You can say this is subjective, but there’s been a couple phrases we like to use: Consistent subjectivity equals objectivity. And how do you know if that consistent subjectivity is meaningful, if it makes sense, is right? That’s where your correlation to winning comes in.”


There is subjectivity in Corsi analysis as well - what constitutes a shot? And scoring chance metrics are even more so, but I wonder with Werenka's system if there isn't some built in biases. If for example, you were a defensive defenceman when you played and believed that your practice of off the glass and out was a valuable skill - after all, you're relieving pressure and possibly allowing linemates to change - might you rate similar players high?

They're very proud that their analysis has the top players in the game consistently at the top of their charts, but since there's a high level of subjectivity, might part of that be due to marker bias?

I've also been thinking about how this plays in to the hiring of General Managers. I think the mistake that Arizona and Florida made is that the spreadsheet guy is really just a part of your scouting department. Advanced stats is about player analysis, which is only one piece of the puzzle when it comes to being a GM. What does a 26 year old number cruncher know about negotiating contracts or managing the salary cap? I think your best GM has a negotiating background, as he has to handle much of that himself, and is a good leader with great delegation skills and the ability to hire well and trust those he brings in. Having Lamoriello as your GM with Dubas as the assistant is a much better route than say, hiring Tyler Dellow to run your team.

Anyhow, I'm getting off topic, but I appreciated the post on stats, and it will be interesting to see how it evolves in the NHL, and how much we'll actually get to see (since the best analytics guys keep getting hired away).


Inter-rater reliability is, as I mentioned, something to think about. But there are ways to assess it and take it into account.

But I think you are spot on with you GM view. GMs need strong negotiating skills, but they also need an open mind re: stats. With Chia, I fear that he does not have an open mind; he has preconceived notions about what metrics he thinks are important, then directs the stats department to assess those metrics (that is the only way I can imagine Russell gets 4x4), which causes the big picture to get obscured. Instead, he should give the stats department a blank slate to analyze the numbers however they see fit, then have them report back to him.



Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #734240 is a reply to message #696658 ]
Wed, 27 March 2019 15:54 Go to previous messageGo to next message
steve.kreys  is currently offline steve.kreys
Messages: 112
Registered: November 2015
Location: Edmonton North

No Cups

Problem with advance stats is it comes down to your knowledge of how they were gathered. I will use a very old player as an example. Larry Murphy. He got ran out of TO and end up going to the Hall of fame with the Wings. What happened. Teams have been using analytics for nearly 40 years, not all teams but the good teams have and there is a reason why the Wings caught so many teams sleeping on so many players. Murphy was 35 and most thought he was done because he was so bad with the leafs. The problem was they were using him wrong and over using him. A few years ago I attended(paid for) a statisticians conference at Gralph and two of the Wings analytics team were there. Both had been working for the wings since the late 1970's. Both Uni grads in mathematics and both played Uni football (as starters--not bench warmers). This took a few people by surprise.

With Murphy, they said what the numbers did not tell you was is how to look not only who was on his line but in net as well and what the coaching style. All cases this hurt Murphy in TO because the coache had told the goalies to come out and handle the puck the moment icing was waived off. Murphy was a PMD with good wheel and a good first pass. Not the best D man in the world. Leafs goalie would pass/shoot the puck up to the red line and most times it got intercepted and Murphy was on his back skates--good skater but not the best D man in the world. There is a reason why the Oilers had Coffey with Huddy and Muni so often. The coaches plan hurt Murphy and they had him mostly with a D man who playing his off wing and was not a sound D man. They did not mention who is was. Leafs D was a waste land that year.

They said, while they crunched all the numbers, they also look at videos and kept track of who was in net and who was his D partner. Murphy at this time had been in the league for nearly 20 years already and had won cups with the Pens. One reason everyone in TO thought he was done.

They said, when he went the the Wings he fitted perfectly with Bowman's coaching style and what they needed from a PMD--they had enough defensive of D men but needed a PMD. Yes the wings had a lot of depth but they knew how is use Murphy and the natural assets he had.

The numbers told them he was done. It was when they saw the videos of the leafs playing, they style the played, the coach they had(Mike Murphy) and who he was playing with, that the wings understood the problem with Murphy with the Leafs. It was not him. it was the team he was on. They said several of the players the picked up for next to nothing were in the same vain. Good players in bad situations.

They also made the point of saying that they had yet to see any on-line analytical website (this was in 2015) that they would consider to be reliable with proper information.

What thing that amused them was when someone asked about Shields and type of equipment. When it came to face shields and a certain player, they asked if the person (who ran an advance stat website and was very proud of it) knew what eye vision ALL nhl players had. Some had better some had worse. And when it came to the player the guy asked about; the guys firmly said 20/20. Both Wings guys said no--he did not have 20/20 vision. The player had been tested by a few NHL teams and they did not say if he 20/10(which is very good) or 20/30 (which is bad). They did say that when talking about visors and other stuff you need to know what the players vision is.

It was funny listening and watching the guy who ran a site tell two guys who worked for an NHL team and each on had a couple of rings--that they were wrong on many different things. One of the sitting profs asked if the guy had ever played hockey at either a pro or uni levels? The answer was no. They then asked him how he got his numbers and he explained-so not from the teams--from the NHL or news papers. The Wings guys pointed out that most teams do not release their stats--all stats come vie the NHL or someone watching the games and thing they know what is going on.

"Getting an honest stat from a team is like getting an honest injury report in the playoffs" not going to happen.

The wings actually did share their stats for awhile only to discover that other teams were nowhere near the wings when it came to analyzing analytical stats.

Most of the wing analytic guys have either retired or went on to other teams.

There were a few bloggers who believed they "understood" analytics. The first question the Wing guys asked was "how many games do you see in person and where do you sit.?" The bloggers said what does that matter?

" We sit first row center ice, nose bleeds" was the response the Wings guys gave. They were asked why did the not sit with the other scouts. "We are not scouts and do no count as scouts and so we do not have to tell anyone we are in the arena; home or away".

One of the evil OBC talked about the EYE TEST. The wings guys said that when it came to how they formed their numbers and understanding them; it was 70/30. 70% straight stats and 30% eye test.

Getting back to Murphy they explained how when they saw the numbers for Murphy with the Leafs it did not match the numbers he had with both the Pens, North Stars and Caps. That is when they saw the games and saw what the problem was. The stats were correct. But the answer would not be found in any analytics. They then showed Murphy's stats for the final ten games with the leafs and then the first ten with the wings. Night and Day and if you looked at the charts you would not believe it was the same player. They wings shaved 3 minuted off his game, Bowman's coaching style gelled with Murphy, the goalie did not do what the leafs goalie always did and the Wings put him a more stay at home D man who got back quicker then Murphy.

When it comes to analytics not all numbers are not correct and the problem can be properly understanding them.

Another example they gave was low % shots vs high % shots. They said they were watching CBC HNIC where one of the panelists was talking about low% vs High % and it was obvious to the wings guys that the guy did not know what he was walking about.

They pointed out that common conceptions of High % vs low % shots are often misunderstood. They then showed about 12 high lights and explained why a few shots that most lay analytic people would lable as being high % were in fact low % due to who the goalie was and what was going on in the play. No two goalies are the same. Goalies maybe similar but that are not the same. Two examples they gave were Patrick Roy and Martin Broadeur. Both goalies had either 20/15 or better vision while other goalies only had 20/20. So using a standard line of low and high % shots would not help. They also talked about some goalies having below average vision 20 feet from them or vice versa. No team releases eye sight tests

Just a few example of how when looking at numbers you need to look beyond that which you see




Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #738112 is a reply to message #734240 ]
Tue, 28 May 2019 14:33 Go to previous messageGo to next message
GabbyDugan is currently online GabbyDugan
Messages: 2240
Registered: May 2002
Location: Edmonton, AB

2 Cups

Apologies for dragging up an old thread as a new era dawns in Oilerville, but the new coach mentioned that he has used advanced statistics since the 1990's. A few random thoughts:

1.If Ziltoid is right (and I believe he is), how can the Oilers modernize and streamline their use of advanced stats? Re-hiring Mudcrutch would be fine, but he is now in Upper Management in the Devils organization. The Oilers have a lot of catching up to do.

2. Can advanced statistics measure how effective coaches and assistant coaches are?

3. Anything in advanced statistics that can measure how effective a goalie is likely to be, both game-by-game and on a seasonal basis? I have no idea why the Oilers brought in Stolerz from Philadelphia, other than he was cheaper than Cam Talbot and Edmonton needed to make cap space to sign Sekera. I also don't understand why Koskinen was given a three-year deal with a NMC, but he's here for the next three seasons so Holland has to find a cheap goalie who can play about half the games each season. Would advanced stats at least help the Oilers find a decent goalie to play in tandem with Koskinen? I don't think the Bakersfield goalies are quite over-ripe or even ripe yet, but the Jets were able to find Brossoit as an effective back up to Hellebuyck after he had an awful season in Edmonton. Why did Brossoit have so much success in Winnipeg after being an epic failure with the Oilers?

4. Can advanced statistics help with who gets called up from Bakersfield? I thought that, for the most part, the guys who came up from Bakersfield during the past season did quite well, but they mostly didn't play very minutes. It would be interesting to see how guys called up compare with their NHL and AHL advanced statistics. (I would think guys like Puljujarvi and Yamamoto would have enough data available from both the AHL and NHL to at least point them in the right direction).

5. Given their salary cap situation, the Oilers aren't going to be able to sign very many (if any) UFA's come July 1, and that's okay. So many of those signings turn out to be costly and not very useful. However, there are always a number of guys who deserve an invitation to training camp without a contract such as Chaisson last season. How much can advanced statistics help the Oilers decide who gets invited?


















a




Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #738113 is a reply to message #738112 ]
Tue, 28 May 2019 14:49 Go to previous messageGo to next message
CrudeRemarks  is currently offline CrudeRemarks
Messages: 3643
Registered: November 2010
Location: Edmonton

3 Cups

GabbyDugan wrote on Tue, 28 May 2019 14:33

Apologies for dragging up an old thread as a new era dawns in Oilerville, but the new coach mentioned that he has used advanced statistics since the 1990's. A few random thoughts:

1.If Ziltoid is right (and I believe he is), how can the Oilers modernize and streamline their use of advanced stats? Re-hiring Mudcrutch would be fine, but he is now in Upper Management in the Devils organization. The Oilers have a lot of catching up to do.

2. Can advanced statistics measure how effective coaches and assistant coaches are?

3. Anything in advanced statistics that can measure how effective a goalie is likely to be, both game-by-game and on a seasonal basis? I have no idea why the Oilers brought in Stolerz from Philadelphia, other than he was cheaper than Cam Talbot and Edmonton needed to make cap space to sign Sekera. I also don't understand why Koskinen was given a three-year deal with a NMC, but he's here for the next three seasons so Holland has to find a cheap goalie who can play about half the games each season. Would advanced stats at least help the Oilers find a decent goalie to play in tandem with Koskinen? I don't think the Bakersfield goalies are quite over-ripe or even ripe yet, but the Jets were able to find Brossoit as an effective back up to Hellebuyck after he had an awful season in Edmonton. Why did Brossoit have so much success in Winnipeg after being an epic failure with the Oilers?

4. Can advanced statistics help with who gets called up from Bakersfield? I thought that, for the most part, the guys who came up from Bakersfield during the past season did quite well, but they mostly didn't play very minutes. It would be interesting to see how guys called up compare with their NHL and AHL advanced statistics. (I would think guys like Puljujarvi and Yamamoto would have enough data available from both the AHL and NHL to at least point them in the right direction).

5. Given their salary cap situation, the Oilers aren't going to be able to sign very many (if any) UFA's come July 1, and that's okay. So many of those signings turn out to be costly and not very useful. However, there are always a number of guys who deserve an invitation to training camp without a contract such as Chaisson last season. How much can advanced statistics help the Oilers decide who gets invited?


















a

My first question would be, if you are acting with full autonomy, do you even need advanced stats?



You can't always get what you want, but if you try sometimes, you just might find, you can get a lottery pick.


Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #738121 is a reply to message #738113 ]
Tue, 28 May 2019 17:32 Go to previous messageGo to next message
MJ  is currently offline MJ
Messages: 2320
Registered: November 1997
Location: Victoria

2 Cups

CrudeRemarks wrote on Tue, 28 May 2019 13:49


My first question would be, if you are acting with full autonomy, do you even need advanced stats?


Always. The more tools in your toolbox, in any job, the better you will be at it. This is not a science, but any barometric measures you can find on the players you are trying to resign, sign from UFA, even draft, are important.



Send a private message to this user  

 Re: Advanced stats, causality, and why numbers matter [message #738133 is a reply to message #738112 ]
Tue, 28 May 2019 23:02 Go to previous message
ziltoid  is currently offline ziltoid
Messages: 425
Registered: January 2011

No Cups

GabbyDugan wrote on Tue, 28 May 2019 14:33


1.If Ziltoid is right (and I believe he is), how can the Oilers modernize and streamline their use of advanced stats? Re-hiring Mudcrutch would be fine, but he is now in Upper Management in the Devils organization. The Oilers have a lot of catching up to do.



Honestly, I doubt they have much in the they way of data, or a data pipeline for that matter. First step is putting that in place, hopefully via an acquisition over building it entirely from scratch (i.e., like how the Flames purchased Puckalytics a couple years back).

Quote:


2. Can advanced statistics measure how effective coaches and assistant coaches are?



Depends on how you define "effective." Most likely yes (e.g., comparing actual vs. expected goals), but leave it to the Oilers to define it in terms of 80s jock sniffing per shiraz.

Quote:


3. Anything in advanced statistics that can measure how effective a goalie is likely to be, both game-by-game and on a seasonal basis? I have no idea why the Oilers brought in Stolerz from Philadelphia, other than he was cheaper than Cam Talbot and Edmonton needed to make cap space to sign Sekera. I also don't understand why Koskinen was given a three-year deal with a NMC, but he's here for the next three seasons so Holland has to find a cheap goalie who can play about half the games each season. Would advanced stats at least help the Oilers find a decent goalie to play in tandem with Koskinen? I don't think the Bakersfield goalies are quite over-ripe or even ripe yet, but the Jets were able to find Brossoit as an effective back up to Hellebuyck after he had an awful season in Edmonton. Why did Brossoit have so much success in Winnipeg after being an epic failure with the Oilers?



I don't follow goalie stuff that much, but I suspect there is a basket of tools... that said, all the above likely just boils down to the OBC being out of touch jack-asses over a lack of analytics.

Quote:


4. Can advanced statistics help with who gets called up from Bakersfield? I thought that, for the most part, the guys who came up from Bakersfield during the past season did quite well, but they mostly didn't play very minutes. It would be interesting to see how guys called up compare with their NHL and AHL advanced statistics. (I would think guys like Puljujarvi and Yamamoto would have enough data available from both the AHL and NHL to at least point them in the right direction).



Again, I suspect the Oil don't have the proper data pipelines in place to do much in this regard. If they did, they certainly could use it to help inform their call-ups, though it also important to balance bringing up the best player vs brining up the person you need information on the most (e.g., player X may be the better candidate for a call-up by the numbers, but player Y is a pending RFA and you need to see more NHL data before you decide whether to qualify or trade or let walk).

The more interesting question is "what prevents the successful transition from the AHL to NHL." I.e., how do you spot the Anton Landers and Ty Ratties of the world. Prospects don't develop in a straight line and require years of investment, so the sooner you can spot a dud and dump them, the better. This is where I'd place most of my minor league modelling efforts (given the data, of course).


Quote:


5. Given their salary cap situation, the Oilers aren't going to be able to sign very many (if any) UFA's come July 1, and that's okay. So many of those signings turn out to be costly and not very useful. However, there are always a number of guys who deserve an invitation to training camp without a contract such as Chaisson last season. How much can advanced statistics help the Oilers decide who gets invited?



$ per <metric> is handy as you can find people who under/over perform their contract that way. The trick is finding the right metrics to use; do you use one, such as goals, or do you use a basket of metrics to paint a more wholistic portrait of the player and their value? I prefer more wholistic metrics / models, but how you build and verify them is hard.

Looking at young-ish (22-25) bubble players from other team's farm based off said metrics / models is another. For example, I wanted to pick up Stefan Noesen on a 2 year deal around 800k in the summer of 2017... he had solid defensive numbers in his limited NHL time, and ended up re-signing with the Devils for 1x660k (after being traded by the Ducks). He then proceeded to put up 13-14-27 in 72GP, but is also why I am scared of re-signing Chiasson as he (Noesen) re-signed again for 1.75MM and put up 3-5-8 in 41 GP. This approach is a bit risky as you are dealing with small sample sizes due to them being bubble players, but sometimes that's the hand you're dealt.

What's most important, and what is most difficult, is finding ways to capture the interactive effects between players. So-called "With or Without You" metrics, or WOWY, are helpful in this regard, but it is still a burgeoning area that lacks a real good solution (so far as I know of.... hard to say what's locked behind NDAs). If you can bring in a player and get a 1.x boost out of their performance simply by playing them with the right people, then you are going to get more bang for your buck.




What scares me the most is who's driving the bus as they define what's "good" and what "stuff" the nerds are supposed to quantify and model. I do not trust the OBC to make those definitions.



Send a private message to this user  

 
Previous Topic:Summer Targets (Oilers)
Next Topic:Coaching Changes
Oilers NHL Minors Speculation For Sale 


Copyright © OilFans.com 1996-2021.
All content is property of OilFans.com and cannot be used without expressed, written consent from this site.
Questions, comments and suggestions can be directed to oilfans@OilFans.com
Privacy Statement


Hosted by LogicalHosting.ca