Last month, Fangraphs ran a piece by Jeff Sullivan about Michael Saunders having, by one measure, the most unclutch season going back to 1974 when data for the "clutch" statistic begins. Truth be told, I don't really understand the stat (not the mechanical calculations, which are straightforward, but what the results actually mean in baseball terms) but fortunately there's a more conventional way to see this. According to Fangraphs, in 224 low leverage PA, Saunders has a world-beating 181 wRC+; in 219 medium leverage PA he has an average 99 wRC+; in 54 PA high leverage PA he has a putrid 10 wRC+
Since then, whenever Saunders comes up in an important situation, it's been common to see a reference to this and how Saunders can't hit or sucks in high leverage (or some such variant). Yet for all the verbiage in that piece, there is no meaningful analysis as to the significance of this observation, or really even attempt at it other than just some bland bromides and trite platitudes in the last paragraph that amount to a big shoulder shrug.
But that is really the most important part, especially if many are going to presuppose failure every time he comes up in an important situation on the basis of this observation, as has been occurring. Saunders has been horribly unclutch this year, it does not mean he's actually unclutch just as Russell Martin being an awful hitter for the first 50 days of the season did not mean he was an awful hitter. So since Fangraphs couldn't be bothered to do the serious analysis, that's what I want to delve into today: what is the significance of these results, and what can be said about Saunders being unclutch or not.
The 2016 data
Fangraphs uses a cutoff of 2.00 for high leverage, Saunders has 54 such plate appearances. At the outset, it's worth pointing out just how small a sample it is at barely 10% of his plate appearances. Offensively, production is driven by basically four things: strikeout rate, walk rate, BABIP and power (ISO). To make things easy from a statistical perspective, I'll use single rate (1B/PA) as a proxy for BABIP, with extra base rate (2B+3B+HR) as a proxy for ISO.
On all of these dimensions, Saunders has negative differences in high leverage vs. otherwise: 37% vs. 27% strikeout rate, 7% vs. 10% walk rate, 12.6% vs. 9.3% singles rate, and 4% vs. 12% extra base rate.
A simple statistic test can tell us how likely these differences would be likely to occur randomly. It's like if you had a bag of jelly beans, equal amounts of five colours, and you picked 10 jelly beans randomly, what's the chance of picking five of a specific colour randomly. If it was extremely unlikely, it might be possible to infer that there was something that made picking them non-random, maybe the shape or feel.
The smallest differences, in walk rate and singles rate, turn out to not be even close to statistical significance in such a small sample size. Strikeout rate and power are more a little more ambiguous. They approach statistical significance (t stat of 1.61 and 1.86 respectively), but fall a little short. And that's before considering that overall, hitters do a little worse in high leverage than overall (93 wRC+ vs. 101 otherwise). And that underperformance largely comes from more strikeouts and less power (with walk rate higher). Adjusting for these generic differences would move even further away from statistical significance.
That said, this is considering all elements separately. With a couple elements on the edge of significance, it's possible that combined they could be significant. So I ran a simulation using the overall distribution of Saunders' events and using a random number generator to create 100 samples of outcomes over 54 PA. And then repeated that a couple times.
Sure enough, about 2-3 of every 100 of these samples would end up similarly putrid in terms of overall production to Saunders in 2016. In other words, there is a low, but non-trivial and real possibility that a hitter having a good season like Saunders could randomly have such a bad season in high leverage by chance alone.
Before moving onto his carer numbers, one final point on the 2016 numbers. Above, I mentioned that the cutoff for high leverage is 2.00, which is a minimum of double the average at-bat. This is a very high threshold, especially compared to the 0.85 cutoff used for low leverage, which restricts the sample. If the cutoff is loosened to 1.50, which is still very important at-bats, there's another 51 PA that can roughly double our sample. Saunders has 8 singles, 2 doubles, 2 home runs, 3 walks and 17 strikeouts, which isn't great but better than 54 high leverage PA.
If we use this enlarged sample and repeat the significance tests, there's some interesting results. The singles rate is essentially completely insignificant. The walk rate differential strengths, but short of significance (t-stat=1.30). But the power threshold does hit significance (t-stat=2.02) and strikeouts comes very close (t=1.92). Given these stronger results, I went ahead and made a rough adjustment for the expected differences discussed above across the league. This weakened the significance to slightly below average.
The career data
There's a similar trend here. In 226 career high leverage PA, Saunders has a 63 wRC+, compared to 103 wRC+ in low leverage and 102 wRC+ in medium leverage. Awful in high leverage, slightly above average otherwise.
Breaking down the components, there's basically no difference in singles rate (12.9% vs. 12.8%) or walk rate (9.2% vs. 9.0%). It's all about strikeouts and power. Saunders strikes out a third of the time in high leverage, a quarter of the time otherwise. He hits for extra bases 5% of the time in high leverage, almost 9% otherwise.
Both of these pass statistical significance, and even adjusting for the generic league gap in high leverage, the strikeouts still end up significant, though the power numbers fall under the threshold.
Even taking out 2016 and looking at his career before (not to cherry pick, but since 2016 accounts for over 20% of his career, if it was an outlier it could skew the data), the strikeout difference is still a robust 6% (31% compared to 25%) and the difference in power was pretty big too (7.8% extra bases compared to 5.2%). These are issues that pre-exist 2016.
Yes, Saunders is probably unclutch
On the balance of the evidence, it's clear that Saunders has drastically underperformed in high leverage situations when it comes to striking out and hitting for power, and to an extent that it is unlikely to have been simply bad luck.
The obvious follow-up question is why this is the case. Is he doing something differently? Being pitched differently in a way he can't adjust (if so, why don't pitchers pitch him like this all the time). I may revisit this at some point, particularly since it impacts, or should impact, his value in free agency.