Introductory Note: This was initially titled "Is Travis Snider Poised for a Breakout Year", as most of the data analysis and write-up that follows was done before he was sent down. I was waiting for Snider to get enough ABs to apply the analysis to him, and almost coincident with that happening he was optioned last Sunday. I decided to wait and let that settle in before writing this. While this investigation was partially inspired by Travis Snider's performance this March, the more significant part is evaluating a claim that has been advanced about Spring Training performance being a predictor of regular season success.
Snider, Rasmus and the 2012 Outfield
In many ways, Colby Rasmus and Travis Snider have a lot in common. Both were left-handed outfielders drafted in the first round out of high school (2005 and 2006, respectively). Both rose relatively quickly through the minors and were on the 2009 Opening Day rosters of their respective teams. Both have had success at the MLB level, both have had well-documented struggles, and both have been in the doghouse with past managers (Tony LaRussa and Cito Gaston, respectively). In Spring Training this year, Snider hit .271/.340/.635 in 48 AB with a 17/5 K/BB ratio, whereas Rasmus is hitting .173/.254/.212 in 52 AB with a 16/6 K/BB ratio. Yet, it was Snider who was the underdog in a battle for the left field job and who was optioned to the minors, while Rasmus was virtually guaranteed the centre field job. How could this be? The answer is pretty simple of course - 50 AB is just not a large enough sample size to make definitive evaluations about a baseball players, especially in light of a much larger body of work over the past three years in which Rasmus was full-time starter who was a league average hitter whereas Snider struggled to maintain consistent mechanics and make adjustments against off-speed pitches, and was bounced between AAA and MLB while battling some injuries. It's a classic example what happens in small sample sizes, and is further compromised by the large variation in competition on the field during the spring. However, notwithstanding this, is there anything Spring Training stats can tell us about what a player is likely to do once April rolls around?
Is There a Signal for Breakout Seasons?
This offseason I became aware of a study done by John Dewan (owner of Baseball Info Solutions and author of The Fielding Bible series), which claimed that players who slug 200 points above their career slugging percentage during the spring (minimum 40 AB in spring and 200 MLB career AB) have a 60% chance of "taking their game up a notch" during the regular season. I was struck by this, both because of the implications it could have and the fact that it is not really widely discussed, relative to what I would expect if it was such a good indication of positive future results. After all, we're always looking for good predictors of the future. I kept this in the back of mind as Spring Training approached and unfolded, and I was intrigued to see Snider's hot start put him on pace to meet this criteria. Particularly so in light of his potential (rated the #6 prospect in baseball before 2009) and the fact that he was battling for the starting LF job. In the end, Snider cooled down but met the criteria with a .635 SLG% (career .423) in 48 AB (career 799), although it's possible he would have fallen below the .200 mark had he continued to get playing time. So did the Jays make a mistake in sending him down? Dewan's test would seem to indicate so, but I remained somewhat skeptical and set out to examine exactly how well indicator predicted predicted regular season success.
Some Methodological Notes
For the last 7 seasons, Dewan has published a Stat of the Week column near the end of Spring Training with a list of players who meet his critera (links: 2011, 2010, 2009, 2008, 2007, 2006 and 2005). It seems that these lists don't always consider the last few games, which could cause some problems with players at the margins (those close to the .200 line or who might have been a couple AB shy and subsequently got over). Nevertheless, I'm going to work with his lists and the numbers presented there for simplicity. In total, there were 226 players identified over the 7 years, or roughly one per team per year. Two of them (Jason Dubois and Reggie Taylor in 2006) did not log any regular season MLB playing time in either 2005 or 2006, so I have excluded them for a total of 224.
One significant issue at the outset is that none of the above Stat of the Week columns describe how players are measured in terms of determining if they see an improvement. The 2011 column claimed that "two-thirds of hitters who had spring slugging percentages at least .200 higher than their career total went on to best their career average that season". Assuming that the career average meant SLG%, the problem I have is that slugging is only one way of measuring production, and an incomplete one. Additionally, the language used to describe a player achieving regular season success varies and seems to hint at different things (2005 and 2006: "take a step forward"; 2007: "a step forward...might be headed for above-average seasons"; 2008: "performed better than their career average"; 2009-10: "take their game up a notch"; 2010-11: "breakout").Some of these descriptions imply improvement versus previous seasons, others imply looking at only the season itself.
In light of the above, I decided on a flexible approach by measuring each player on three dimensions - whether he "broke out", whether he had an above average season, and whether he had a great season. I tried to measure an offensive breakout by comparing a player's wRC+ to the previous season, as well as to his career average (to ease data collection, I used the ten seasons prior as a proxy for career). I was looking for a number in excess of both, and also considered the number of plate appearances since some may have have minimal playing time and that shouldn't be considered a breakout. Since the designation of a breakout is somewhat subjective, I had three possible options: yes (Y) for a clear breakout, possible (P) for a borderline case, and no (N). I used fWAR (which includes defence and positional adjustments) to assess an above average season (>2.5 fWAR) and great season (>4 fWAR), and again used the three options Y, P and N. Since WAR is a counting stat, I also considered the PA and where appropriate did some prorating. Above all, the goal was just to be consistent, since this is looking at performance using summary statistics from 30,000 feet rather than looking in depth at 224 player seasons.
Looking at Blue Jays
In total, 9 Blue Jays have met Dewan's criteria, including Jose Bautista twice. However, Bautista has basically been the only one to break out or put up a good season, as in general it's been a dismal indicator of success for Blue Jay players. Columns 1, 2 and 3 are my assessments for breakout, good seasons and great seasons respectively.
Below is a summary table showing the breakdown for how many players on a yearly basis I classified in each category (Y, P, N for yes, potentially/borderline and no respectively) along each dimension that I evaluated, along with the percentage meeting the criteria (Y%) or at least coming close (Y+P%) :
In total, 18% of the players identified by Dewan experienced what I considered a clear breakout season, and an additional 11% had season that could possibly be called breakout seasons, for a total of 29%. Conversely, 71% of players identified did not have a breakout season. In terms of those players having good seasons (greater than 2.5 fWAR), around 38% of players clearly had good years and another 10% had borderline good years, meaning almost half of the identified players had a good season. Most of the potentially good seasons were players who didn't get to 2.5 fWAR, but who had significant playing time and were on pace for 2.5 fWAR/650 PA or greater, maybe due to injuries or platoons - so I think the borderline category works well here. Finally, 20-25% of the players had great seasons (4 fWAR or greater).
Finally, since it's not entirely clear to me whether Dewan was saying that these players will improve from year to year or just have good seasons, I decided to combine these two together. After all, a team would be happy to get either a breakout season or a good season, both are positive outcomes. This is the last set of columns above, "Breakout or Good". For the "Y" category, I counted a player if he had either a clearly good season or a clear breakout season, regardless of what the other was (and didn't double count). 42% of players Dewan identified had either a clear breakout season or a good season. I then went further and included players who didn't clearly meet either of those two, but who had a borderline positive, which added 11% to bring the total to 53%. Since 48% of players had good seasons or borderline good season, that means very few players has breakouts without being good players, which makes sense intuitively. Conversely, 47% of players did not have breakout seasons or good seasons.
Based on the above results, it seems like Dewan's test is a fairly good indication of at least a moderately positive upcoming season with a roughly 50% success rate, which is pretty good for a sport with a lot of below average and replacement level players. However, this doesn't tell us how valuable Dewan's test is, since there's no baseline against which to evaluate it - if we looked at all players with 40+ spring AB and 200+ career AB, how many of them would be similarly successful? After all, while plenty of Spring Training AB go to non-roster players, in the end playing time leaders are generally projected starters who are better players. Ideally, I'd go back and check all players who met that criteria, but that would be very labour intensive. Instead, I created a group of 60 randomly chosen players. For 2010 and 2011, I chose the player for each team with the 7th most spring AB for his team (7th most to make sure the player would have about 40 AB, any number higher than that could have done as well). If there was a tie, I used (AB+BB) as the first tiebreaker and then alphabetization. For a few teams, this player did not have 200 career AB, so I went with a player who was tied where applicable, or the player with the 8th highest AB. Below are the same summary results for those players:
As can be seen, this set of comparison players had a slightly lower overall breakout rate at around 20% rather than 28%, although the number of clear breakouts was similar. If the true likelihood of experiencing a breakout season was 28%, the chance of getting 20% or fewer players meeting the criteria in a sample of 60 through chance is 11%. Likewise, if the true likelihood of a breakout was 20% identified in the comparison set, the statistical likelihood of observing a 28% randomly is nil, which means we can be reasonably sure that Dewan's success in identifying a higher than average breakout rate is not due to random variation. Further, the number of good seasons was very similar while the number of great season was higher among the comparison set (which could be explained by the fact that players capable of putting up great seasons are almost invariably starters and get plenty of spring AB, meaning they're likelier to be in the team's top 10; whereas Dewan's test would capture more lesser players battling for jobs and getting significant time to make their case). Similarly, the overall rates of having either a breakout season or a good season are similar. So it appears that Dewan's test actually does identify players with a greater than average shot at breaking out breakout rate, though it really doesn't tell us if players are more likely to have a good season.
Does an Extremely Good Spring Increase these Likelihoods?
So far, the test for inclusion has been a binary one once the AB criteria is met - either the player had a slugging differential of 200 points of more, or he didn't. However, there is little practical difference between a player who has a differential of .195 and .205, especially in a small sample of AB. Trying to extend the differential backwards would be a lot of work, but I do have the differential for each player, so I can separate them into buckets of increasing SLG% differentials, and see if the success rates change (the buckets would ideally be of equal size, but due to the relatively small sample I wanted to limit the number of buckets to make the comparison meaningful):
The last bucket only has 18 players, but I thought it would be a good idea to separate it since there some truly unique performances there (Andruw Jones slugged 1.175 in 2005 compared to .493 career). But the small sample means some caution is due interpreting though numbers. For example, the rate of those players clearly breaking out is much higher among these players than those in other buckets. Intuitively, this could make sense, but some caution is due in making the inference that these players are more likely to success. More broadly, the chance of experiencing some sort of breakout increases with the differential. The chance of a good season is roughly similar from bucket to bucket (the decrease in the last bucket seems to be a small sample size issue), as is the rate of a great season.
FInal Thoughts and Conclusions
One last thing that worried me was that the home parks could be affecting this, since these are raw unadjusted numbers and there was a some large differences in from team to team in how many players were included from each team:
I remain concerned about this, but am somewhat reassured that it could just be random variation: 22 of 30 teams (73%) fall within 1 standard deviation, which is a little high but reasonable; and all but Kansas City within two (97%).
One other thought is that without an adjustment for the number of AB that the player has in spring (and to a lesser extent in his career), there might be bit of a selection bias in terms of which players are identified. This is significant because we are looking at players who are outperforming, and over more AB such players will tend to regression towards a lower mean. So a player who has a SLG% differential of .180 over 60 AB might actually have been better than a player who had a SLG% differential of .210 over 40 AB. Given that the total sample was 224 players, this probably isn't a huge issue, but something to think about if one were to try and refine the methodology from scratch.
Overall, it seems that looking at a player's spring-career SLG% differential gives a somewhat higher expectation of that player experiencing a breakout, but given the relatively small difference I'm not sure it's something that should outweigh other factors. Applied to the Travis Snider-Eric Thames battle for the left field job, other factors that ultimately gave Thames the edge (better 2011 performance, getting Snider more reps with his new mechanics and learning to better recognize off-speed pitches) probably trump the fact that Snider's Spring Training performance indicated a better than average shot at breaking out in 2012 and tapping into his potential. Finally, here are the top 5 qualifying Blue Jays other than Snider in terms of their SLG% differential and one bonus, guys who might be worth keeping an eye on (some are currently shy of 40 AB but will get there with 3 games left):
1) Edwin Encarnacion: .453 career SLG%, .610 spring SLG% = .157 differential
2) Jose Bautista: .481 career SLG%, .635 spring SLG% = .154 differential
3) Adam Lind: .466 career SLG%, .595 spring SLG% = .129 differential
4) J.P. Arencibia: .432 career SLG%, .529 spring SLG% = .098 differential
5) Eric Thames: .456 career SLG%, .533 spring SLG% = .077 differential
Bonus: Brett Lawrie*: .580 career SLG%, .821 spring SLG% = .241 differential
*Lawrie only has 150 career AB, so technically he doesn't qualify to be included. On the other hand, more AB would have likely resulted in the SLG% regressing downwards, increasing the gap. Considering the level of play he put up in 6 weeks in 2011, it's crazy to imagine he could be an above average candidate for a breakout, but it's certainly interesting.