I did not intend to write this. Especially after Part 2 ended up about twice as long as I initially intended. But there was an excellent point/question raised in the comments by SuckaMD, and it's important enough that once I thought about how to answer it both the length/depth and the importance of the point merited a third part.
To start, I'm just going to copy the comment over in its substantive entirety:
I wonder if maybe this is something of a self-fulfilling prophecy. You found two successful Jays relievers of late, identified some commonalities and then found that Cecil is similar to that as well.
However, there are maybe (possibly? likely?) other second-to-fourth round college pitcher whose numbers worsened slightly over their minor league career, couldn’t start in MLB, were subsequently tried in the bullpen but then failed. Intuitively, this would seem like a fairly common career path for once-touted-but-then-flamed-out pitching prospects. Do you have reason to believe that Cecil is more likely to follow the Downs/Janssen path to success than the hypothetical road to failure I outlined.
I realize the crux of the article was to point out that Cecil’s failings as a starter don’t preclude him from realizing excellent-to-elite value as a reliever. But given his career path, what do you estimate are his chances of coming close to Downs/Janssen versus flaming out? 10%, 50%, 85%?
I would characterize it not as a self-fulfilling prophecy, but rather as selection bias; this does not at all diminish the validity of the underlying point. I worried about this very point as I was gathering data and writing the first two points: that I was taking two success stories after the fact, looking for and finding commonalities (while perhaps diminishing important differences or contrary points), essentially confirmation bias. I tried to avoid this by rigourously establishing a profile, and including as many elements as possible like minor league performance, major league performance, and Pitchf/x data to avoid omitting contrary information (why Part 2 ended up so long). However, this in itself presents selection bias issues.
Ultimately, I did not address this in Part 2, because it was already way too long, and also because I couldn't thing of a manageable, comprehensive way of testing and looking for false positives (players who fit the profile, but who did not succeed). On further reflection, there was enough data available to establish a decent screen for a group of players who would fit a number of elements of the profile, and this could be used as a baseline to assess conversion success.
BUILDING A COMPARISON GROUP
One part of the profile was being a college pitcher drafted fairly early. This was a convenient starting point since Baseball-Reference has a wealth of accessible draft information. I started with all players drafted in rounds 1 to 5 from 2000-04. That's a bit wider than the initial profile, but it's better to cast a wide net here and cut some names later can be overly restrictive. I choose 2004 because college draftees from that era should be fully established in their careers, and worked backwards from there. I didn't want too many years or the disparities between the length of career could be too big for comparative purposes.
That gave 806 draftees, actually a fewer players as there's some repeats from players who don't sign and get redrafted. B-R helpfully indicates the type of institution from which the player was drafted, so I dumped all high school and juco draftees, leaving just 4 year institutions. That cut the list down to about 400. Dumping hitters brought it down to about 225 college draftee pitchers.
Of those 225 college pitchers, only about 100 made it to MLB. I cross-referenced this against a list from Fangraphs of pitchers who accumulated at least 50 innings from 2000-12 (any less is no more than a cup of coffee, and not going to remotely fit the profile), and got a final list of 72. Basically, only one in three college pitchers drafted in the top 5 rounds will even pitch 50 MLB innings, which surprised with just how low it was (although it probably should not have).
Another important part of the profile was having made it to the majors as a starter, and not succeeding in this role before being converted to relief. I charted the remaining players according to innings started vs. innings relieved:
Though Scott Downs is not part of this data set, for reference I added him, denoting him with the red star. Note the difference in the scales of the two axis - along the horizontal axis, there are a bunch of pitchers who have accumulated a lot of starting innings and few if any relief innings, with a fairly significantly break between approximately 400 and 500 innings pitched. This group of 21 includes pitchers such as: Haren, Lee, Verlander, Weaver, Blanton, Weaver, Francis, Saunders, Guthrie, Marcum, Prior, Niemann, and our friend & controversial acquiisition J.A. Happ. Basically, all pitchers who have stuck in big league rotation as inning eating pitchers or better, so they do not fit the profile or belong in the comparison set. Even the one with the most relief innings, C.J. Wilson, is an established starter, so there's really reason to keep any of them in. This leaves a group of 51 that can be examined in sharper contrast:
At this point, there are some very clear clusters, which I've separated into six groups that I think broadly represent different types with this comparison groups. I want to go through each one briefly to look at suitability for inclusion in the comparison group. Skip over this section to the summary if details aren't your thing.
Group A: Just as there was a large number of starting pitchers along the horizontal axis, there's a large cluster along the vertical axis of 15 pitchers who never started, and another four who started less than 25 innings and have relieved for more than 100. It's an interesting mix of top tier and next tier names such as Papelbon, Cordero, Huston Street, Aardsma, Qualls, Crain; as well as more obscure names such as Chris Bootcheck, Royce Ring, Jon Switzer, Taylor Tankersley, etc. Another important part of the profile was pitchers who were good enough to be starters on the way up in the minors, and got real opportunities to start & stick in the MLB rotations. Having spotchecked a number of these guys, most if not all were relievers in the upper minors. When this happens, it's generally for one of two reasons: one or two excellent pitches but lacking a third (classic closer profile), or lacking the stuff to even have a shot at starting. Neither fits the Janssen/Downs mold we're going for, so the 19 in this group don't belong in the comp set.
Group B: Consists of Janssen, Lance Cormier, Aaron Heilman, J.P. Howell, Brian Tallet (and Downs if he were included as well). All have logged significant starting innings, but ultimately more relief innings. Slam dunk inclusions.
Group C: Bryan Bullington (1st overall), Mike Gosling (66th), Justin Wayne (5th), Dan Meyer (34th,1s). Initially I wasn't going to include 1st rounders since they're of a much higher general pedigree, but opted to include and remove later as the situation dictated. All four started on the way up, and started in the initial MLB stint(s). Meyer and Gosling belong on balance even though they didn't accumulate many innings in either role. Bullington and Wayne are judgment calls depending on how literally you stick to the profile and neither pitched much.
Group D: All these pitchers logged 75 to 150 starting innings, but not as many relief innings. Most likely, these are where the failed convertees lie; however of the 13, eight have less than 26 innings as a reliever. In cases such as Zach Jackson, he made a few relief appearances but there was never a serious commitment to converting him. In other cases, it may have been injury, but even measuring performance in a sample that small is an issue. The other five profile as good comparables.
Group E: Dewon Brazelton (3rd overall) and Phil Humber (3rd) don't fit at all, they're out. Brad Hennessey (21st), Glen Perkins (22nd), other than being being somewhat higher picks, fit the profile quite well.
Group F: These six got more starting innings than either Janssen or Downs, but a more similar number to Cecil so it makes sense to include any borderline cases. However, only one even remotely approaches that threshold, Kirk Saarloos, and he went back to starting after relieving. Jeremy Sowers (6th overall) made only one long relief appearance, so he's out. Rich Hill's relief conversion came after labrum surgery and Tommy John. Since injury risk is intrinsic to pitching and it's not clear inferior stuff forced him to the pen, I don't think including him makes sense. Tim Stauffer is similar, his relief innings came after labrum surgery ans has since returned. Ross Ohlendorf only relieved breaking into the majors, has been a starter since. Todd Wellemeyer was the opposite, reliever initially then got into the rotation.
Summary: Of the 51 candidates who weren't successful starters, 34 are not suitable for a comparison groups, 14 fit the profile very well, and three are borderline inclusion candidates.
DETERMINING SUCCESS OR FAILURE
In addition to splitting out inning by SP/RP, Fangraphs also splits out runs above replacement (RAR) by role; this used to be on each player's page, I forgot it could be accessed by customizing leaderboards. This can be coverted to RAR/9 to measure success in each role. One issue is that fWAR/fRAR valuation excludes the results of balls in play and strand rate variation. For marginal starting pitchers, this potentially overstates their performance as they will be quite hitable. Likewise, so really good relievers, this likely understates their performance since they can maintain a lower BABIP. This must be considered in assessing performance, and so I also looked how the.
Below I have made a list of the players in the comparison group, along with IP and RAR/9 as a starter and reliever, as well as total relief RAR. I then classified each player's conversion success as on a "Yes", "Mixed", "No" basis on two dimensions: first, whether there was improvement; second whether the pitcher was an effective reliever. A "Yes" required both evaluations be positive, a "No" both dimensions negative
J.P. Howell does not look like a success, but as starter he had an ERA of 6.34 and as a reliever it's 3.08. Clearly, he's much improved and an effective reliever. On the mixed guys, mileage will vary depending on the person. Lance Cormier was much improved (also on an ERA basis), but he was still a replacement type reliever. Success? Not? He did pitch 350 IP out of the bullpen, so at some point someone thought he was a decent enough option.
Overall, seven of the 14 best comparison fits have not realized a successful conversion, including a couple notable Blue Jays in Brian Tallet and David Purcey. I would categorize four (around 30%) as being definitively successful, with another three being arguable cases. Of the three additional borderline cases, none would count as successful conversions. I would hesitate to read much into these cases, as they don't fit the profile well.
In a larger sense, while my enthusiasm about Cecil's potential is somewhat tempered by these results, I hesitate to read too much into it. There are several elements of the profile not addressed in this analysis, including platoon splits and whether they changed, raw stuff and minor league progression. Additionally, I'm not familiar with several of the players in the comp set at a deep enough level to know whether or not they truly fit as comparables, or just superficially appearto be. I would note that by a cursory look at the four successful convertees, none of them seemed to have great raw stuff as starters, and it ticked up in relief.
Overall, it's not a very large sample with only three identified comparable players per year covered. To me, this suggests that while a top-down analysis can give some idea about the probability, it's probably more important to look more deeply at the individual players. On that basis, as discussed in Part 2, I still hope to see Cecil get a good chance.