During a game thread last week, the term pure quality start came up. Since I plan to use PQS as an important indicator of how pitchers have been doing this year (and consequently how well they'll do going forward), I'd like to do a quick post just to explain what it is, how to score it and why I think it is worthwhile.
The PQS is a metric Ron Shandler, of Baseball Forecaster, developed to measure a pitcher's overall effectiveness in a game. While the traditional "Quality Start" has a very arbitrary, blanket and luck-driven rule (6 IP, 3ER or 8 IP, 4 ER), PQS seeks to adjust for luck on balls in play. It is scored 0-5, but an easy and convenient way of looking at it is that a zero or 1 is indicative of a "disastrous" start, while a 4 or 5 is indicative of a "dominant" start.
The scoring of PQS is actually pretty simple. There are five basic components, each of which, if met, is worth one point. They are as follows:
1. Minimum of 6 innings. This measures stamina.
2. Hits can be no greater than the number of innings pitched (7 hits in 7 innings meets standard, 7 hits in 6 innings does not). This measures hit suppression.
3. Strikeouts plus two must be equal or greater to number of IP (5 K in 7 IP meets standard, 5 K in 8 IP does not). This measures dominance.
4. Strikeout to Walk ration must be 2:1, regardless of IP (4 K, 2 BB meets standard, 10 K, 6 BB does not). This measures control.
5. Less than two homeruns yielded, again regardless of IP. This measures his ability to keep the ball in the park.
There is one final caveat, which is that a pitcher who does not complete at least five innings receives an automatic score of zero.
One relatively frustrating aspect to PQS is that it really does not reward pitchers for going deeper than six innings (and frequently punishes them, as K-rate tends to decrease as pitchers tire and two HR in 9 IP is seen as just as bad as two in 6 IP). On the other hand, the regular Quality Start has the same problem.
Nonetheless, while PQS has its drawbacks, I am still of the opinion that it is a far superior method of evaluating past (and predicting future) pitcher performance because it scores a pitcher based mainly on factors within his control (strikeouts, walks and homeruns). Shandler extrapolates an ERA prediction based on PQS scoring and start Dominance and Disaster rates. He claims that this "qERA" actually predicts a pitcher's ERA in the following year better than his actual ERA, which does not shock me because PQS -- at its heart -- seeks to measure a pitcher's skills, while ERA takes defense and luck into account. In my opinion, PQS is a much better gauge of a performance than Quality Start.
Bill James has also developed the "Game Score" -- a metric whose scoring system is probably superior to PQS in analyzing a pitcher's starts (it differs most in that it takes runs into account) -- but the system for calculating a start's Game Score is too cumbersome for quick calculation at a glance. PQS is convenient because it can be calculated in seconds by looking simply at the boxscore. A practical example of how PQS and Game Score can be used for Roy Halladay's Retroactive 2008 Cy Young Award Campaign (yes, we're still bitter about it!) follows below. It is also the impetus for the title of this post, coming from the song "Electioneering" by Radiohead. By the way, I had no idea what the end result of this analysis would yield, as I'd never run these numbers before.
Roy Halladay's 2008 PQS Logs
4, 4, 4, 3, 3, 4, 5, 5, 4, 4, 5, 4, 3, 3, 5, 3, 4, 5, 5, 4, 5, 5, 5, 4, 3, 5, 5, 2, 5, 4, 5, 3, 4
Halladay's 2008 PQS mean was 4.12, so Doc's average start was dominant. He had no starts that would be classified as disastrous, and scored a 2 just once. He managed a Dominance rate of 76% and a Disaster rate of 0. He achieved a 5 (the highest score) 13 times.
Roy Halladay's 2008 Game Score Logs
52, 55, 76, 54, 47, 75, 71, 50, 48, 80, 70, 54, 54, 41, 70, 39, 85, 63, 90, 39, 64, 63, 77, 48, 72, 62, 59, 56, 62, 41, 60, 48, 71
The mean of Halladay's 2008 Game Scores is 60.48. When you consider that 50 is considered a "quality start," a mean of 60 is quite good. Halladay had just eight starts of 33 in which he did not reach a score of 50 (and four of those times he scored at least 47, which is obviously quite close).
If you followed the link, you would see that a game score of 100 is an All-Time great game and a 90 is probably one of the best pitching performances of the season. In the case of Halladay's 90, he pitched a 2-Hit, 8 K, 1 BB shutout against the Yankees. I remember watching it in the bar with my Jays jersey on and everyone was hating on me. In case you were wondering, Halladay has scored at least a 90 four times, as well as scoring an 89 in his second career start in his near no-walk no-hitter, (darn you Bobby Higginson and Craig Grebeck!).
In comparison, 2008 Cy Young Award Winner Cliff Lee's numbers for 2008, which were also quite good:
4, 5, 5, 5, 3, 5, 5, 1, 4, 3, 3, 3, 4, 4, 5, 4, 5, 5, 3, 5, 2, 2, 4, 4, 5, 4, 4, 4, 2, 3, 4
The mean of these scores is 3.84, which is not quite as good as Doc's. He had a Dominance rate of 68% and a Disaster rate of 3%. He also scored 2 or lower three times, whereas Halladay only scored one 2. He scored 5 ten times. In sum, Lee scored fewer fives, and his mean PQS and Dominance rate were lower. He also had a higher Disaster rate and scored more twos. Advantage: Doc
Cliff Lee's 2008 Game Scores
67, 82, 85, 90, 47, 72, 76, 28, 56, 38, 31, 50, 55, 64, 80, 68, 53, 68, 61, 72, 28, 53, 67, 67, 60, 62, 81, 64, 46, 41, 44
The mean of Cliff Lee's 2008 Game Scores is 56.24, again worse than Doc's. He scored in the 20's twice, which Doc never did, and scored in the thirties twice, which Doc also did twice. Lee scored at least 80 five times, while Doc only did three times. However, Lee scored between 70 and 80 only three times (so was above 70 eight times), whereas Halladay scored at least 70 eleven times. Halladay started 33 games, two more than Lee's 31, so even if Lee scored 70 in both of those "lost" starts, he would still have one fewer 70. Advantage: Doc
A lot has been said about how the qualities of the offenses that Halladay and Lee faced should be a point in Doc's favour as well, but even without accounting for the quality of opponents faced, there is a marked difference between Halladay and Lee according to PQS and Game Score, which is Halladay's consistency.
The two are almost indistinguishable if you look only at their best performances (slight edge to Doc, in my opinion), but Halladay's lows were markedly better than Lee's. While most conventional metrics would give the nod to Cliff Lee, both PQS and Game Score point to Roy Halladay as 2008's definitive Cy Young Award Winner. Granted, PQS is designed more to look at least as much at a player's potential going forward as to indicate his past performance, but Game Score is an excellent indicator statistic, and Halladay is better by both metrics.
This is not to take anything away from Cliff Lee's excellent 2008 season. To put things into perspective, Daisuke Matsuzaka, who placed 4th in Cy Young Award Voting (3rd among starters), managed just a PQS mean of 3.1 (48% Dominance, 14% Disaster) and Game Score mean of 56.38. If you were wondering, Matsuzaka's high Game Score mean was largely driven by his tendency to strand runners.