I have always been critical of the “Verducci Effect” or “Year After Effect” as I have yet to see the study that proves its existence. Now that it’s February again, Tom Verducci has released his top ten high-risk pitchers based on the “Verducci Eeffect”. As I was reading the article I became a bit upset by his defensive nature and use of circumstantial evidence to prove his point. I am a scientist in drug discovery and if I ever used his “evidence” to prove the efficacy of a compound I would be laughed out of a job or selling Homeopathic medicine.
So, what actually is the Verducci Effect? The general idea is a pitcher under the age of 25 who increases his number of innings by more than 30 from his highest total is at risk of injury in following years. On the surface the theory seems plausible; with proper evidence and data we could formulate a study to allow the theory to be critized and able to stand behind on its own merits.
My first criticism is that Verducci allows the evidence to not just be injuries, which are already very common among pitchers under 25, but also uses decreased performance as a proof of the theory. It’s even more maddening that he uses ERA to prove this decrease in skills. Being a reader of Fire Brand I’m sure you know ERA season by season is not a very accurate measure of a pitchers skill, as the eventual ERA value assigned per year isn’t solely dependent upon the pitcher’s performance. Numerous factors outside the pitcher’s control have a great affect on ERA.
Before we get to this years list let’s look at some examples of his 2009 “successes”.
At this time last year Mets pitcher Mike Pelfrey tried to convince me why he should not be on my 2009 list despite his 48-inning jump. He was a big guy, he said, who learned to be more efficient with his pitches. What happened? His ERA shot up from 3.72 to 5.03.
If you asked anyone with an understanding of FIP, xFIP or advanced metrics they would have said Pelfrey was not the 3.72 ERA pitcher and with a xFIP of 4.49 in 2008 it wasn’t very surprising to postulate that he would regress in 2009. His xFIP was 4.52 in 2009; he was just as good, but things out of his control led to the ballooning ERA.
Jon Lester was also on the 2009 list, but defied the “Verducci effect”, so what did Verducci say about that?
Jon Lester, with only a slightly higher ERA in a fine 2009 season, merits mention, too.
Wait, a huge jump in strikeouts and a similar walk rate and he “merits mention”? That’s really stretching your evidence to prove your theory. What about the rest of 2009? His four who escaped injury or “poor performance” are Ubaldo Jimenez, Tim Lincecum, Clayton Kershaw and Jair Jurrjens.
Besides Lester and Pelfrey he counts Cole Hamels, Chad Billingsley, John Danks and Dana Eveland. So while none of these pitchers sustained any significant injuries, suddenly their performance becomes the metric to confirm the his claim? I thought the Verducci Effect was set up to establish injury predictions? None of these pitchers spent a single day on the disabled list! The goalposts have been moved.
Last year was the worst year for the “Verducci Effect” as well. In the last four years he has made 34 predictions and only four have been better (only by ERA) and health in the following year. That includes the three from this year.
Honestly, I don’t think the theory is completely untrue and young pitchers should be treated cautiously, but when you are using ERA to cover up for lack of health troubles then it’s clearly disingenuous. (Will Carroll of Baseball Prospectus tested the theory and found that found that pitchers who break the rule tend to get injured. Although I have not seen the sample tested and if it accounts for the natural testing bias of selecting from a limited group.)
There is another inconsistency with the Verducci Effect; something Brian Burke of Advanced NFL Stats discussed at Roto Savants last year. By picking only from pitchers with a large increase in innings pitched from previous years, Verducci ultimately creates a selection bias. Pitchers who suddenly have a sharp spike in innings are normally a group with exceptional and unexpected performance. Naturally, that means we should be expecting some regression the following year. This subset of pitchers now selected are designed to adhere to Verducci’s ERA backup claim.
Perhaps Verducci knows there have been questions about the theory as he said:
I try to stress that the effect is not a predictor — it’s just a guideline of risk.
That’s fair to say, but at the same time he pats himself on the back for the Joba Rules, Orioles shutting pitchers down and teams setting “target innings”. The league is adapting to his rules that he even sells are at best a guideline and not a predictor. Of course he then “predicts” these ten pitchers will be injured or worse in 2010:
Cesar Carrillo, SD
Bud Norris, HOU
Mat Latos, SD
Joba Chamberlain, NYY
Homer Bailey, CIN
Josh Johnson, FLA
Rick Porcello, DET
Max Scherzer, DET
Felix Hernandez, SEA
Wade Davis, TB
I’m not sure how Carrillo can get worse since his ERA in the minors was not impressive and he only had 10.1 IP in the majors. Chamberlain is an easy layup in my mind as he already has injury problems, but I think it’s unlikely he’s much worse than 2009 in ERA if he’s healthy. Of course the obligatory pick of Bailey as he pitches for Dusty Baker is not a surprise either.
Looking over this list I think 2010 will be Verducci’s toughest year to get away with the worse ERA response to show he was correct. Homer Bailey was another pitcher with a poor ERA and the high xFIP to match. At the same time, he took the pitchers with injury history who lack the “skill” to stay healthy. I don’t think anyone thinks Joba is a sure thing to pitch 170+ IP.
My opposition to the “Verducci Effect” is less to do with the theory and more to do with the “evidence” and selection bias. He has a set of rules that he publishes, but he then selects a subset of that group. Why is the “Verducci Effect” not a yearly listing of all players who fit the two rules? The small grouping makes it harder for the theory to be analyzed properly, whether to accept or reject. Also, by tweaking the selection criteria to his needs, the Verducci Effect is being kept in a positive light unfairly.
In my opinion there are no golden rules for statistically proving a higher risk of injury in players that can be used for everyone. Things like mechanics, scouting, Pitch f/x and medical staffs should be more trusted then a baseline arbitrary innings increase rule. Here is one study who agrees there isn’t anything he can find, but more testing is needed.