Saturday, January 22, 2005

Finding Fish in the Fruit Salad

If you do your share of baseball reading, you've probably read several articles recently about the need for scouting and statistical analysis to coexist. I absolutely agree. Sometimes, however, I see something that makes it very hard to stomach the notion that most scouts are competent at what they do.

Sean McAdam's article about the best base stealer in baseball had that effect on me:

Consulting with a number of scouts and front office executives, the usual suspects surfaced: Seattle's Ichiro Suzuki, Tampa Bay's Carl Crawford and the White Sox's Scott Podsednik included.

But the one name everyone mentioned, without fail, was the Florida Marlins' Juan Pierre, who has stolen 203 bases over the last four years, an average of nearly 51 per season. Last year, he stole 45 (Podsednik had 70 to lead the majors; Crawford's 59 was tops in the AL).

Still, this isn't who steals the most, or even who runs the fastest. Pierre would lose both of those contests. Crawford may be baseball's fastest player, and Podsednik has more total steals over the last two seasons. It's generally agreed that Ichiro could steal far more often if he -- or the Mariners -- so chose.

But one expert after another said Pierre is the guy they'd want running if they needed someone to steal a big base.

"First of all,'' explains one major league executive, "he's got good instincts. Speed is one thing; instincts are another. He gets great leads, and that shows he's not afraid to get picked off. And that's another trait you look for: fearlessness. The great ones have it.''

Added a longtime scout: "Some guys -- for lack of a better word -- outrun the ball. Pierre's not one of those guys. But he's plenty fast enough and he uses his speed well. He's the kind of guy who maximizes his ability.''

That all looks pretty reasonable... except that Pierre was caught 24 times last season, with his 45-for-69 good for a 65% success rate. If you've read your Tom Ruane, you know that that's just not acceptable. 2004 was Pierre's worst SB year, but his success rate of 75% entering 2004 was only a little bit ahead of the break even point, and for a fast runner that's pretty unacceptable since the marginal value of a stolen base is less. For someone with Pierre's speed to be as unsuccessful as he's been, I'd tend to think of him as one of the worst base stealers in baseball. It's pretty tough to agree with the above statement that "the great ones have [fearlessness]" if that fearlessness means that they make decisions on the basepaths that reduce their team's ability to win the game. In fairness, it was a baseball executive who gave that quotation. However, the longtime scout's assessment that Pierre "maximizes his ability" clearly flies in the face of the available evidence.

Ah, but McAdam has the apologia for Pierre's success rate:

Pierre was caught 24 times in 69 tries last year -- a far less successful ratio than Podsednik, who was nabbed just 13 times in 83 attempts. But that, too, can be deceiving.

"He steals when it means something,'' says another talent evaluator. "He's not padding his total. Everyone knows he's going and he still makes it most of the time. That, to me, is the mark of a really great basestealer.''

That's a reasonable argument; if Pierre runs at times when the stolen base is very clearly the best strategic option, his numbers could be skewed. Let's take a look, using the splits at

2 out: 17 for 23, 74%
0-1 out: 28 for 46, 61%
"Close and Late": 4 for 4, 50%
Not close or not late: 41 for 61, 67%

2 out: 41 for 49, 84%
0-1 out: 61 for 95, 64%
"Close and Late": 10 for 14, 71%
Not close or not late: 102 for 130, 78%

That certainly doesn't support the thesis that McAdams reports; Pierre is actually having a disproportionate number of successes with 2 outs, which is the time when the negative value of a caught stealing is at its lowest, meaning that he's doing the opposite of maximizing his value by failing on so many steals with fewer than two outs. His 64% success in close and late situations over the past three seasons does somewhat skew his numbers, but he's only at 75% otherwise, so it doesn't drive down his numbers that much and he doesn't attempt too many of them anyway. As such, it's extremely difficult for me to lend credence to the scouting evaluations in question.

If you want to know what a 19-year-old player will do five years down the line, scouting will probably tell you as much or more than statistics. But there's a point at which scouting reaches diminished marginal returns, and statistical modeling gets better proportional to the amount of data available. As such, if you want to make an assessment of who the best fielders, baserunners, and so forth are, scouting will do a great job if you don't have the numbers, but if you have a reasonable amount of data collection taking place scouting is, from an evaluation standpoint, not going to add much to the discussion too often.

You're right, many scouts are totally insane. The stolen base analysis is spot on. But we must be aware that statisticians make mistakes as well, considering the many flaws of McCracken's DIPS system for instance. Of course those flaws are usually discovered by other statisticians.
Good article, as usual. In Adams's defense (sort of): I took his apologia and corresponding quote from the scout to be saying, "look, Pierre goes when everyone in the house knows he's going and he still gets there." To an extent, don't the numbers sort of show that? I'm assuming the common perception is that with two-outs there's not much to lose by sending the runner, and thus the defense is going to be tending to look for the steal. Pierre's rate in those situations is good--81%. But it seems like a Pyrrhic victory: first, it looks like Pierre attempts a steal at roughly the same rate for each out situation--0,1 or 2. (Or at least, he has roughly twice as many steals with 0 and 1 outs as he does for 2 outs). Second, while arguably his good success rate with two outs (when the defense is looking for the steal) indicates skillon the basepaths, his percentage counterinutively goes DOWN when the defense is arguably not anticipating the steal, which sort of seems to torpedo the argument in support of Pierre. Even if he somehow has a preternatural ability to get the base when people are looking for him to do so, the value of those steals is severely undermined by his lower success rate in situations where a baserunner is more valuable. I wonder--is there any indication that stealing with 0 and 1 outs in general is more difficult? In, particular, I'm just wondering if having the middle infielders at double-play depth might provide them easier access to the bag (and perhaps a better target for the catcher?). Keep up the great work!
Not only is traditional scouting suspect, but so is traditional baseball analysis. Most of the traditional pundits annually predict the same teams that won last year. There are many writers, such as yourself, who write much more interesting and compelling articles than their traditional blather.

But this is probably why Depodesta has fielded so much criticism. It took years for Bill James to get a baseball job, and hopefully his ilk will soon make inroads with mainstream analysis (the best of which, strangely enough, can be read in the Wall Street Journal).

Regarding base stealers, I'm surprised nobody has mentioned the biggest stolen base of the year, if not the decade; Dave Roberts. He did it when the season was on the line and with the entire known universe watching. Twice.

It is also interesting that LA had no interest in keeping him, but at the same time he was highly coveted by both San Francisco and San Diego.

I think the biggest advantage of the base stealer is not the stolen bases or the smart base running, but the psychological havoc they wreak on certain pitchers. Roger Clemens, for one, obviously became unglued in the 2003 Series against the Marlins in that situation. His game 7 loss this year was similar, if I remember correctly. It would be interesting to look at the BA of the guys hitting behind the base stealers when they are on base. If it is .100-.200 higher than normal then maybe you got something.
I think the 70% rule is pretty effect most of the time, but I also think there's situational times where the percentage of effectiveness is increased. Just for example, let's say the player on deck bats .280 normally but with runners in scoring position he bats .360? By successfully stealing you've effectively raised the batter's average 80 points. You would think that was worth something.
From what I've read, the concept of clutch hitting is pure myth, which may or may not be the case.

But I'm talking not talking so much batting with runners in scoring position, but a base-stealer at first putting pressure on the pitcher to throw fastballs, which can often get him to worry too much about the runner and lose the batter. Or, give up the steal, get rattled by it and then make 3 mistakes in a row.

I don't know if you can track #2 batters in the situations where the leadoff guy gets on, but if you could, I think it would be very interesting to see the result. If the result was big enough, it might make a new argument for having a base stealer.

Of course he would still have to have a high OBP, and even if that effect is measurable, it still may not be worth keeping Dave Roberts over Jason Werth. As much as I like Roberts and what he did against NY, I think I'd still rather have Werth.
Stephen Bright
I agree... I guess all I was suggesting was that there's more to it than just the 70% rule would indicate. I don't know if RISP is a myth, but if it is then shouldn't career RISP averages and normal batting averages be virtually identical? They're not though. Some people bat significantly better with RISP (some worse) and it may even be partially lineup related (which is also not supposed to matter). I suppose what I'm getting at is there's more data to be analyzed than seems to be quickly surmised in the steals rule, and I'd like to read something where people take those other factors into consideration.

Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?