A few weeks ago, Doug Drinen posted a study on his blog starting here- http://www.pro-football-reference.com/blog/wordpress/?p=243 that looked at evidence of irrational coaching behavior, by looking at the difference between run/pass distribution on 2nd and 10, depending on whether the previous unsuccessful play was a run or a pass.
That post got me thinking about other potential examples of irrational coaching behavior, and the first circumstance that came to mind were 4th and 1 situations after a team was "stuffed" on the previous 3rd down play and failed to get the first down. My impression is that coaches get a little more conservative in this situation than, say, if they get stuffed on first down and then get to 4th and 1 after a pass. I think there is probably some mentality of not wanting to fail back to back that keeps coaches from going for it in this situation as often as they should.
Before I get to the numbers, some other thoughts. A run that would be classified as a 1-yard run if it occurred on 2nd and 6 would be classified as a 0-yard gain if it occurred on 3rd and 1, even if it gained a full yard or close to it, if it failed to make it past the first down marker. Thus, I would expect that 4th and 1's resulting from 0-yard rushing gains on 3rd down would be, on average, closer to the first down marker than other 4th and 1's. How much closer I cannot say, but it seems intuitive to me that some percentage of these 0-yard gains actually moved the line of scrimmage closer to a first down.
On the other hand, I would expect 4th and 1's resulting from officially recorded positive gains to be more randomly distributed, ranging from 4th and inches to almost 4th and 1 and 1/2 yards. Thus, if all else was equal, and there was no bias against going for it after being stopped with little to no gain on the previous play (and if the relative distance of a 4th and 1 was a determining factor), I would expect that the attempt rate and success rate on 4th and 1 following failure on 3rd and short to be, at least to some degree, greater than following 3rd and medium or long plays.
Now to the actual numbers from the 2006 seasons, with a couple of caveats. I looked at all 4th and 1 opportunities between the opponent's 40 yard line and the opponent's goal line, with the thought that this was the area where teams are actually willing to go for it. I excluded plays occurring in the last minute of each half. I counted it as an opportunity even if a team took a delay of game penalty, with the thought that this was a conscious decision "not to go for it" by the team in question. I did not count it if there was another pre-snap penalty (false start, e.g.). If a team attempted to go for it, but was called for an accepted offensive penalty during the play, I counted it as an attempt, but did not count it against the success rate. The thinking here is that it is neither a success nor a failure in the same sense that conversion vs. failed attempt is, because the team could then punt or attempt a field goal. However, it was an "attempt" because the intent of the coach was clearly to go for it.
Here are the numbers, broken down by field position. The first number is the total number of opportunities, the second number is the attempt percentage (attempts/opportunities) and the third number is the success percentage (conversions/total attempts (minus above caveat)).
4th and goal from the 1
- 1 yard or less gained on 3rd down 23/ 0.696 /0.500
- 2 to 4 yards gained on 3rd down 8 /0.625 /0.800
- 5 or more yards gained on 3rd down 6 /0.500 /1.000
4th and 1, 10 yard line or in (excluding goal to go)
- 1 yard or less gained on 3rd down 17 /0.412 /0.833
- 2 to 4 yards gained on 3rd down 6 /0.500 /0.667
- 5 or more yards gained on 3rd down 16 /0.438 /0.571
4th and 1, 11-20 yard line
- 1 yard or less gained on 3rd down 16 /0.125 /0.500
- 2 to 4 yards gained on 3rd down 9 /0.333 /0.667
- 5 or more yards gained on 3rd down 10 /0.500 /0.600
4th and 1, 21-30 yard line
- 1 yard or less gained on 3rd down 18 /0.500 /0.875
- 2 to 4 yards gained on 3rd down 8 /0.625 /0.800
- 5 or more yards gained on 3rd down 18 /0.611 /0.636
4th and 1, 31-40 yard line
- 1 yard or less gained on 3rd down 24 /0.708 /0.824
- 2 to 4 yards gained on 3rd down 11 /0.818 /0.556
- 5 or more yards gained on 3rd down 18 /0.944 /0.688
4th and 1, all between opponent's 2 and 40 yard line
- 1 yard or less gained on 3rd down 75 /0.467 /0.818
- 2 to 4 yards gained on 3rd down 34 /0.588 /0.650
- 5 or more yards gained on 3rd down 62 /0.645 /0.641
Now, I cherry-picked out the goal line situations, where the majority result from short to no gains on the previous down. Here, I think the rationale is different. In my subjective opinion, if there is something NFL coaches as a group love, it is to pin the opponent on the goal line. An overinflated sense of being able to pin the opponent deep must be what drives some coaches to punt on 4th and 1 from the opponent 36, for example. And so here, success is immediate gratification, and failure leaves the other team snapping in their own end zone.
As for the remainder of 4th and 1 situations, the coaches for teams that gained little yardage on 3rd down are going for it less frequently, but being far more successful when they do, suggesting that the 4th and 1 attempts may be closer on average. More data is necessary, but it appears that in 2006, coaches who were stuffed on 3rd down were more conservative on fourth down than their peers.
FYI, here are the teams with the most times facing 4th and 1 following a no gain play on 3rd and 1:
Green Bay - 5
Dallas - 4
Detroit - 4
Oakland - 4
Washington - 4