Room for Debate ran a feature recently asking “Why Didn't the U.S. Foresee the Arab Revolts?” The answers vary by expert, but most mention the same themes: groupthink, the unpredictability of human events, lack of resources, and higher-ups who don't want to face unwelcome news.
One of the experts consulted is Peter Bergen, who is the author of "The Longest War: The Enduring Conflict between America and Al-Qaeda." Both this book and “Marching Toward Hell: America and Islam After Iraq” by Michael Scheuer advance similar claims. In their experience, it is not a failure of intelligence services, but rather a failure of the political elite to use the intelligence in America's interest.
Both Scheuer and Bergen also advance the idea that the conflict with Al-Qaeda and other Islamist terrorists has been misrepresented. When politicians claim that terrorist organizations hate freedom or the American way of life, these statements do not line up with the public positions that Al Qaeda and its affiliates have taken. In fact, they contrast the statements of Bin Laden with those of Ayatollah Khomenei, who attacked the supposedly decadent and depraved American way of life. For the most part, Al-Qaeda sticks to discussing American foreign policy and does not care very much what Americans do within the United States.
This analysis could either be a source for hope, or for despair. On the one hand, Scheuer makes the case that lower level intelligence, consular, and military personnel are doing their jobs well and ably. An article by David A. Andelman in the World Policy Journal emphasizes this point, but also points out that these dedicated public servants are often ignored and under-appreciated. Essentially, it means their hard work will have little or no effect on policy outcomes.
What is interesting is that most of the experts consulted for Room for Debate advanced different ideas than Scheuer, Bergen, and Andelman. Room for Debate suggests that intelligence is faulty or limited, while Scheuer et al say the problem is not an intelligence failure at all, but rather a lack of political will, for a variety of reasons, to act based on intelligence. If the problem is a political one, then it is difficult to know whether additional funding for intelligence gathering will have an effect.