Why don’t good methods prevent project planning failures?

Is there now some scientific thinking to suggest method
development is a fool’s errand? 

In his
recent book, “Streetlights and Shadows; 
Searching for the Keys To Adaptive Decision Making”, Gary Klein makes an
interesting case.  He presents a number
of arguments, with many disparate real world examples, to show that the more
complex the situation, and the more extensively documented the procedure (such
as most IT development and implementation methods!), the less likely a
competent professional will use it, and the more likely that an inexperienced person
will perform at a mediocre level at best.

Let’s run that by one more time.  If the target audience for an IT method is
experienced project managers, they probably won’t use it…  And if the project manager is inexperienced
and the project is a complex one, we are pretty sure that’s a recipe for
disaster…

Hmmm…

Maybe we need to re-think this a bit.

Historically, project managers have railed at methodologists
for not understanding the complexities of project life.  (There is an old joke that asks, “What’s the
difference between a methodologist and a terrorist?  Answer: 
“You can sometimes negotiate with a terrorist.”) 

So when the project manager asks the methodologist, “What’s
the shortest time that this type of project can be done, with acceptable quality,
under the following circumstances?”  The
answer often sounds like, “I can’t tell you, but you better have these 1200
line items in your work breakdown structure.”

Why are we surprised, then, that good methods do not appear
to prevent project planning failures? 
Indeed, my experience suggests that most project managers rarely
reference method materials, but they almost always reference work plans from
trusted sources (and sometimes the only source they trust is themselves). 

And the corollary might be: 
why are we surprised when “bad” methods (like the seductive siren song
of Rapid Application Development or Agile Development) are adopted with almost
religious fervor and mis-applied in complex and mission-critical situations?

So what’s the answer? 
Well, if we can’t get methodologists to adequately address the real
world of project management, why don’t we educate project managers to have a
better understanding of methodology?

I am not arguing that project managers re-invent a method
for every project (although many like to do that).  I am saying we might limit the prescription
of methods to, say, 25 tasks (5 life cycle phases with 5 tasks each, perhaps) and
then teach project managers to build plans that answer my two favorite
questions:

          1.       How
will we know when we are done? 

          2.       How
will we know we have a good solution, not just any solution?

This, of course, will
necessitate extra effort and intelligence at the planning stages of complex
projects.

Oh, right, that could be a good thing…