Let’s start with a real classic. Tom DeMarco’s „Why does software cost so much? And other puzzles of the information age“ (http://www.amazon.com/Why-Does-Software-Cost-Much/dp/093263334X)
Well, it is a bit aged but given the projects I have seen, it is far from being outdated. So what is his answer? It’s Peopleware not Software and people have to function in their roles and sometimes they don’t.
DeMarco lists as root causes: Scheduling errors („The schedule is crap, when even high performers have no slack“), missing accountability by management („I don’t ask for an estimate, I ask for a promise!“), missing prioritization („All these recommendations for improving ourselves are great. But what if only one thing succeeds? What would it be?“), and the general tendency to ‚fuck up‘ the end-game (i.e. value capturing after implementation).
And of course DeMarcos specialty – Software Development Metrics. He adds the nice insight that measuring something without a clear idea how to improve on that metric is a waste of time and money. It might be worthwhile to sample business case points etc. for a while, but in the long-run only defect counts should be institutionalized.