Author Archive

Spurious Correlations and Ratios

Freitag, August 24th, 2012

Kronmal, Richard A. (1993) „Spurious Correlation and the Fallacy of the Ratio Standard Revisited“, Journal of the Royal Statistical Society. Series A (Statistics in Society), 156 (3), pp. 379-392.

This has been on my mind for a while. A lot of our research uses looks at cost overruns as the variable to measure the project performance. More precisely we most often use Actual/Estimated Cost – 1 to derive a figure for the cost overrun. A project that was budgeted for 100 and comes in at 120 thus has +20% cost overrun. If the scale needs to be transformed, which in most cases it does, the simple Actual/Estimated ratio offers some advantages, i.e., figures being non-negative.

Most criticism for this comes from the corner of Atkinson (1999)*, i.e., that the holding the project accountable for its initial Cost-Benefit-Analysis (+Time) is an unfairly narrow view that ignores the value of building stuff itself, the wider and possibly non-quantitative benefits for the organisation and the wider and most likely non-quantitative benefits for the stakeholder community.

However, a second corner of critics has also a powerful argument. Ratios cause all sorts of statistical headaches. First, dividing a normally distributed variable by another normally distributed variable creates a log-normal distributed variable, i.e., it creates outliers that are solely an artefact of the ratios.

More importantly than distributional concerns are spurious correlations. This is an example from the article

„… a fictitious friend of Neyman (1952), in an empirical attempt to verify the theory that storks bring babies, computed the correlation of the number of storks per 10000 women to the number of babies per 10000 women in a sample of counties. He found a highly statistically significant correlation and cautiously concluded that ‚. . . although there is no evidence of storks actually bringing babies, there is overwhelminge videncet hat, by some mysteriousp rocess, they influencet he birth rate‘!“ (Kronmal 1993:379)

What happened in that example. The regression should have been the test of the number of storks and the number of babies in a county. The argument for the ratio is that it will control for the number of women in the county. The argument against it is that that creates a spurious correlation. Better would be an ANCOVA type structure. Or as the article puts it

„This example exemplifies the problem encountered when the dependent variable is a ratio. Even though Y, the numerator of the ratio, is uncorrelated with X, the independent variable, conditional on Z, the ratio is significantly correlated to X through its relationship to Z, the denominator of the ratio.“ (Kronmal 1993:386)

Three more observations are made in the article

  1. Using the two variables and their interactions instead of a ratio commonly makes for a worse model than using the ratio, particularly in stepwise regression models.
  2. Ratios are an interaction and can only be adequately interpreted in an equation that includes both of these variables (the main effects)
  3. Use a full regression model with interactions, then include the ratio if it adds to it

The final advice is

But what if the ratio is the ’natural quantity of interest‘, just like in our performance measurement?

The division of the outcome by the estimate is to remove its effect from the numerator variable. Kronmal questions whether „this is the optimal way to accomplish this“. He goes on further „…even when such rates are used, there is no reason not to include the reciprocal of the population size as a covariate. For other ratios, the purpose of the denominator is usually to adjust for it. In these instances, there is little to commend the use of this method of adjustment.“ (Kronmal 1993:391)

I will think about this a while, get in touch if you want to share thoughts on this.

* Atkinson, Roger (1999) „Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria“, International Journal of Project Management, 17 (6), pp. 337-342.

How to write a good essay

Mittwoch, August 8th, 2012

Yesterday at lunch I had a discussion with two of our MSc students on how to write. We started of on how to write a good thesis and ended up talking about how to write a good essay. This morning I got an e-mail from the Chair of the Examiners, who is the person running a committee that decides the marks for student work.

N.B. marking in Oxford is its own case study of accountability, transparency, and power. I don’t understand how such an intricate system has evolved that relies on double-blind processes combined with committee decisions and multiple-levels of hierarchy to quality control all to derive ‚objective‘ marks while the revelation that facts are constructed came to this institution as a big surprise.

The email I got this morning asked me to give some students feedback on one of their essays. I have to admit switching from communication by powerpoint to communication via unformatted, double-spaced, prose was one of the greatest challenges of starting with this DPhil. I also just read Dan Ariely’s brilliant blog post and the subsequent op-ed in the LA Times on this topic.

Drum roll. Here is my list on ‚How not to write you essay

  1. Answer a different question. Well, why wouldn’t you. Time is short, the deadline looms. Luckily, in this other course there was a required reading, which you still remember and which could shine some new light on the question. Brilliant idea! Of course there are bonus points to be earned for bringing in new literature. This is perfect murder of two birds with one stone. Unfortunately the execution often falls through. The argument, already a basket case full of apples and oranges, doesn’t get the cream and chocolate sprinkles on top, which it deserves but rather gets a completely new addition, which looks more like a block of cheese with a smell of old socks rather than a fresh idea.
  2. Look up the etymology of the key concepts. No argument has ever been advanced by looking up the etymology, well outside the realm etymologists anyways. It is always good to know that the word project can be traced back to 1450. Always good way to use space.
  3. Give good solid definitions for all concepts. A good essay ought to start with a long laundry list of working definitions for key concepts. Let’s define risk, organisations, bias, projects, and my favourite major programmes. Once that is out of the way we can actually start looking at the question. Again a great way to use the space.
  4. Write up the lecture slides. Just on the off-chance that the marker hasn’t read the slides, just copy them and expand the text a little. Did you make a recording of the lecture. Even better. Easy peasy lemon squeezy.
  5. Cover everything that has been touched upon in class. Decision-making is hard, to decide what concepts to use and which ones to ignore is risky. Avoid cutting something out whenever possible. On the flip side if you cut something out you should not talk about why you took a specific lens.
  6. Make shit up. Drop names. I do have 10 years of experience in this, so let me tell you what I think. I think that the following 8 factors are the key to success in the field. Also, since it is my own opinion I don’t need to add references. Time saved! Damn, they want a reference. Let’s just put an article here whose title sounds as if they would agree with my thinking. Done!
  7. Be Malcom Gladwell „A cursory reading of 5 journal articles has brought me here today to tell you…“

My list for a good essay

  1. 1 idea per paragraph, first sentence explains how this is important to answer the question, last sentence gives the so what? and answers the question. Sounds simple, then go on and do it!

My background is in Computer Science and my old prof Eric Schoop introduced me to information mapping most essays I have to read would certainly benefit from bringing stronger principles to writing.

Research Featured in Harvard Business Review

Donnerstag, Juli 26th, 2012

After 2 years of researching ICT projects the on-going research has been picked up by the Harvard Business Review and is on the cover of their September 2011 issue.

„Why your IT projects may be riskier than you think?“

By now, I collected a database of nearly 1,500 IT projects – in short we argue that the numbers in the hotly debated Standish Report are wrong, but their critics don’t get it quite right either. We found that while IT projects perform reasonably well on average the risk distribution has very fat tails in which a lot of Black Swan Events hide. 1 in 6 IT projects turned into a Black Swan – an event that can take down companies and cost executives their jobs.

Enjoy the read!

More background reading on the HBR article can be found in this working paper.

Most activity of this blog has shifted

Mittwoch, September 8th, 2010

It was in the air for some time and then I read it in wired and last week in the Economist, the face of the web is changing once again. WWW becoming the less dominant force. Well, I used gopher:// when I was little and the university’s library still offers telnet access.Times are changing and I now write mostly at: http://www.facebook.com/pages/Oxfords-BT-Centre-for-Major-Programme-Management/122182117824942 Looking forward to seeing you there! -Alex

Managing the development of large software systems (Royce, 1970)

Donnerstag, April 23rd, 2009

DSC02121

Royce, W. Winston: Managing the development of large software systems; in: Proceedings of IEEE Wescon (1970), pp. 382-338.
http://leadinganswers.typepad.com/leading_answers/files/original_waterfall_paper_winston_royce.pdf

It’s never to late to start reading a classic.  This is one for sure.  The original paper which proposes the waterfall software development model.  This is now extremely common place – but and that is what stroke me odd as well, the model shows a huge number of feedback loops which typically are omitted.

The steps of the original waterfall are as follows

  1. System Requirements
  2. Software Requirements
  3. Preliminary Program Design which includes the preliminary software review
  4. Analysis
  5. Program Design which includes several critical software reviews
  6. Coding
  7. Testing which includes the final software review
  8. Operations

Among the interesting loops in this model is the big feedback from testing into program design and from program design into software requirements.  By no means can is this model what we commonly assume to be a waterfall process – there are no frozen requirements, no clear cut steps without any looking back.  This is much more RUP or AGILE or whatever you want to call it than the waterfall model I have in my head.

A Principal Exposition of Jean-Louis Le Moigne’s Systemic Theory (Eriksson, 1997)

Dienstag, Januar 6th, 2009

DSC01427

Eriksson, Darek: A Principal Exposition of Jean-Louis Le Moigne’s Systemic Theory; in: „Cybernetics and Human Knowing“. Vol. 4 (1997), No. 2-3.

When thinking about complexity and systems one sooner or later comes across Le Moigne.  Departing point is the dilemma of simplification vs. intelligence.  Therefore systems have to be distinguished to be either complicated = that is they are reducible, or to be complex = show surprising behaviour.

Complicated vs. Complex
This distinction follows the same lines as closed vs. open systems, and mono- vs. multi-criteria optimisation.  Closed/mono-criteria/complicated systems can be optimised using algorithms, simplifying the system, and evaluating the solution by its efficacy.  On the other hand, open/multi-criteria/complex systems can only be satisfied by using heuristics, breaking down the system into modules, and evaluating the solution by its effectivity.

In the case of complex systems simplification only increases the complexity of the problem and will not yield a solution to the problem.  Instead of simplification intelligence is needed to understand and explain the system, in other words it needs to be modelled.  As Einstein already put it – defining the problem is solving it.
Secondly, to model a complex system is to model a process of actions and outcomes. The process definition consists of three transfer functions – (1) temporal, (2) morphologic, and (3) spatial transfer.  In order to make the step from modelling complicated system to modelling complex systems some paradigms need to change:

  • Subject –> Process
  • Elements –> Actors
  • Set –> Systems
  • Analysis –> Intelligence
  • Separation –> Conjunction
  • Structure –> Organisation
  • Optimisation –> Suitability
  • Control –> Intelligence
  • Efficacy –> Effectiveness
  • Application –> Projection
  • Evidence –> Relevance
  • Causal explanation –> Understanding

The model itself follows a black box approach.  For each black box, its function, its ends = objective, its environment, and its transformations need to be modelled.  Furthermore the modelling itself understands and explains a system on nine different levels.  A phenomena is

  1. Identifiable
  2. Active
  3. Regulated
  4. Itself
  5. Behaviour
  6. Stores
  7. Coordinates
  8. Imagines
  9. Finalised

Framed! (Singh, 2006)

Dienstag, Januar 6th, 2009

DSC01426

Singh, Hari: Framed!; HRD Press, 2006, ISBN: 0874258731
Amazon-Link

I stumbled upon this book somewhere in the tubes.  I do admit that I felt appealed to combine a fictional narrative with some scientific subtext.  Unfortunately for this book I put the bar to pass at Tom DeMarco’s Deadline.  On the one hand Singh delivers, what seems to be his own lecture on decision-making as the alter-ego of Professor Armstrong; on the other hand the fictional two-level story of Larry the first person story-teller and the crime mystery around Laura’s suicide turn murder does not really deliver.  Let alone the superficial references to Chicago, which I rather found off putting, I think a bit more of research and getting off the beaten track could have done much good here.  Lastly, I don’t fancy much the narrative framework driven style so commonly found in American self-help books – and so brilliantly mocked in Little Miss Sunshine.

Anyhow let’s focus on the content.  Singh calls his structure for better decision-making FACTNET

  • Framing/ conceptualising the issue creatively
  • Anchoring/ relying on reference points
  • Cause & effect
  • Tastes for risk preference & role of chance
  • Negotiation & importance of trust
  • Evaluating decisions by a process
  • Tracking relevant feedback

Frame – Identify the problem clearly, be candid about your ignorance, question presumptions, consider a wide set of alternatives
Anchoring – Anchor your evaluations with external reference points and avoid group thinking
Cause & effect – Recognise patterns and cause-effect-relationships, try to regress to the mean, be aware of biases such as the halo effect
Tastes for risk & Role of chance – be aware of compensation behaviour, satisficing behaviour, cognitive dissonance, signaling of risks, gambler’s fallacy, availability bias – all deceptions which negatively impact decision-making
Negotiation & Trust – just two words: Prisoner’s dilemma
Evaluating decisions by a process – Revisit decisions, conduct sensitivity analyses
Tracking relevant feedback – Continuously get feedback & feed-forward, be aware of overlooked feedback, treatment of effects, split up good news and bundle bad news, think about sunk costs, man & machine, and engage in self-examination

Three methods for decision-making are presented in the book – (1) balance sheet methods with applied weighting, (2) WARS = weighting attributes and scores, and (3) scenario strategies.

Lastly, Singh reminded me again of the old motto „Non Sequitur!“ – making me aware of all the logic fallacies that occur if something sounds reasonable but ‚does not really follow‘.

10 Mistakes that Cause SOA to Fail (Kavis, 2008)

Dienstag, Januar 6th, 2009

DSC01430

Kavis, Mike: 10 Mistakes that Cause SOA to Fail; in: CIO Magazine, 01. October 2008.
I usually don’t care much about these industry journals. But since they arrive for free in my mail every other week, I could help but noticing this article, which gave a brief overview of two SOA cases – United’s new ticketing system and Synovus financial banking system replacement.

However, the ten mistakes cited are all too familiar:

  1. Fail to explain SOA’s business value – put BPM first, then the IT implementation
  2. Underestimate the impact of organisational change – create change management plans, follow Kotter’s eight step process, answer everyone the question ‚What’s in it for me?‘
  3. Lack strong executive sponsorship
  4. Attempt to do SOA on the cheap – invest in middleware, invest in governance, tools, training, consultants, infrastructure, security
  5. Lack SOA skills in staff – train, plan the resources, secure up-front funding
  6. Manage the project poorly
  7. View SOA as a project instead of an architecture – create your matrices, war-room; engage collaboration
  8. Underestimate SOAs complexity
  9. Fail to implement and adhere to SOA governance
  10. Let the vendor drive the architecture

Comment function temporarily disabled

Donnerstag, November 27th, 2008

Sorry everyone, on this wonderful Thanksgiving day this Blog has been flooded with Spam in the comments section. I don’t know why it should be that Thanksgiving increases the need for medication for back pains, restless leg syndrom, or the old-time classic e.d.

Facelift

Mittwoch, September 17th, 2008

I admit neglecting this little blog in the last couple of weeks. In the spirit of a true marketer I gave it a facelift and will start posting some new articles soon – since the new issue of the Internation Journal of Project Management just landed on my desk.

P.S. It’s official Google’s Chrome needs an update – it doesn’t work with wordpress’s WYSIWYG-Editor, it’s more WYSIWYGWLB (what-you-see-is-what-you-get-without-line-breaks).

Why this Blog?

Mittwoch, Juli 2nd, 2008

Dear Internet:

as you might know from my many searches on scholar.google.com, I currently try to build something, which could be the basis for a thesis on project management. My personal style of working is most effective when I summarise articles and books into pages of my notebook. Today I wanted to look something up, I knew I had written down a couple of weeks ago. Unfortunately that made me realise that I spent a significant amount of time flipping through what is now some 70 pages of my notebook, with no X1 or Google Desktop to help me out.

Solution: This blog!