Survey of Software Development Project Outcomes

Thanks to the PM Podcast

2009-06-09T15:59:00.000-07:00

For publicizing this survey, I'd like to thank Cornelius Fichtner of the PM Podcast, which is the project management profession's leading podcast. You can check it out at:

www.thepmpodcast.com

Link to Survey

2009-05-26T12:05:00.000-07:00

A direct link to the survey, hosted by Survey Monkey:

Click Here to take survey

Here is a plain text link to the survey, which you can copy into your browser's address bar:

http://www.surveymonkey.com/s.aspx?sm=oOq7Hzgz6BCYZfgZoNC72w_3d_3d

Results as of May 21, 2009

2009-05-21T18:03:00.000-07:00

With this update, I am introducing a newer, easier-to-analyze format, the graphics provided by Survey Monkey. This first question constitutes the (really) short form:

For folks willing to fill out a bit more information, I broke the question out into not just a success metric, but also into costs:

This is turning into an interesting experiment in survey design, and an illustration as to how various surveys can come up with different results. Notice that, in the short form, the vast majority considers their project outcome to have been mostly successful. In the long form a different picture emerges: a considerable proportion of end users are disappointed with projects that also usually come in late and cost more than planned.

I am not sure how the picture of success on one form meshes with a decidedly more mixed one on another. Perhaps there is selection bias at work here - those who answer the short form tend to participate in more successful projects. Or perhaps developers consider their project successful even when end users do not. I tend to think that end user satisfaction is more important :)

Results as of May 15, 2009

2009-05-15T18:36:00.000-07:00

Two more responses, new summary:

Totally successful: 3 out of 21
Mostly successful: 13 out of 21
Partially successful: 2 out of 21
Failure: 3 out of 21

The long form is where things get interesting. It turns out that while projects are coming out "successfully", it often is at unforeseen cost:

1. In terms of schedule, the project was finished (or canceled):

Early: 0 out of 9
On time: 3 out of 9
Late: 2 out of 9
Very late: 4 out of 9

2. In terms of budget, the project was finished (or canceled):

Under budget: 1 out of 9
On budget: 2 out of 9
Over budget: 5 out of 9
Way over budget: 1 out of 9

3. In terms of satisfaction, end users were:

Disappointed: 4 out of 9
Satisfied: 5 out of 9

Results as of May 13, 2009

2009-05-13T18:56:00.000-07:00

For this installment, I added a "long form" section. First for the short:

Totally successful: 2 out of 19
Mostly successful: 12 out of 19
Partially successful: 2 out of 19
Failure: 3 out of 19

The long form is where things get interesting. It turns out that while projects are coming out "successfully", it often is at unforeseen cost:

1. In terms of schedule, the project was finished (or canceled):

Early: 0 out of 8
On time: 2 out of 8
Late: 2 out of 8
Very late: 4 out of 8

2. In terms of budget, the project was finished (or canceled):

Under budget: 1 out of 8
On budget: 1 out of 8
Over budget: 5 out of 8
Way over budget: 1 out of 8

3. In terms of satisfaction, end users were:

Disappointed: 4 out of 8
Satisfied: 4 out of 8

Results as of May 10, 2009

2009-05-10T11:00:00.000-07:00

Updated results...

Totally successful: 2 out of 11
Mostly successful: 7 out of 11
Partially successful: 2 out of 11
Failure: none

2009-05-08T22:07:00.001-07:00

Results are coming in... a bit slowly so far. But you guys (and gals) deserve to be rewarded with the results. At this point:

Totally successful: none
Mostly successful: 4 out of 5
Partially successful: 1 out of 5
Failure: none

Keep the responses coming, please!

Results Will Be Posted As They Become Available

2009-05-08T11:50:00.000-07:00

Hi everyone,

I am just now submitting the survey link to various places. As results come in I'll post them, along with any interesting comments.

Lee