Why does development always seem to take longer than expected?
We’ve been asking ourselves this question for a long time. The number of articles out there written on the same topic suggest to me that it’s something almost every IT company faces and struggles with. Whether you’re a customer to a development firm or you’re involved in the development process yourself, at some stage you’ve probably wondered the same thing: why do development projects almost always take more time than anticipated?
For a long time we were reading around the subject and asking both ourselves and our developers what the reasons might be and what could be done about it, but somehow our efforts didn’t seem to be leading to any substantial changes. Eventually we decided it was time to get serious, get our hands on some real data and do something about it rather than working off of speculation.
I want to use this article to draw back the curtain on some of our internal procedures, examine the new process we’ve established for dealing with time creep and face this issue from a new perspective.
Understanding the problem
Very often developers end up spending more hours on a project than they initially estimated. Typically it does not affect the client’s wallet or the project deadlines, at least in our case. This is because our projects have a fixed price and the deadlines we set leave a little room for the unexpected. So for us it’s merely an internal problem; when the developers underestimate the time they’ll need it’s the company that suffers, not the client.
It seems like such a simple, obvious problem, yet it’s surprisingly difficult to predict and solve.
The hard reality is that developers often overestimate their capabilities (in terms of the speed they can code at) or underestimate the project (in terms of complexity). Given that they usually have just a few hours in which to estimate all of the hours they’ll need to work over 4 to 6 months of development it’s an understandable oversight. But whilst we understand that there are a lot of things that are hard to foresee, underestimating leaves us with a tension which requires careful management.
The process we’ve implemented
Here I’ll go into detail about what we’ve done to address the problem so far. The process we’ve set up allows us to keep track of the reality of the situation and stay aware of project timescale issues from start to finish.
I’ll describe the process here step by step, but one of the really nice things about it is that it’s very simple. There’s nothing incredibly unique about it, in fact for some it could even be a little boring (like so many data-driven processes). The results this activity produces however are quite exciting.
You’ll only need 2 tools to set up this process and it’s likely that you’re already using them elsewhere – Google Sheets and some sort of time-tracking sheet or software, in our case Redmine.
Step by Step
1) It all starts with the developer’s estimate of the project.
Whenever a new project is estimated, we capture the actual developer’s estimate in a Google Drive folder, writing the details of the estimate (who estimated, what the scope was at the time, the date of the estimate and of course the actual hours quoted).
2) Once the project is signed off for development, we add the original estimate numbers to the Google Sheet where we track ongoing performance and time spent against the actual quote.
The spreadsheet looks like this:
As you can see, the spreadsheet shows us the project deliverables (i.e. platforms), hours estimated, hours spent, and the names of the developers working on the project. In addition, it’s presented in a nice chart that’s easy to read, which shows actual data of how the developers are doing compared to what they originally estimated (we’ll talk a bit more about the data itself after I’ve explained the whole process).
3) Updating the document
Every Monday our office manager Katya takes the number of hours spent by each project developer for the past week (from Redmine), and adds it to the spreadsheet, which automatically updates the total time and charts.
4) Reviewing of the document by the project team
Then as our project managers kick off the week with their teams, they make sure to review the document together, to see the current state of things and how the development aligns with the estimate.
How does this help?
Well, first of all, it helps us to see the real picture and foresee problems. When we can see that we’ve spent 50% of the project hours, but the team knows that the project is not 50% done, it’s a problem, but we’ve spotted it and we can start looking for a strategy to solve it. Without the system in place, we might only realise that there was a problem near the end of the project when it was already critical!
In the example above you can see that we’re almost out of hours for the backend development, which should mean that by this time all of the main functionality should be finished and we should be working on bug fixing and polishing the system.
It helps us keep the developers accountable, as seeing that their own estimate does not align with their performance motivates them to work better and faster as well as to improve their estimating skills.
In this case the web app guys have obviously exceeded the estimated amount of time, which means that there’s a big need for a retrospective meeting, to analyse what went wrong and why the result is as it is.
One thing we’ve found is that developers actually love this document, because they can see their performance against the estimate and get a better feel for the state of the project over all. Managers, on the other hand, enjoy it because they can see who among the developers needs to work on improving their estimates, and also because it makes the process of managing project costs/timescales a lot easier.
Can we leverage this data further?
Whilst a lot of the value we get from this document comes simply out of it’s existence and our ability to ongoingly check our actual progress against our initial estimates, we’re interested to find other ways that this data can improve our performance. It has obvious relevance to our estimating skills, development speed and work processes, but perhaps you can see a way we can leverage it even further to improve in all these areas and make sure that actual development time is as close as possible to our estimates. If you have any ideas – please do share them with us in the comments below, we’d love to hear your thoughts!
Simple yet effective
This solution is simple, I know, even obvious. But despite seeming like common sense, it’s far from being standard practice. I’ve spoken about the idea with representatives of a range of different development companies, including some much larger than ekreative (500+ employees) and whilst they think it’s a good idea and say that they ought to implement something similar, they’re not doing it at the moment.
So give your team a fresh performance boost by implementing a similar system! You’ll find that the data does a lot of the talking itself; faced with the reality of how their own words line up with actual performance, developers and managers alike will start looking for innovative ways to align those two columns. The process of improvement starts with a true understanding of reality.