Agile approaches insist that customer cannot come up with the reliable requirements at the beginning of the project. Martin Fowler provides four reasons:

1. Software development is a design activity, and thus hard to plan and cost.
2. The basic materials (technology and tools) keep changing rapidly
3. So much depends on which individual people are involved, and individuals are hard to predict and quantify.
4. In today’s economy the fundamental business forces are changing the value of software features too rapidly.

However, the most interesting reason is inside customers minds – they cannot reliably predict what they will need from software in the future.

Lets face inconvenient reality – most of the software ideas, decisions and requirements are not the result of scrupulous research, well prepared theories and mathematical models. If it were so, software development would take so much time, money and energy that it will almost eliminate any business benefits. Also it is very difficult to formalize anything related to irrational human beings (who accidentally are the main users of the software). In the most cases, software ideas are born from intuition, insights, copying other ideas and own experience mixed with some vague expectations. This empirical mess becomes source of the vision, requirements and future problems.

Customer dissatisfaction with software is the problem number one in software development. And the root cause is that many of customer’s expectations and predictions are in vain – mostly because future is fundamentally different than it appears in their thoughts. Why? There are six challenges that customers are facing to predict the future and get their ideas about the software right.

1. Subjectivity and our individual experience

alikaragoz
Software ideas are mainly based on our individual experiences, thoughts and understanding. Our individual experience is unobservable to everyone except ourselves. It is very difficult to communicate it right and completely understand other people. Translations and interpretations of software needs are unreliable – too much depends on what experiences we had, our memory and level of knowledge.
We see problems and possible solutions through lenses of our individual experience and often they shape and distort what we see. In addition, even if software customers can get objective view on their problems, they would see it from completely different perspective than domain experts, developers and end-users – people who shape ideas, build and use software. Subjectivity makes expression and understanding of software ideas very difficult.

2. Imagination and reality

denn
We use our imagination and interpretation of reality to come up with software requirements. We believe that things are in reality as they appear to be in the mind.
Imagination is a powerful tool, but it has shortcomings rooted in shortcoming of our memory and perception. Memory doesn’t store entire experience in the memory. Rather it has reduced version of critical things, summary (e.g. meeting was disappointing) and relations between things. Later when we want to remember, our brains are quickly feeling gaps and unconsciously fabricating missing details.
We should recognize distinction of things in the world and in the mind, and realize that we see only our internal interpretation. And our interpretation of the potential solution is troublesome – we usually fail to imagine important details about future software use, environment and difficulties as we do not remember every detail of what happens with us.

3. Reductionism and complex systems

Software is a complex thing. And it is much easier for us to understand and reason about complex things by reducing them to elements and particular examples and consider them separately. Decisions and ideas are becoming based on this fragmented and incomplete understanding, which is far from holistic and comprehensive knowledge about the system. We leave out details of elements interactions, emergent system behavior and many cases, which are difficult to predict, but essential for building successful and useful software.

4. Time horizon and disappearing details

84263554@N00
We differently imagine near and far future and therefore perceive them differently. Near future is concrete and detailed and we think in terms ‘how’. Far future is smooth and blurry and we think in terms ‘why’. When we are thinking in terms of ‘why’ instead of ‘how’, in terms of causes and consequences instead of execution, we fail to consider the fact that the detail-free glorious future we were imagining would not be the detail-laden future we would ultimately experience.
As Daniel Gilbert put it in Stumbling on Happiness: “Babysitting next month is ‘an act of love’ and babysitting right now is ‘an act of lunch'”.
Father the customer see in the future, more promising, full of sense and rose colored it becomes. As a result, customers become more optimistic, bold and creative with software ideas and avoid thinking about these small and evil details.

5. Distorted views and manipulation with own experience

malingering
Our brains tend to interpret events the way it wants to, which influence our interpretation of current and future situations. We manipulate our experience to make it easier to appreciate ourselves in positive light even with negative experiences. Psychological immune system defends our mind against unhappiness, cooks facts and shifts blame.
We unconsciously incline to information that supports our favored views and avoid or rigorously scrutinize challenging for our position facts.
Therefore, customers with established ideas about software could unconsciously ignore troubled opposing ideas and defend their own. This biased approach will come back with useless software and unhappy users.

6. Ignoring changes

meltingmama
We have tough time to imagine that we will ever think, want, or feel differently than we do know. People overestimate the extent of their future appetite when are hungry.
We fail to recognize that our future selves won’t see the world the way we see it now. If we want to predict experience in the future we must consider what kind of comparison we will be making in the future and not in the present. It explains why we love new things when we buy them (in one context) and then stop loving them shortly after.

How can we overcome these challenges?

The most reliable way to predict our future is to make it happen and examine all the consequences. But it could be too late. We can also try to

1. Ask people who had solved similar problems about their experience, good and bad consequences of their decisions, methods and solutions.
2. Experience the outcomes of solution as early as possible. Frequent agile iterations is the best method in the software development for getting most feedback from the working system early.
3. Establish rich and effective communication channels with all participants, boost flow of ideas, consider all opinions and views and try to create shared picture, understanding and knowledge.
4. Build diverse, independent and self-organizing team, which will bring more realistic vision and cancel each other interpretation errors.
5. Learn from mistakes, correct our views and approaches. And go ahead again with gained experiences to build better software.

One final thought. Inability of customers to come up with right requirements demands shift in thinking of software development teams (which also cannot come up with right requirements alone for the same all reasons we discussed above). Developers shouldn’t be just coders, who are blindly following customers requirements. They should become intelligent translators, helpful collaborators and equal partners in the tough mission of building good software and overcoming human psychological barriers, biases and mistakes to achieve success.

Resources:
The New Methodology, Martin Fowler
Stumbling on Happiness, Daniel Gilbert

Posted by Andriy Solovey | Permalink | Trackback