New Scientists reports that 80 per cent of human behavior is largely automatic and determined by instincts alone. Most of the time we are simply reacting instinctively to the world around us. Only afterwards we make up reasons to explain what we did. Our brains routinely push the control of some activities from conscious to unconscious, such as playing musical instruments or driving a car. Surprisingly, this automatic control influencing important matters such as economic decisions and our dealing with others.
Perhaps, the best way to understand human behavior is to ignore the supposedly rational, consciously generated actions of individuals.

Comments:

1. We often use an automatic behavior even in complex social interactions as in software development. So, it is possible to predict the behavior of team members in more than 80% situations according to the article. It also means that the team will consistently repeat successful or failing patterns of interactions. How can we increase occurrence of successful  and eliminate failing outcomes ? Should we force to turn our conscious more often?
2. Do we use automatic thinking while programming? Is this always only conscious effort as we should expect?  If we use unconscious thinking, how does it effect our results?
3. It is another prove for the correctness of the description of human mind in one of my posts – The Mind of the Programmer 🙂

Software development is the translation of a user need or marketing goal into a software product. – Wikipedia

Why it is so difficult to understand the nature of the Software Development? Because it is ideal and metaphysical. Software Development doesn’t stand on objective physical world, but deals with fragile and subjective world of human ideas. That is why Software Development is much more than Software Engineering – it just cannot fit into Procrustean bed of well defined requirements, processes and laws (maybe except programming for science or physical devices).

Lifecycle of the most software projects:

1. People generate software ideas based on people needs. Humans have difficulty understanding what they need and even more difficulty to clearly explain it to others.
2. Software professionals try to understand these ideas and communicate to each other. Before coming to programmers these ideas are interpreted (more often misinterpreted) by bosses, marketers, project / product managers, architects, business analysts, etc.
3. Programmers are trying to translate these mutated ideas into the program code. Many of them are more interested in creating technical puzzles (or doomed to fight them) than in complete understanding of business ideas – often changing and sometimes illogical.
4. People use this software. Does it meet their needs? How happy do you think they are?

How to improve the flow of ideas?

Read full post >>

Software Creation is very dependent on oil. Our computers, keyboards and printers are made from plastic. Our cars use gasoline to get us to work. Furthermore, almost everything in our modern civilization depends on oil. Global oil production will peak any time from now. Will end of oil cause crash of our civilization? I was always worry about this 🙂
New Scientist assures that governments, technology and biology will rescue our civilization.

• 70% of oil is used for fuel; petrochemicals are only 3.4%. Technology, government regulations and social trends are actively influence reduction of oil usage for energy and fuel. Sweden, for example, plans to be independent of oil exports by 2020, using no oil for heating and halving its transport fuel consumption.
• The new breed of biorefineries is using biomass to produce biofuels and bio-based chemical products. The biomass could be supplied by anything from corn, sugar cane, grasses, wood and soybeans to algae. For example one company is converting turkey guts into oil and fertilizer. By 2025, the US Department of Energy wants 25% of industrial organic chemicals to be delivered from biomass. There are already 120 ethanol refineries in US.
• Biorefineries are more profitable at smaller scales than conventional petrochemical refineries, because they operate at lower temperature and pressure, so they are less complicated to build. However, it is still difficult to compete with some oil-based products.

Therefore, life will continue after oil is over. Proactive thinking, society and government involvement could significantly reduce impact, especially considering that solutions correspond with fighting global warming.

Disconnect between business and IT is increasing and could signal serious change how enterprise technology is run, reports IT World Canada. Michael O’Neil from Info-Tech Research Group thinks that the future of the IT professional no longer lies in acting as an important support system or innovative visionary but as a “utility.” The role where IT functions as a discrete, advisory body will disappear while the majority of IT professionals are assimilated into areas pertaining to business processes and strategy.
There is also an interesting discussion on InfoQ if Agile can rescue this widening gap.

Comments: I don’t think IT should be rescued. The narrow specialization of IT and Software Development professionals and their remoteness from the problem domain and decisions are the serious problems in software creation. I consider disappearance of specialized IT services and departments as a good trend. I hope that combination with another trend – disappearance of centralized and hierarchical corporations – will produce powerful cumulative effect. The companies will become conglomerates of small cross-functional and diverse teams, empowered to make most decisions. Programmers will have deep domain expertise in addition to specialized knowledge how to build programs. Benefits are better decision making, market response and people motivation. See my post The Ideal Software Company.

The Economist article claims that men with a lot of testosterone level make more curious choices. Dr. Burnham of Harvard University research shows that high-testosterone men reject low ultimatum game offers more often. In this experiment, men who reject low offers ($5 out of $40) have significantly higher testosterone levels than those who accept.
The research supports the idea that people really strive for relative rather than absolute prosperity. They would rather accept less themselves than see a rival get ahead. In a variety of species, testosterone is associated with male seeking dominance. If low ultimatum game offers are interpreted as challenges, then high-testosterone men may be more likely to reject such offers.
Economists often refer to this sort of behavior as irrational. In fact, it is differently rational. The main outcome for men is a social status—that brings desirable reproductive opportunities. Money could help, but if another route brings that status more directly, money is irrelevant.

Comments: How this research is related to programmers and software creation?

1. Many programmers (their bosses and, recursively, bosses of their bosses) have high testosterone level, and therefore could exhibit curious and irrational behavior including politics, conflicts and bad decisions for the software project. Is it possible to prevent such behavior? I believe the practices that modern society are using for fighting with dark sides of human nature could help. Transparency of decisions, free opinions and equal intolerance to unsocial behavior disregarding ranks. Can The Ideal Software Company help?
2. Social status in the company could be stronger motivator than money for high-testosterone men: better computers, more praise and ability to influence decisions. However, the knowledge that somebody gets more money for the same work could be very depressing for any testosterone level men :).
3. High testosterone level could be the strong driving force for achievements and very useful if properly applied. But too many of this kind of people could cause a lot trouble and disturbance in the Force :).
4. Here is an interesting quick method to determine testosterone level. Married man with children have the lowest level.

Humans can justify almost anything thanks to cognitive dissonance, explains Newsweek.
Cognitive dissonance is the extreme emotional discomfort we feel when two important beliefs, attitudes or perceptions collide. Humans cannot tolerate dissonance for long, so they ease the tension by making a change in belief or attitude—and justifying the change.
Why can’t we just live with inconsistency and contradiction? We experience unpleasant emotions whenever life forces us to choose a course of action, but we are not fully convinced it’s the right course. When we make an important commitment, the mind instinctively seeks out proof that we’ve done the smart thing.
What if you make a choice that is really bad and you can’t hide from it? People who don’t want to recognize own mistake become even more entrenched in their belief once it is proven wrong. They throw good money after bad in the market, grab for straws in a dying relationship or send yet more troops to fight a misbegotten war.

Comments: All of us have beliefs, and all of us have times when our beliefs are challenged by other people and reality. It is a real pain :). Cognitive dissonance is a natural psychological reaction that cause us to change or entrench deeper, but somehow resolve the issue. Being stubborn and hold to wrong beliefs is bad, but changing them frequently is bad too as it prevents us from building stable and predictable mental concepts of the surrounding reality. The best way to stay in touch with reality is testing our beliefs, get feedback often and keep mind open. This approach is a core of Agile and iterative development and this is one of the best methods to resolve cognitive dissonance in software creation.

Angela Duckworth and Martin Seligman research shows that both IQ and self-discipline are correlated with academic performance and GPA, but self-discipline is a much more important contributor: those with low self-discipline have substantially lower grades than those with low IQs, and high-discipline students have much better grades than high-IQ students. Further, the study found no correlation between IQ and self-discipline—these two traits varied independently.

Comments: Does this research applies for software creation? Can we say that self-discipline is more important than IQ for programming? I believe that self-discipline, emotional intelligence and control over unconscious mind are more important than IQ in programming. You cannot get anything useful done without self-discipline; however low IQ is a problem too.
My formula for the talent in programming: Talent = IQ * Self-discipline * Creativity

Eric Wise argues that software development is not software engineering. Reddit tries to compare it with science. Both discussions have many interesting and insightful comments, huge variety of opinions and show no signs of their reconciliation. I also have post related to understanding of the nature of software development as many other people in their blogs :).
Goethe said that between two opposite opinions you’ll find not the truth, but the problem. Can we ever define an unified theory of the nature of Software Development?

If you want to build a ship, don’t herd people together to collect wood and don’t assign them tasks and work, but rather teach them to long for the endless immensity of the sea. – Antoine de Saint-Exupery

I have a dream about a company that always makes users happy. The company that quickly builds high quality, simple and usable software that fully meets user needs. The company that reacts rapidly on customer demands, anticipates new needs and immediately use these opportunities. The company where people are happy, motivated and could realize their potential, dreams and life goals. The company that is successful on market and in the hearts of users, investors and employees. The Ideal Software Company.

The Barriers

What are the main problems with building software? What are contributing factors to these problem? What prevents many existing software companies from being ideal?

1. Unclear, incomplete or misinterpreted user needs. The software programs often don’t solve needs on adequate level, have bad usability and contain many unused features.
2. Difficulty to estimate required effort. Missed deadlines, doomed schedules and budget overruns are the common deceases in software development.
3. Inability to adopt to changes. Often it is even impossible to estimate effort at the beginning as many things are changing during the course of the project: customer needs, performance requirements, business context. But for many software companies these changes still bring chaos to their projects.
4. Poor, rigid or low quality solutions. Implementation defines project success. Talent, experience or knowledge are necessary to build good solutions, and they are often missing in the project teams.

The main reasons for these problems

1. Top-down decisions – only few people make decisions with limited information, lack of feedback and independent information
2. Poor aggregation of information from different sources
3. Lack of diversity: homogeneous groups that are responsible for only specific aspect of the project (requirements, design, programming, etc.)
4. Lack of accountability for the end results, mismatch in company and personal employees goals, politics and suboptimal decisions.
5. Low motivation and morale – minimal control over decisions, poor relation of incentives and payout to contribution into the company success.
6. Centralization – authoritative control causes inflexibility, lack of independence and adaptability; missing local tacit knowledge in decision making.

James Surowiecki in his book The Wisdom of Crowds clearly shows that traditional top-down companies are inefficient in making the best decisions.

The Ideal Software Company

A. Company organization

1. Powerful teams – the company represents a conglomerate of powerful teams. Projects teams are small, diverse and empowered to make most project decisions.
2. Egoless management – management carry mostly support, coordination and representation functions. Leadership is moved to the teams level. Top management is a consulting body, which includes highly skilled professionals in finances, technology, sales, etc.
3. Shared and competing infrastructure – company infrastructure could include services as hosting, customer support, recruitment or distribution. These in-house services are shared and compete with outside providers.

Read full post >>

Who is smarter – a diverse group or the best expert? Can a boss, a project manager or the most experienced specialist make better decisions than a whole team? What are the problems with decision making in traditional corporations?

James Surowiecki in his book The Wisdom of Crowds argues that given the right conditions a group consistently outperforms best experts in decision making. Few people can do better than the group, but each time they will be different people. A group answer will often be at least as good as an answer of the smartest member.

The author provides interesting examples:

1. 800 different people (mostly non-experts) tried to guess weight of an ox at the International Exhibition of 1884 in London. The average guess of the crowd was only one pound off (ox weighed 1,198 pounds) and was better than any estimate of the cattle experts.
2. Stock market knew what company was responsible for the Challenger disaster within a half hour of the shuttle blowing up. Mostly uninformed investors dumped shares of Thiokol almost simultaneously.
3. Team of the men with wide range of knowledge (e.g. mathematician, submarine specialists, salvage men) correctly estimated location of disappeared U.S. submarine Scorpion. The submarine was found 220 yards from collectively estimated spot (original search area was 20 miles circle) and it was better than any individual guess.

Crowd is wise when these criteria are present:

1. Diversity of opinions (each person have some private information)
2. Independence (people’s opinions are not determined by the opinions of others)
3. Decentralization (people are able to specialize and draw on local knowledge)
4. Aggregation (some mechanism exists for turning private judgments into a collective decision)

Read full post >>