Part 1. Gaining processing power
Part 2. Becoming intelligent
Part 3. Interacting with humans
Part 4. Building useful programs
Part 5. Future of human programmers

Word Freak

Can computers compete with us, human programmers, in the near future? The short answer is yes – if computers will gain enough processing power, become intelligent, could effectively interact with humans, build useful programs and… still be interested to serve humans.

Computers take over more and more people jobs and areas considered exclusively human. Deep Blue beat the human best chess champion Garry Kasparov, it is not possible to win checkers against computers, they create drugs, carry 70 % of foreign currency trades and will do 50% of stock trades in 2010.

But to become better than a human programmer, a computer should compete with very powerful processing machine – our brains.

Read full post >>

Opinions about developers varied from janitors to “prima donnas” in comments to my previous post Do We Need Software Architects? 10 Reasons Why Not .
Beside discussion about the role of the software architects, the underlying philosophical problem is whether software development is primarily top down (centralized and planned) or bottom up (emergent and adaptive) process. If software development is top down, the architects are essential and crucial people on the project, who concentrate knowledge, establish technical leadership and guide development teams. If software development is bottom up, the developers become primary force for evolving the system, make key technical decisions and care about the architecture; architects (if they still needed) play coordination and mediation roles.

As Goethe said – between two opposite opinions you’ll find not the truth, but the problem. I incline to consider software development as a bottom up process, which occasionally (often in time of changing direction or crisis) needs centralized effort and top down approach.

Evolutionary, adaptive and emergent development of the software system leads to the most optimal solution. However, any software project usually has specific business goal, constraints and cannot evolve forever as natural systems do. Therefore, the main architecture concern is how to balance these two approaches.

Dynamic of the Complex Systems

Complex Adaptive System

Many complex systems show organization – galaxies, biological, market, society, etc. These systems can be explained and studied by referencing their parts, properties and laws (gravitational, supply / demand, etc.). Another approach is to look for the system as a whole, studying dynamic of the elements interaction and system properties – the science of self-organization.

Can we say that the development of the software system has features of self-organization? Mostly yes, as we have

• Fluctuations – optimal solution is not obvious and needs search; many useful vs. noise factors affect this search
• Local interactions – the software solution emerge from local interactions between involved people, business needs, technical platform and IT environment
• Dissipation – a software project consumes energy to keep going, mostly in the form of money 🙂
• Instability – software development deals with constantly changing situations
• Multiple equilibria – there are many possible satisfying solutions
• Complexity – any non-trivial software project has complexity
• Hierarchies – there are many perspectives and levels – organizational, technological and solution domain

The only fundamental property, which sometimes is missing for true self-organization – autonomy or absence of external control. Traditional corporation will always exercise some form of the control over software projects. And this is important to resolve – self-organized systems (evolving bottom up) achieve better solutions than rigid controlled systems. But why self-organized systems can produce better solutions? And what is the best way to control them?

Read full post >>

Recently I was reviewing requirements for Microsoft Architect Certification and had found description for architect roles. After reading I had a question: do we need this kind of architects in a software team? Can they really help with building successful software?

The summary of architect responsibilities (as per Microsoft):

• Enterprise architects: set the overall vision and framework for the IT environment from a strategic business perspective.
• Solutions architects: design the solution to take advantage of the existing assets, integrate them into the existing environment, follow the enterprise architecture, and solve the business problems of the business owner or unit.
• Infrastructure architects: responsible for creating an architecture that meets the business and service level agreement requirements of the business owners and supports the applications and solutions that are required to operate their day-to-day businesses.

The bottom line is that company hires expensive and canny software architects before start of the project. They talk with the business, come up with solutions and make bright key decisions about architecture. But they don’t program. During the course of the project they ensure that developers don’t spoil this great architecture.

Now business owners can hire not so bright programmers, who should just follow directives and implement this great architecture. There is even correspondence with theories of some management consultants that programmers are just house painters.

Completely irrelevant questions: Can you trust advise of your friend about Paris, if he saw the city only on TV? Can you trust battle plans if battle didn’t begin and your commander knows little about enemy? How many chess moves ahead can you predict with high certainty before start of the game?

I don’t reject a need in architecture, but I don’t agree who, when and how should make architecture decisions in a software project.

Read full post >>

42… Do you still need more explanation?

42 is the Answer to The Ultimate Question Of Life, the Universe and Everything (Google confirms this). According to The Hitchhiker’s Guide to the Galaxy, a hyper-intelligent race constructed the second greatest computer in all of time and space, Deep Thought, to calculate the Ultimate Answer to the Great Question of Life, the Universe, and Everything. After seven and a half million years, Deep Thought answered: “forty-two.” The computer rejected complains:
“I checked it very thoroughly, and that quite definitely is the answer. I think the problem, to be quite honest with you, is that you’ve never actually known what the question is.”
This is probably the largest known communication failure in the Galaxy. What about you? Do you know how to communicate effectively, avoid communication failures and still get things done?

Communicating with humans

We know that effective communication with humans requires skills, experience and effort. It is not easy and straightforward. Professor Albert Mehrabian explains that in communication

• Only 7% of meaning is in the words that are spoken.
• 38% of meaning is in tone of voice.
• 55% of meaning is in facial expression and body language.

It is not overstatement. Our unconscious parallel processing in the brains is much more powerful than controlled sequential thinking. It is matter of our survival to use “gut feeling”, all senses and analyze many clues – much more than we can do with the limited conscious mind. In addition, we cannot trust words, we often use them for convincing explanations without really knowing how we came up with them.
Therefore, person-to-person, face-to-face communication is the most effective. But excessive communication could be a big waste of time and prevent us from accomplishing other things. To survive in the modern communication flood we should learn how to communicate effectively.

Read full post >>

Programmer’s Side

Programmers in the software team work well together, trust and support each other. But they don’t talk about their salaries – the most important reason why they work for the company. It is taboo. Tell them that one of their fellow programmers earn 50% more than others. You’ll see smiles disappear, people don’t look at each other and become silent. They will rebound, but these things leave scars on team morale and cohesiveness. Feel of unfairness is not the best companion in the software development.

But why does compensation is so important and sensitive for us?

• We work to live. We spend more than half of our conscious adult life earning money on work. We need money to live. Money enable for us almost everything material in this world. Any non-material incentives cannot compensate insufficient money income.
• Money are social status. The better compensation means you worth more and more attractive from social and even biological perspective. People strive for relative prosperity. They could accept less themselves than see a rival get more.

Read full post >>

Marc Andressen quotes Frans Johansson’s book “The Medici Effect”:

In a [1987 study, researchers] concluded that brainstorming groups have never outperformed virtual groups. Of the 25 reported experiments by psychologists all over the world, real groups have never once been shown to be more productive than virtual groups. In fact, real groups that engage in brainstorming consistently generate about half the number of ideas they would have produced if the group’s individuals had [worked] alone.

It is hard to believe (as with any categorical statement) that this is a final verdict. Some bloggers correctly point that done right, brainstorming is a powerful tool (here and here). Mindtools.com and Wikipedia describe effective brainstorming process.

However, group brainstorming indeed has potential risks for generating less and lower quality ideas:

1. Ideas that challenge conventional wisdom are suppressed or considered mistaken
2. People can change or even disregard own ideas while listening to ideas from others, especially authoritative sources
3. Sticking with the crowd causes less willingness for risky and innovative ideas.

Effective brainstorm process avoids these problems when:

1. People investigated the topic before the meeting and come prepared with own ideas and considerations.
2. People are diverse, have different perspective and background. They bring specialized and tacit knowledge to the table.
3. People are encouraged to think independently, creatively and don’t criticize others. Everybody contributes as equal; ideas could be even presented anonymously.

I experienced myself both successful and unproductive brainstorming sessions. I found that good brainstorming session produces ideas, which are better and more than just selection from individual ideas. It is real enjoyment to see when people create ideas, enrich each other thinking and generate new wonderful and unseen ideas. A side effect is important: people consider themselves as a part of the process, feeling ownership and control. They accept final ideas readily, better understand these ideas and are committed to implement them. This is a way to become self-organizing team.

4 Effective Leadership Styles in Software Development

What should be the qualities of the best leader for the software team: strong decisive knowledgeable or quiet supportive cooperative? Best leaders have two main concerns – people and getting things done (Blake and Mouton’s Managerial Grid). Both concerns have cumulative effect – high concern for people makes them motivated and therefore more productive, high concern for production creates sense of achievement and makes people more satisfied at work.

However, there is no one leadership style that suits all situations, projects and individuals. There are 4 main leadership styles (based on The Hersey and Blanchard model and Vroom and Yetton’s Normative Model), which could be effectively used in software development: The Commander, The Coach, The Supporter and Self-Organization.

Read full post >>

I have just recently read an interesting post in Cedric blog:

“..software engineering is nowhere near as advanced as building engineering is. We are still working on these nuts and bolts, and whenever a new software project starts, we still can’t be sure it won’t collapse under its own weight after just a year. To make a parallel: imagine a world where every time a new construction takes place (say, a bridge), the future of that bridge depends on the team of engineers and workers you choose to build it…”

Bridges do collapse. Tragedy shows that even standard engineering solution which was following strict guidelines, regularly inspected and maintained could fail.

One bridge is a relatively simple construction. We can probably compare it to one week of programmer’s work considering complexity, number of elements and relations between parts. What could we compare in the engineering to the several month project with many developers and subsystems involved? It is comparable to the very complex construction like combination of hundreds interlinked bridges (don’t ask me to prove my ballpark estimates :)). It is comparable to the construction of Boeing. How much does research and development of Boeing 787 cost? $11 bln dollars.

Typical software projects have much lower cost, much less people and don’t have luxury of relying on stable laws of the physical world and established engineering principles. But programmers carry in their brains the same amount of information, many system relations and complexity as in advanced engineering projects . These are essential difficulties that software developers are facing. In addition they deal with the changeable world of people and business needs.

Major bridges collapse every 30 years and engineers learn from these mistakes and make better and more reliable designs. When we complain that Software Development doesn’t have the same sound practices as engineering, we shouldn’t accuse just immaturity of our profession. It is one of the most intellectually complex human activities dealing with very complex systems. I just hope we will experience less and less collapsing bridges both in physical and software worlds.

“There are very few true artists in computer programming, most are just house painters.” – Tim Bryce’s Law

Management consultant Tim Bryce doesn’t like programmers. Many programmers don’t like him either (here, here, here, here and probably in many other places).
Mr. Bryce’s view of programmers:

• programmers often bamboozle others and heighten their own self-importance
• the average programmer has a lower IQ than any other worker with a college degree
• programmers show signs of sloppiness and mental laziness
• they appear disorganized to make it difficult to judge how they are progressing on their work effort and reveal inadequacies in workmanship.
• the typical programmer often laments he/she is being overworked, underpaid, and unappreciated.
• to the programmer’s credit, they usually possess a curiosity about technological developments. However, this must be carefully nurtured by management – too much information may distract programmers from their job.

Responses on Mr. Bryce’s post (referenced above) provide many excellent points why Mr. Bryce is wrong. However, I want to comment points where he is probably right. Also I want to understand why some management consultants with more than 30 years of experience could have such view? Certainly, I reject Freudian view that an unknown programmer hurt feelings of Mr.Bryce in childhood (the main reason is that at this time the world had only few programmers and all of them are known).

Mr.Bryce’s target audience is not programmers (whose low IQ would probably prevent understanding his Theory P anyway), but IT managers and business decision makers. The underlying premise of Theory P is: “The more effectively we manage the people who program the computer, the better we can utilize the systems to support the information needs of the business”. This theory is not about live people, but about pragmatic business and lets consider the theory from this perspective. There are three points where Mr. Bryce could be partially right.

Read full post >>

“I will work harder” – the horse Boxer (from George Orwell’s Animal Farm)

bichuas

The System Thinking Laws from Peter Senge’s book “The Fifth Discipline” applied to Software Development.

1. Today’s problems come from yesterday’s solutions.

We, humans, are happy when we solve problems. We often don’t think much about consequences. Surprisingly, our solutions could strike back and create new problems.

• A company decides to reward few key members of the very successful team with bonuses and promotions. The rest of the team feel unfairness and loss of motivation. Eventually tension between members is increased. The following projects are no longer successful.
• A project manager frequently asks developers to fix a new bug or work on urgent requests from customers. Developers do their best to fulfil these requests. Frequent distractions prevent them from finishing their main tasks for the iterations. Project shows only little progress.

Read full post >>