Thought-provoking and interesting author Nicholas Carr has published a new article in Financial Times, where he expands on his ideas about transformation of IT services into utility. He compares IT with electric power generation 100 years ago:

Like data-processing today, power generation was assumed to be an intrinsic part of doing business. But with the invention of the alternating-current electric grid at the turn of the century, that assumption was overturned.

Suddenly, manufacturers did not have to be in the power-generation business. They could run machines with electric current generated in distant power plants by big utilities and fed to their factories over a network of wires. With remarkable speed, the new utilities took over the supply of industrial power. Scores of private power stations were dismantled.

Read full post >>

Programmer’s Side

Programmers in the software team work well together, trust and support each other. But they don’t talk about their salaries – the most important reason why they work for the company. It is taboo. Tell them that one of their fellow programmers earn 50% more than others. You’ll see smiles disappear, people don’t look at each other and become silent. They will rebound, but these things leave scars on team morale and cohesiveness. Feel of unfairness is not the best companion in the software development.

But why does compensation is so important and sensitive for us?

• We work to live. We spend more than half of our conscious adult life earning money on work. We need money to live. Money enable for us almost everything material in this world. Any non-material incentives cannot compensate insufficient money income.
• Money are social status. The better compensation means you worth more and more attractive from social and even biological perspective. People strive for relative prosperity. They could accept less themselves than see a rival get more.

Read full post >>

Software Creation is very dependent on oil. Our computers, keyboards and printers are made from plastic. Our cars use gasoline to get us to work. Furthermore, almost everything in our modern civilization depends on oil. Global oil production will peak any time from now. Will end of oil cause crash of our civilization? I was always worry about this 🙂
New Scientist assures that governments, technology and biology will rescue our civilization.

• 70% of oil is used for fuel; petrochemicals are only 3.4%. Technology, government regulations and social trends are actively influence reduction of oil usage for energy and fuel. Sweden, for example, plans to be independent of oil exports by 2020, using no oil for heating and halving its transport fuel consumption.
• The new breed of biorefineries is using biomass to produce biofuels and bio-based chemical products. The biomass could be supplied by anything from corn, sugar cane, grasses, wood and soybeans to algae. For example one company is converting turkey guts into oil and fertilizer. By 2025, the US Department of Energy wants 25% of industrial organic chemicals to be delivered from biomass. There are already 120 ethanol refineries in US.
• Biorefineries are more profitable at smaller scales than conventional petrochemical refineries, because they operate at lower temperature and pressure, so they are less complicated to build. However, it is still difficult to compete with some oil-based products.

Therefore, life will continue after oil is over. Proactive thinking, society and government involvement could significantly reduce impact, especially considering that solutions correspond with fighting global warming.

Disconnect between business and IT is increasing and could signal serious change how enterprise technology is run, reports IT World Canada. Michael O’Neil from Info-Tech Research Group thinks that the future of the IT professional no longer lies in acting as an important support system or innovative visionary but as a “utility.” The role where IT functions as a discrete, advisory body will disappear while the majority of IT professionals are assimilated into areas pertaining to business processes and strategy.
There is also an interesting discussion on InfoQ if Agile can rescue this widening gap.

Comments: I don’t think IT should be rescued. The narrow specialization of IT and Software Development professionals and their remoteness from the problem domain and decisions are the serious problems in software creation. I consider disappearance of specialized IT services and departments as a good trend. I hope that combination with another trend – disappearance of centralized and hierarchical corporations – will produce powerful cumulative effect. The companies will become conglomerates of small cross-functional and diverse teams, empowered to make most decisions. Programmers will have deep domain expertise in addition to specialized knowledge how to build programs. Benefits are better decision making, market response and people motivation. See my post The Ideal Software Company.

If you want to build a ship, don’t herd people together to collect wood and don’t assign them tasks and work, but rather teach them to long for the endless immensity of the sea. – Antoine de Saint-Exupery

I have a dream about a company that always makes users happy. The company that quickly builds high quality, simple and usable software that fully meets user needs. The company that reacts rapidly on customer demands, anticipates new needs and immediately use these opportunities. The company where people are happy, motivated and could realize their potential, dreams and life goals. The company that is successful on market and in the hearts of users, investors and employees. The Ideal Software Company.

The Barriers

What are the main problems with building software? What are contributing factors to these problem? What prevents many existing software companies from being ideal?

1. Unclear, incomplete or misinterpreted user needs. The software programs often don’t solve needs on adequate level, have bad usability and contain many unused features.
2. Difficulty to estimate required effort. Missed deadlines, doomed schedules and budget overruns are the common deceases in software development.
3. Inability to adopt to changes. Often it is even impossible to estimate effort at the beginning as many things are changing during the course of the project: customer needs, performance requirements, business context. But for many software companies these changes still bring chaos to their projects.
4. Poor, rigid or low quality solutions. Implementation defines project success. Talent, experience or knowledge are necessary to build good solutions, and they are often missing in the project teams.

The main reasons for these problems

1. Top-down decisions – only few people make decisions with limited information, lack of feedback and independent information
2. Poor aggregation of information from different sources
3. Lack of diversity: homogeneous groups that are responsible for only specific aspect of the project (requirements, design, programming, etc.)
4. Lack of accountability for the end results, mismatch in company and personal employees goals, politics and suboptimal decisions.
5. Low motivation and morale – minimal control over decisions, poor relation of incentives and payout to contribution into the company success.
6. Centralization – authoritative control causes inflexibility, lack of independence and adaptability; missing local tacit knowledge in decision making.

James Surowiecki in his book The Wisdom of Crowds clearly shows that traditional top-down companies are inefficient in making the best decisions.

The Ideal Software Company

A. Company organization

1. Powerful teams – the company represents a conglomerate of powerful teams. Projects teams are small, diverse and empowered to make most project decisions.
2. Egoless management – management carry mostly support, coordination and representation functions. Leadership is moved to the teams level. Top management is a consulting body, which includes highly skilled professionals in finances, technology, sales, etc.
3. Shared and competing infrastructure – company infrastructure could include services as hosting, customer support, recruitment or distribution. These in-house services are shared and compete with outside providers.

Read full post >>

Who is smarter – a diverse group or the best expert? Can a boss, a project manager or the most experienced specialist make better decisions than a whole team? What are the problems with decision making in traditional corporations?

James Surowiecki in his book The Wisdom of Crowds argues that given the right conditions a group consistently outperforms best experts in decision making. Few people can do better than the group, but each time they will be different people. A group answer will often be at least as good as an answer of the smartest member.

The author provides interesting examples:

1. 800 different people (mostly non-experts) tried to guess weight of an ox at the International Exhibition of 1884 in London. The average guess of the crowd was only one pound off (ox weighed 1,198 pounds) and was better than any estimate of the cattle experts.
2. Stock market knew what company was responsible for the Challenger disaster within a half hour of the shuttle blowing up. Mostly uninformed investors dumped shares of Thiokol almost simultaneously.
3. Team of the men with wide range of knowledge (e.g. mathematician, submarine specialists, salvage men) correctly estimated location of disappeared U.S. submarine Scorpion. The submarine was found 220 yards from collectively estimated spot (original search area was 20 miles circle) and it was better than any individual guess.

Crowd is wise when these criteria are present:

1. Diversity of opinions (each person have some private information)
2. Independence (people’s opinions are not determined by the opinions of others)
3. Decentralization (people are able to specialize and draw on local knowledge)
4. Aggregation (some mechanism exists for turning private judgments into a collective decision)

Read full post >>

Secrets of Creativity

http://www.sciammind.com/article.cfm?articleID=00028EE8-4369-123A-822283414B7F4945
Why some people are more creative than others? The Scientific American article provides some insights:

• Intelligence is not a crucial ingredient. A crucial variable is the difference between “convergent” and “divergent” thinking.
• Convergent thinking aims for a single, logical and correct solution to a problem. Divergent thinking proceeds from different starting points and changes direction as required
• Creative people can free themselves from conventional thought patterns and follow new pathways to unusual or distantly associated answers, which leads to multiple solutions, all of which could be correct and appropriate.
• These two processes took place in different brain regions – left hemisphere is responsible for convergent thinking and the right hemisphere for divergent thinking.
  • The left side examines details and processes them logically and analytically but lacks a sense of overriding, abstract connections.
  • The right side is more imaginative and intuitive and tends to work holistically, integrating pieces of an informational puzzle into a whole.
• Our creative talent is gradually repressed from childhood. Schools place emphasis on teaching children to solve problems correctly, not creatively.
• The brain is a creature of habit; using well-established neural pathways is more economical than elaborating new or unusual ones, which wither with time if not used.
• Creative people are generally very knowledgeable about a given discipline, it is very improbable to come up with a grand idea without deep involvement in this area.
• Creative solutions result from examining challenge from all sides, disassembling and reassembling the building blocks in an infinite number of ways. The problem solver must thoroughly understand the blocks.
• Too much specialized knowledge can stand in the way of creative thinking. Experts often often internalize “accepted” thought processes, so that they become automatic. Intellectual flexibility is lost.
• Creative revelations come to most people when their minds are involved in an unrelated activity. A little relaxation and distance changes the mind’s perspective on the problem – without us being aware of it. The brain clear away thought barriers by itself and at some point, newly combined associations break into consciousness, and we experience sudden, intuitive enlightenment.
• The neural processes that take place during creativity remain hidden from consciousness and we cannot actively influence or accelerate them.

Comments:

1. Creativity plays a very important role in the software creation. IQ is not enough for developing the best programs as it will lead for technically complex, standard and expensive solutions. Solving changing business need with limited resources, dealing with inconsistent people and complex systems requires unorthodox solutions.
2. Companies should start hiring people based on “Creativity Quotient” in addition to IQ and EQ. The article has interesting examples of the potential interview tests.
3. We should specifically train our creativity and always try to look for solutions in novel ways, considering unusual alternatives, combinations and associations; breaking traditional approaches. We should strive to come with more than one solution.
4. Creativity needs knowledge, serious preparation and understanding what challenges to solve. The brain will not work on the problem until it has enough food for thoughts :). And brain can work out solutions without our conscious involvement, automatically. We just need to learn how to prepare our brains properly.

Material Software Companies

http://technology.guardian.co.uk/opinion/story/0,,2112850,00.html
Software companies are coming to earth and turning from nonmaterial to very physical companies. Nicholas Carr explains why.

• Google has data centres around the globe, reportedly holding as many as 2m or 3m computers altogether, and it continues to spend billions of dollars a year to put up new ones. Its arch-rival, Microsoft, is building a 140,000 sq m six-building data centre on a former bean field. Also rushing to build or lease data centres are Yahoo!, Ask.com, Intuit, Salesforce.com and Deutsche Telekom’s T-Systems unit, among many others.
• Nature of the software business is changing. In the past, software companies only had to concern themselves with writing code, copying their programs on to discs and selling them. It was up to the buyers of the software to maintain the computers, storage drives and all the other hardware needed to run the programs. Now companies are beginning to rent programs over the net for a monthly fee. Even sophisticated applications for maintaining customer accounts, tracking finances, managing workers and performing other complicated tasks are now being offered as web services. The burden of running software is shifting from the buyer to the seller.

Therefore software companies are finding that they have to compete not just on the elegance of their programs, but on their ingenuity and efficiency in buying and deploying physical assets – land, buildings, computers, and other gear – as well as managing the huge amounts of energy required to keep all the machines running.

Comment: I feel we will miss this nonmaterial world, where we should care only about own computer, a table and a chair to write programs 🙂 . But I still believe that the essential and core business of the software companies will be programming, not management of all these physical assets. What do you think?

Discouraging Incentives

http://www.nytimes.com/2007/07/02/opinion/02schwartz.html
Do you think more incentives are better? Not always argues Barry Schwartz.

Economists assumption: the more motives the better. Give people two reasons to do something and they will be more likely to do it. Unfortunately this assumption is false. There are circumstances in which adding an incentive competes with other motives and diminishes their impact.
When you pay people for doing things they like, they come to like these activities less and will no longer participate in them without a financial incentive. The intrinsic satisfaction of the activities gets “crowded out” by the extrinsic payoff.

Comment: Money is the most powerful motivator. However we should carefully offer additional incentives for something that people like doing. It could leave a bitter taste and kill passion, motivation and dedication even if they continue doing this. But not giving money when people expect them could be even worse. So, probably, it is a good idea to discuss / understand person’s internal motives and expectations before finding a way to encourage desirable behavior. Maybe just public and unreserved admiration will be the best reward. We, humans, are so complicated 🙂

Computers carry 70% of foreign currency trades

http://www.newscientisttech.com/channel/tech/mg19426061.500-gordon-gekko-makes-way-for-trading-software.html (need subscription)
Computers make one third of all trading decisions in US markets. Experts predict that more than 50% will be done by 2010. Machines can make multiple trades and monitor thousands of stocks at breakneck speed. They can use tactics that makes trades indistinct and hide their intent; for example, by spreading the deal over many small trades. There are big profits to be made before market realizes these opportunities. Companies are moving their servers as close as possible to stock exchange systems to reduce order time: milliseconds matter in competition of computers. There is a new arm race between trading “algos” (algorithms) – they now try to guess and sneak signs of other algos trading. David Cliff designed one of the first commercially successful algos in mid 90s. Human traders still have place on less understood and illiquid markets where instinct and experience are still important; software helps them with simulations and test of ideas. People are still much better in predicting market trends. How much longer?

Comment: First, people, who try competing with these machines for day trading, have less and less chances. Second, the most important that another intellectual human area is given up to computers along with chess and drug creation. What is next? Computers can became smarter than people in the next 20-30 years. They just need to learn how to write programs for themselves.

Human-Aided Computing

http://www.technologyreview.com/Infotech/18962/
Researchers at Microsoft are trying to tap into some of the specialized–and often subconscious–computing power in the human brain, and use it to solve problems that have so far been intractable for machines. Today it takes relatively large supercomputers many hours to recognize faces–something a human can do almost instantly. One application for this face-recognition technique could be to use it for quickly sorting snapshots from surveillance videos to find frames with faces and those without. This strategy could be useful for identifying other types of objects, such as dogs or cats, and different types of words. Subconscious brain power could therefore improve automated image search by preclassifying objects to help a computer more accurately identify pictures.

Comment: Now Microsoft evil guys 🙂 teach computers how to use our brains. If it will go further computers will not only become smarter (see previous comment), but also hungry for our brain power.

Forgetting helps you remember the important stuff

http://news-service.stanford.edu/news/2007/june6/memory-060607.html
Bruce Kuhl and Anthony Wagner at Standford University have discovered that the brain’s ability to suppress irrelevant memories makes it easier for humans to remember what’s really important. This function, which is carried out in the prefrontal cortex region behind the forehead, helps the brain; it doesn’t have to work as hard in the future when it tries to remember an important memory because the competing but irrelevant memories have been weakened.
Memory allows humans to be predictive about what’s likely to be relevant to them as they go through life, Wagner explained. “What forgetting does is allow the act of prediction to occur much more automatically, because you’ve gotten rid of competing but irrelevant predictions,” he said. “That’s very beneficial for a neural information processing system.”

Comment: I knew this before – brain automatically forget irrelevant memories! You can read and learn as much as you want, but you’ll forget most of it. It doesn’t make sense for us to learn all the language constructs syntax, all the classes in framework and read so many books. We’ll forget if it is irrelevant. Now the most important question is how to make this information relevant. I believe the best way to do this – create abstractions, actively aggregate in your head conceptual pictures or just practically use it.

Mind Control: Unwanted Thoughts

http://www.psychologytoday.com/rss/pto-20040204-000001.html
Now if you want to forget something 🙂
Trying hard not to think about something almost guarantees that it will pop up in your consciousness. When you are actively avoiding a thought, one part of your brain is busily working to keep the upsetting thought at bay. It’s searching out distractors—something else to focus on that will protect you from the idea you’re trying to avoid.

At the same time, another part of the mental machinery has to keep checking to make sure that the job’s being done properly. Inadvertently, this monitoring process calls attention to the unwanted thought, and makes you more vulnerable to the very ideas you’re fleeing from. Article gives 2 advices: make secret preoccupation open or do the opposite of the thing you want to do.

Comment: These facts show again power of our automatic brain systems over our conscious mind and importance of cooperating with unconscious mind. You can try 2 additional things when you build program and you have unwanted thoughts:

• don’t program at all – quality of your work will be low
• or find most interesting problem in the program and you’ll quickly forget your thoughts

New approaches to software development

http://www.scottberkun.com/blog/2007/asshole-driven-development/
Scott Berkun provides information about the new (or reinvented) systems for software project management: Asshole Driven development (ADD) , Cognitive Dissonance development (CDD), Cover Your Ass Engineering (CYAE) and many others.

No comments.

“Somebody has to pay for all this.” – Kent Beck

Why do software projects exist?

A rational self-interest is a foundation of economics – maximizing expected benefits and minimizing expected cost. Software is one of the important instruments in hands of businesses, organizations and individuals that helps to achieve both goals. We could make money developing software and we could become more efficient and reduce cost. Benefits are not only limited to profit and money as with government, science or open source volunteers.

Economic forces start, drive and end software projects. These forces are complex, unpredictable and demanding. And they are the main reason for existence of the software.

What are economic forces?

Economic forces operate on 3 levels – internal, market and macro. You can find below description and examples of these forces.
Read full post >>

Software is everywhere: inside our computers, cars, phones and even toasters. Software tells these devices what to do.

Everybody can develop software. Hundreds of millions do. We use similar skills as in writing a cooking recipe or telling a friend how to find a shopping mall – we just need to come up with the set of instructions. Basic logic and knowledge of instruction language is enough. You don’t need to have a computer science degree or even finish courses to become a good programmer.

Does it sound simple? Creation of a program should be a routine job now as growing potatoes or building a bridge. And we have at least 2 good reasons to hope for this:

• We had more than 60 years to learn how to properly instruct computers what to do. The first computer ENIAC was built in 1946.
• Today 19 millions of professional programmers in the world and multi-billion industry are very serious about effectively building good software .

And still, software creation is unpredictable, unreliable and often fails.

Why? We should understand better what is software development.

Software development is the translation of a user need or marketing goal into a software product. – Wikipedia

We can add more to this definition.
Read full post >>