“The important thing is not to stop questioning” – Albert Einstein

A typical programmer faces thousands lines of code, huge number of details and millions of situations and states during the software system execution. A programmer should understand the system, know how to modify it and support knowledge about the system with explanations, justifications and answers. How can a software developer gain, maintain and operate this knowledge and make sense of these volumes of information and complex logic changing every day?

The ways how our memory works could give us few hints.

Our Human Memory Architecture

Read full post >>

“Variety is the spice of life”

Did you see a software team with members that are

• always stressed and full of fear to fail or completely indifferent to end results?
• totally agree with each other and satisfied with their product, but very far from harsh reality?
• competing and fighting for the true solution, but cannot achieve common goals?
• always inventing something interesting, but cannot focus on work and deliver software?
• overly disciplined, planning everything, but deliver mediocre boring results without any spark of imagination?
• having excellent communication, rapport and environment, but spend most of the time talking than working?
• sitting in separate cubicles without talking, barely understanding what others are doing?

Can these teams be successful? What is the root cause of their problems? How these problems could be prevented?

Our personalities

One of the most interesting things about humans is their personality. We are different in behavior, reactions and feelings in the similar situations. These wonderful personality traits make our life refreshing and sometimes totally unpredictable. Do they matter in software development, apparently logical and rational activity? I believe that people are the most important factor in the software development and they determine success. A programmer’s personality affects behavior and therefore it must have significant impact on the work itself.

Empirical research shows five broad dimensions of personality:

• Openness – appreciation for art, emotion, adventure, unusual ideas, imagination, curiosity, and variety of experience.
• Conscientiousness – a tendency to show self-discipline, act dutifully, and aim for achievement; planned rather than spontaneous behavior.
• Extroversion – energy, positive emotions, and the tendency to seek stimulation and the company of others.
• Agreeableness – a tendency to be compassionate and cooperative rather than suspicious and antagonistic towards others.
• Neuroticism – a tendency to experience unpleasant emotions easily, such as anger, anxiety, depression, or vulnerability; sometimes called emotional instability.

What are the best personal traits for the programmer? Before answering this question, I want to mention research of rudimentary personality traits in wild animals. Small birds, great tits, have personality ranging from highly exploratory to more cautious one. Scientists found that the both personality types are optimal, but under different conditions. When resources were scarce it paid the birds to dispense further. However, when resources are abundant more cautious birds were more likely to survive. Morale is that our evolution preserved different personalities for the good reason. They help us to survive and advance under various unpredictable conditions. They made overall population resilient to changing environment, threats and challenges.

Software teams need diversity not only in skills, experience and knowledge. They need the optimal balance of personalities to effectively build software, especially in our rapidly changing technological, business and social worlds.

Read full post >>

“Ruin in community is not caused by witches with broomstick but it’s something that starts in people’s heads.” – Prof. Preobrazhensky (Michail Bulgakov’s “The heart of a dog”)

The newborn software system looks beautiful inside after the few weeks of development. Several talented programmers put their souls together into this amazing software idea. But week after week the system become older and uglier. It could still look exciting for users, but the system rots inside. It is much more difficult to modify, it breaks often, development time and cost soars. Programmers become scared to work with it. The system becomes Frankenstein. Why could it happen?

Three reasons

There are hundred objective reasons why it could happen – complex technology, changing customer needs, management pressure, time to market and many more. But some teams still deliver a good system under these conditions. And some teams cannot deliver even in the most favorable conditions. Why?

Degradation of the system starts in the people heads and there are three main reasons:

1. Incomprehension – developers don’t understand the purpose, ideas, design or technology behind the system
2. Inarticulateness – developers unable to express ideas through clear and effective architecture, design and code
3. Inconsistency – developers cannot act consistently or don’t care about the system

Read full post >>

Thought-provoking and interesting author Nicholas Carr has published a new article in Financial Times, where he expands on his ideas about transformation of IT services into utility. He compares IT with electric power generation 100 years ago:

Like data-processing today, power generation was assumed to be an intrinsic part of doing business. But with the invention of the alternating-current electric grid at the turn of the century, that assumption was overturned.

Suddenly, manufacturers did not have to be in the power-generation business. They could run machines with electric current generated in distant power plants by big utilities and fed to their factories over a network of wires. With remarkable speed, the new utilities took over the supply of industrial power. Scores of private power stations were dismantled.

Read full post >>

New Scientist published three interesting articles related to the human psychology. This post tries to apply these findings to the programmer’s mind.

Should a programmer trust gut instinct

In the article The subconscious mind: Your unsung hero, author argues that contrary to common beliefs non-conscious thinking is better in some cases than rational, conscious thought:

• where people have to make difficult choices based on large amounts of hard-to-assess information. Subconscious thought processes allow us to integrate complex information in a more holistic way
• in problem solving, verbalizing what they are doing has no effect on people’s ability to solve analytical, mathematical or logic problems but actually hinders performance on insight problems, such as solving a riddle
• subconscious thinking is the source of our inspiration – it is central to creativity. In highly creative people subconscious information is more likely to overspill into consciousness, giving them richer mental resources from which to make creative connections

Creativity and problem-solving are required for any descent programmer. This article shows that these skills are dependent on quality of our subconscious thinking – gut instinct. Can we consciously improve our subconscious thinking? Yes, and the best way is getting experience and making many mistakes. Certainly, it will work only if you learn from them. As famous quote says: “Good judgment comes from experience. Experience comes from bad judgment.” Especially it is true in the software development.

Short Summary: trust and develop your gut instinct.

How a programmer could keep focus

Many of us struggle with distractions during our work. They really hurt our productivity. Psychologist Nilli Lavie says in the article Focus feature: Keeping your mind on the job:

You don’t have voluntary control – you can’t say ‘stop’ to a ‘distracter’. If you wish to ignore something, that doesn’t mean that you will succeed.

Surprisingly, he found that when a more visually intensive task “loads” the brain’s attention, we become increasingly blind to distractions, and our performance on the task will improve: reaction times get faster, and error rates drop. That means that the harder you are forced to concentrate, the less likely you are to be distracted.

Therefore, conclusion is simple for programmers – a challenging task will force you to ignore distractions, a boring task will make you very susceptible to them.

Short summary: load your brain to avoid distractions

Procrastination in programmer’s life

Another productivity killer is procrastination. Piers Steel at the University of Calgary in Alberta found a formula for procrastination:

Utility = ^{E x V}/_{G x D}
This calculates how likely you are to do something immediately – the task’s utility – by taking into account the four key variables: how confident you are of succeeding in the task (E); how pleasant you perceive the task to be (V); how easily distracted you are (G); and how much time will elapse before the reward for completing the task arrives (D).

He suggests strategies to cope:

• Make a firm commitment to your boss or partner to finish a task by a certain time.
• Strip your workspace of all distractions.
• Get a good night’s sleep and try tackling the most unpleasant and difficult tasks early in the day.
• Set a series of realistic goals.
• Promise yourself a reward for each goal that you meet.
• Believe in yourself.
• Outsource your motivation. Get someone else to regularly goad you into action.

I can promise – you’ll find even more strategies in various sources. I had a related post too: 5 steps to cooperate with you unconscious mind. But after all, you’ll get results only if you have desire to improve and change yourself. As Plato said: “The first and the best victory is to conquer self.” Chinese wisdom adds: “A journey of a thousand miles must begin with a single step.”

Short summary: it is possible to fight procrastination… if you really want

Links (unfortunately, all require New Scientist subscription):
The subconscious mind: Your unsung hero
Focus feature: Keeping your mind on the job
Procrastination: The thief of time

davisayer

Thomas J. Bing and Edward F. Redish of University of Maryland analyzed how physics students work on assignments using computer programs:

Symbolic calculators like Mathematica are becoming more commonplace among upper level physics students. The presence of such a powerful calculator can couple strongly to the type of mathematical reasoning students employ… Mathematica plays an active role in focusing and sustaining their thought around calculation. These students still engage in powerful mathematical reasoning while they calculate but struggle because of the narrowed breadth of their thinking. Their reasoning is drawn into local attractors where they look to calculation schemes to resolve questions instead of, for example, mapping the mathematics to the physical system at hand.

These computer tools affect thinking and how students frame and focus their work.

Can we find similarity with the way tools affect programmer’s mindset ? In essence, software programming is operation on ideas and knowledge. We translate people ideas into the source code. But we cannot do it effectively without programming platform and tools – editors, compilers, debuggers and may others. And they have profound effect how we approach and solve customer needs. Our platforms and tools could frame and alter our thinking. PHP, Visual Basic, Java, .NET, Ruby On Rails programmers with the same level of experience will build not only technologically different solutions, but in the most cases solve problems different ways.

Programmer’s platforms and tools affect our mindset in three ways:

• Tools could prevent normal flow of idea implementation. Inconvenient tools, cumbersome environments and long feedback cycles stagnate productivity and lead us to the path of the least resistance, often far from optimal.
• Tools could make ineffective implementation strategies more attractive, affecting how we frame customer ideas. For example, UI-oriented programming environments could shift programmer’s mind to think about the problem in terms of screen elements and UI events (Smart UI) instead of reflecting in the code domain concepts, relations and ideas (Domain Driven Design).
• Tools could shift focus to the implementation perspective from the big picture and broader context of the problem at hand. Narrow focus on implementation and technology has negative impact. Often redefinition of the problem, analysis of alternatives or refining of concepts could bring much better results. Inexperienced programmers (and even experienced) often get into this trap, concentrating on tools the same way as novice drivers are more concentrated on the handling a car instead of observing the road and finding the better way.

What qualities should effective programmer tools have?

• Usability – enhance flow of programmer’s ideas or at least don’t impede and interrupt this flow. Tools should be convenient, easy-to-use and aligned with the way programmers work. Programmers should put most of their mental energy to think about customer ideas and much less how to deal with these tools.
• Representation – enable easy for understanding and modification representation of the structure, ideas and domain concepts in the code. Tools should provide a convenient solution view, organization and navigation for understanding, parsing and manipulation with components, classes, relations and concepts. Modern source-based project organization could benefit from tools like Resharper. Martin Fowler blogged about more radical approaches as repository-based code and Language Workbenches. UML, MDA and code auto-generation tools didn’t prove to be useful for effective code representation.
• Agile development friendly – support evolutionary design, short delivery cycles, refactoring, unit testing and other agile practices.

What do you think? What are the best tools for the programmer to enhance concentration on the problem, unobtrusive flow of thoughts and effective implementation of ideas?

Link: Symbolic Manipulators Affect Mathematical Mindsets, Thomas J. Bing, Edward F. Redish

nuanc

Cross Talk complains that the selection of Java as the first programming language in Computer Science education weakens programmers skills and lead to decline in qualification.

The irresistible beauty of programming consists in the reduction of complex formal processes to a very small set of primitive operations. Java, instead of exposing this beauty, encourages the programmer to approach problem-solving like a plumber in a hardware store: by rummaging through a multitude of drawers (i.e. packages) we will end up finding some gadget (i.e. class) that does roughly what we want. How it does it is not interesting! The result is a student who knows how to put a simple program together, but does not know how to program. A further pitfall of the early use of Java libraries and frameworks is that it is impossible for the student to develop a sense of the run-time cost of what is written because it is extremely hard to know what any method call will eventually execute.

Authors noticed other worrisome trends in teaching software professionals:

1. Mathematics requirements in CS programs are shrinking.
2. The development of programming skills in several languages is giving way to cookbook approaches using large libraries and special-purpose packages.
3. The resulting set of skills is insufficient for today’s software industry (in particular for safety and security purposes) and, unfortunately, matches well what the outsourcing industry can offer. We are training easily replaceable professionals.

I want to look into this problem from the broader perspective: what is the best way to become a good programmer for a beginner? There are two fundamental ways:

1. Strong theoretical foundation: learn applied math, theories of algorithms and computational complexity, read Knuth’s The Art of Computer Programming, solve problems with Turing machine, etc.
2. Shu Ha Ri – learning by experience: learn programming by solving practical problems with modern languages and platforms (Java, .Net, Ruby On Rails, PHP). Fundamental knowledge emerges from experience in solving real problems.

Definitely, there are some domains that require and intensively use CS theoretical knowledge: science, engineering, hardware and system programming. However, majority of software professionals build software for solving business, personal and other non-rocket science problems. I know a PHP programmer, who doesn’t know how to write a sort algorithm (without looking somewhere), but still have built several successful web applications. And I’m confident that there are strong computer science theorists who cannot write easy-to-use programs and make their customers happy.

So, what is the best approach? I believe any beginner should excel in following three areas:

1. Train clear logical thinking. The ultimate end result of any programming is a program (I hope it is not a surprise), which is a set of instructions for a computer. You cannot write instructions if you don’t know how to achieve end result. Any language which allows clear procedural expression of logic is fine for start (e.g. Pascal). I would add object-oriented and functional languages on later stages (e.g. Ruby and Lisp).
2. Understand modern software concepts and environments: computer architecture, networking, web, mobile, concurrent programming, cloud computing, etc. High level knowledge will give better orientation, starting points and overall understanding of software systems mechanics.
3. Learn to effectively implement customer needs. Happy customers and useful software is the end result of any healthy software project. Learning evolutionary, agile development and user experience (including communication and psychology) practices is essential for the effective software developer.

I believe that software development is empirical in nature. Math and computer science theories are great, but in many case they will be only brain exercises without practical results, especially for beginners. On the contrary, beginners will become excellent master programmers by succeeding in discussed above three fundamental areas and relentlessly learning from experience in applying them.

What do you think?

Links:
Computer Science Education: Where Are the Software Engineers of Tomorrow?, Dr. Robert B.K. Dewar and Dr. Edmond Schonberg
Is Software Development Empirical or Rational?

Can we say that clearing house from rubble and debris after home repair is a necessary waste for construction workers? Can we say that making design of the car that could be easily supported and repaired by mechanics is a necessary waste for engineers? Can we say that making ideas clear, easy to follow and understand is a necessary waste for the influential writer?

Amr Elssamadisy posted an interesting opinion on InfoQ about refactoring:

… there are two types of waste in Lean: pure waste, and necessary waste. Pure waste is one that has no value to either the team building the software or the customer using the software. Necessary waste, on the other hand, is the best way we currently know how to do a job even if it does not have customer value. Refactoring is clearly an instance of the latter.

Amr tries to make an important point – minimize refactoring that doesn’t contribute directly to customer requirements under development. It is a reasonable point, but I don’t like word the ‘waste’ and belittling of one of the most important practice of the modern developer. This post sparked a very interesting discussion around value of refactoring with many insightful comments (including Kent Beck and Tom Poppendieck).

We, software professionals, sometimes forget that source code is the most important representation of our ideas about how software should work and what it actually does. In the essence, software development is communication, processing and coding of people ideas. We substantially reduce value of our software if we leave messy, convoluted and difficult to understand software ideas, though correctly implemented. This code will waste time and effort of people who evolves and supports our software (including ourselves). And therefore, it will reduce value of the software for the customers who expects that it will be well supported, easy to change and become better over time. Refactoring is the practice that forces us to improve code and represent software ideas better.

Certainly, as with any complex human activity, we should learn when, how and why we should do refactoring. We should learn how to balance refactoring and development of the new features – software consolidation and expansion phases (read an excellent article from Brian Foote and William F. Opdyke). Refactoring should be not only targeted to reduce technical debt, but also to reduce comprehension debt – understanding, refinement and enhancement of implemented ideas. Finally, refactoring is the essential part of our professional mastery, which distinguish ace programmers from amateurs or indifferent hirelings.

Dear readers,

I wish you happy, joyful and prosperous New Year! I sincerely wish and hope that in the New Year all your software projects will be in excellent shape, your customers will be very satisfied and you will feel as a happy, inspired and successful Software Creator the whole year!

2007 is the first year of this blog. And I still have to learn a lot to make the blog better and really useful for you. I want to highlight few articles that I believe were most interesting this year:

• Programmers are lazy capricious pseudo-intellectuals. Really? – why some management consultants could think that programmers are sloppy, disorganized and have hyper-inflated self-importance.
• How to Be Happy At Work. Short tutorial. – most jobs are not created to make us happy – what can we do ourselves to be happy at work?
• What is The Best Leadership Style for The Software Team? – review of four major leadership styles for software teams with pros and cons
• Do We Need Software Architects? 10 Reasons Why Not – 10 reasons why we don’t need non-programming decision-making software architects
• Evolutionary Software Architecture or Why Developers Are Not Janitors – what is effective architecture and overview of complex self-organized systems in this context
• Guide to Job Security for Software Developers: 15 Sure-Fire Methods – few strategies how to cope with job security – some are controversial 🙂
• Can Computers Beat Human Programmers? – 5 posts discuss if computers could replace human programmers
• Software Development is The Flow of Ideas. The Rest is Secondary – idea flow is one of the most important aspect of the software development
• The Mind of the Programmer. Anatomy and 3 Contradictions – uncovering contradictory biological mechanism behind human (and programmers) mind
• Software Requirements Are Elusive: 6 Reasons Why Customers Cannot Get Them Right – 6 psychological reasons why customers cannot predict what they will need from the software in the future
• Lost Personalities: How our company alters us – investigation how other people change our minds and make us behave differently

I write about everything what is interesting for me and can help to understand better people, systems and software development in overall. But certainly, it is not enough to make this blog interesting for you without your feedback. I want to take the opportunity to ask three questions:

• What did you like and find interesting in this blog?
• How can I make it better?
• What do you want to see here in the new year?

Thank you for your interest, patience and participation. Have great holidays and the rest of year!

We can consider software development as the translation of our theories, concepts and ideas into the machine code. Therefore, how we gain, evolve and verify this knowledge is very important for building successful software. There are two main views how we gain the knowledge: Rationalism and Empiricism.

Rationalists appeal to reason as a source of knowledge or justification. They claim that our concepts and knowledge are gained a priori, independently of experience. Empiricists claim that experience is the ultimate source of all our concepts and knowledge. Rationalists come up with a theory and look for the evidence to support it. Empiricists assembles evidence, and builds on the evidence to form a theory.

The main question for the both camps: How do we know that our theory is true knowledge and not a guess?

Rationalism and empiricism have various success in different areas:

• Mathematics is triumph of rationalists – all the complex branches and theories are deducted logically from few intuitive basic premises.
• Natural sciences, which are dealing with dynamic complex systems combine rationalism and empiricism using scientific method: empirical observations and experiments with rational hypotheses and predictions. However, empiricism has the main emphasis – observational experiments are necessary to understand the physical universe.
• Social sciences, politics, economics show inability to come up with reliable rational theories. However, empiricism in these areas has own limitations as it is difficult to make comprehensive social experiments and deduct from them (and history) any true knowledge and theory about human groups and their societies. Human groups are complex, unpredictable and often irrational systems. They don’t behave consistently and don’t submit to permanent laws as physical world does.

What view does work better for software development?

How do these views show up in software development? Pure rationalists would prefer developing complete concepts and knowledge behind the software system using intuition, logic and creative insights before start of programming. Pure empiricists would immediately dive into experiments, programming and getting early practical results and build their knowledge based on this experience.

Read full post >>