I can’t wait to share this simple secret with you right now.

The Secret: Effective Software Systems are the systems that easy to understand and operate with human brains.

Programmers are more productive with effective software systems. Programmers can better learn and grow these system. Programmers have less problems, work faster and make better decision with them.

Now, you can avoid spending time reading this post if you already know this secret and you know how to avoid building the software system that:

• almost impossible to understand in reasonable time
  
• has confusing and convoluted swamp of logic and structure
• scary to change as nobody has any clue what will be broken, but sure that it will be broken
  

If you are still interested, lets find out what makes software systems effective.

Software Development is a pure mental endeavor (except typing on keyboard) that includes 3 main activities:

• Understand - learn and know system concepts and implementation
• Evolve - build, modify and support growth of the system ideas in the code
  
• Share - communicate and exchange ideas about the system

Programmers should care about 7 areas to make the system better suited for our brains:

1. Knowledge Creation and Retention - parsing, memorization and comprehension of the system ideas
   
2. System Organization - elements, relations and structure in the system
   
3. Sustaining Emerging Order - support evolution of the system and gain control over chaos
   
4. Minimize Noise and Purify - avoid adding unnecessary stuff to the system
   
5. System Discovery and Learning - making sense of the system
   
6. Mental Models - our internal explanations for how things are working in the real system
   
7. Shared Knowledge - ideas exchange, reconciliation of opinions and creation of mutually enhanced knowledge.

Read full post >>

Can we say that clearing house from rubble and debris after home repair is a necessary waste for construction workers? Can we say that making design of the car that could be easily supported and repaired by mechanics is a necessary waste for engineers? Can we say that making ideas clear, easy to follow and understand is a necessary waste for the influential writer?

Amr Elssamadisy posted an interesting opinion on InfoQ about refactoring:

… there are two types of waste in Lean: pure waste, and necessary waste. Pure waste is one that has no value to either the team building the software or the customer using the software. Necessary waste, on the other hand, is the best way we currently know how to do a job even if it does not have customer value. Refactoring is clearly an instance of the latter.

Amr tries to make an important point - minimize refactoring that doesn’t contribute directly to customer requirements under development. It is a reasonable point, but I don’t like word the ‘waste’ and belittling of one of the most important practice of the modern developer. This post sparked a very interesting discussion around value of refactoring with many insightful comments (including Kent Beck and Tom Poppendieck).

We, software professionals, sometimes forget that source code is the most important representation of our ideas about how software should work and what it actually does. In the essence, software development is communication, processing and coding of people ideas. We substantially reduce value of our software if we leave messy, convoluted and difficult to understand software ideas, though correctly implemented. This code will waste time and effort of people who evolves and supports our software (including ourselves). And therefore, it will reduce value of the software for the customers who expects that it will be well supported, easy to change and become better over time. Refactoring is the practice that forces us to improve code and represent software ideas better.

Certainly, as with any complex human activity, we should learn when, how and why we should do refactoring. We should learn how to balance refactoring and development of the new features - software consolidation and expansion phases (read an excellent article from Brian Foote and William F. Opdyke). Refactoring should be not only targeted to reduce technical debt, but also to reduce comprehension debt - understanding, refinement and enhancement of implemented ideas. Finally, refactoring is the essential part of our professional mastery, which distinguish ace programmers from amateurs or indifferent hirelings.

Opinions about developers varied from janitors to “prima donnas” in comments to my previous post Do We Need Software Architects? 10 Reasons Why Not .
Beside discussion about the role of the software architects, the underlying philosophical problem is whether software development is primarily top down (centralized and planned) or bottom up (emergent and adaptive) process. If software development is top down, the architects are essential and crucial people on the project, who concentrate knowledge, establish technical leadership and guide development teams. If software development is bottom up, the developers become primary force for evolving the system, make key technical decisions and care about the architecture; architects (if they still needed) play coordination and mediation roles.

As Goethe said - between two opposite opinions you’ll find not the truth, but the problem. I incline to consider software development as a bottom up process, which occasionally (often in time of changing direction or crisis) needs centralized effort and top down approach.

Evolutionary, adaptive and emergent development of the software system leads to the most optimal solution. However, any software project usually has specific business goal, constraints and cannot evolve forever as natural systems do. Therefore, the main architecture concern is how to balance these two approaches.

Dynamic of the Complex Systems

Complex Adaptive System

Many complex systems show organization - galaxies, biological, market, society, etc. These systems can be explained and studied by referencing their parts, properties and laws (gravitational, supply / demand, etc.). Another approach is to look for the system as a whole, studying dynamic of the elements interaction and system properties - the science of self-organization.

Can we say that the development of the software system has features of self-organization? Mostly yes, as we have

• Fluctuations - optimal solution is not obvious and needs search; many useful vs. noise factors affect this search
• Local interactions - the software solution emerge from local interactions between involved people, business needs, technical platform and IT environment
• Dissipation - a software project consumes energy to keep going, mostly in the form of money :)
• Instability - software development deals with constantly changing situations
• Multiple equilibria - there are many possible satisfying solutions
• Complexity - any non-trivial software project has complexity
• Hierarchies - there are many perspectives and levels - organizational, technological and solution domain

The only fundamental property, which sometimes is missing for true self-organization - autonomy or absence of external control. Traditional corporation will always exercise some form of the control over software projects. And this is important to resolve - self-organized systems (evolving bottom up) achieve better solutions than rigid controlled systems. But why self-organized systems can produce better solutions? And what is the best way to control them?

Read full post >>

Recently I was reviewing requirements for Microsoft Architect Certification and had found description for architect roles. After reading I had a question: do we need this kind of architects in a software team? Can they really help with building successful software?

The summary of architect responsibilities (as per Microsoft):

• Enterprise architects: set the overall vision and framework for the IT environment from a strategic business perspective.
• Solutions architects: design the solution to take advantage of the existing assets, integrate them into the existing environment, follow the enterprise architecture, and solve the business problems of the business owner or unit.
• Infrastructure architects: responsible for creating an architecture that meets the business and service level agreement requirements of the business owners and supports the applications and solutions that are required to operate their day-to-day businesses.

The bottom line is that company hires expensive and canny software architects before start of the project. They talk with the business, come up with solutions and make bright key decisions about architecture. But they don’t program. During the course of the project they ensure that developers don’t spoil this great architecture.

Now business owners can hire not so bright programmers, who should just follow directives and implement this great architecture. There is even correspondence with theories of some management consultants that programmers are just house painters.

Completely irrelevant questions: Can you trust advise of your friend about Paris, if he saw the city only on TV? Can you trust battle plans if battle didn’t begin and your commander knows little about enemy? How many chess moves ahead can you predict with high certainty before start of the game?

I don’t reject a need in architecture, but I don’t agree who, when and how should make architecture decisions in a software project.

Read full post >>