The main architecture goal is to strengthen and align system ideas, muscles and structures in areas most impacted by stress, changes and expansion.

In short, architecture covers every important aspect of the software system.

A good thoughtful software developer is torn apart by conflicting approaches for architecting complex software system:

1. Heavy upfront system design leads to rigid, costly and over-engineered solutions
2. Ignored business and technical perspectives cause business irrelevance or failure
3. Poor reliability, performance or flaws could kill promising beautiful software system

What is the best approach to build a sound and successful system?

Problems

Let’s consider several architecture challenges that a software team faces during system development.

1. Idealism (assumed known knowns) – mostly theoretical views, lack of practical experience, illusions. We often exaggerate our intellectual abilities to find right solution theoretically.
2. Open problems (known unknowns) – problems or needs without foreseeable solution. These evil inconvenient unknowns could seriously impact the course of the project
3. Uncertainty (unknown unknowns) – low predictability of the future in technology, business and Universe. Many good decisions could turn to be wrong after some time.
4. Over-complication – over-engineering for domain problems because of over-thinking, misunderstanding, lack of information or technical enthusiasm. Over-complication steals development time and increase cost and complexity.
5. Fragility – poor system quality (performance, reliability, uptime, security, etc.). Neglecting quality could cost reputation and harm system success.
6. Complexity – complex structures, unpredictable behavior and states that turn a system into the Ball of Mud.
7. Inconsistency – dissonance of the system approaches and parts; fragmented and incomplete knowledge in heads. Development teams produce incoherent system without coordination of ideas and implementation.
8. Inertia– difficulties to introduce disruptive changes (for business, users, operations). People and established processes resist changes and have inertia that is difficult to overcome.
9. Misalignment – with business goals, existing systems, infrastructure. Building a software system without considering business vision, existing IT landscape is the path to disaster.

Three Architecture Answers

There are 3 useful approaches to Software Architecture that address these challenges:

1. Strategic
2. Technical
3. Evolutionary

Read full post >>

The software system in the period of active growth is a really wild beast. Excited developers with creative minds and feature obsessed marketers consistently add the fuel to this fire of software creation.

Uncontrollable growth, race for features and engineering wonders sometimes give rise to a monster – a software system bloated with useless features, over-engineered internals and erratic behavior.

Read full post >>

A really great talent finds its happiness in execution. – Johann Wolfgang von Goethe

source

Qualities of well composed code:

1. Quick discovery and understanding of programming logic and components
2. Clear organization (for human brains)
3. Ease of reuse, modification and evolution
4. Close connection between customer ideas and system implementation

Read full post >>

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. – Antoine De Saint Exupery

The approach to programming is concerned with finding the best ways to translate programmer’s intention into the good system design and code.

The programming is communication. The programmer continuously add, change and refine ideas in the code. Source code has two important goals: tell a computer what to do and tell people what the computer should do. The program code is the only true medium for storing and communicating ideas about the software system behavior. Quality of the ideas expression in the code directly affects overall quality of the system.

So, what are characteristics of the good code?

1. clear – easier to work with ideas;
2. minimal – less effort to understand and change ideas;
3. testable – easier to validate ideas.

These are 6 top reasons for bad design and code:

1. lack of expertise
2. unrestrained technical curiosity and creativity
3. missing big picture: system purpose and customer goals
4. blindly following popular methods and over-using technology
5. sloppiness; lack of attention to details
6. over-complicating design to have more work or increase job security

The programmer can write better code (and avoid most of these problems) by improving programming style and approach.

Read full post >>

From the first sight biological evolution looks too unpredictable to have any value for the constrained software development. We just don’t have time, money and resources for these wild experiment, unlimited trials and errors. It really seems that Nature could learn from us how to make things fast and effectively.

However, there are some principles that helped evolution to come up with amazing and efficient designs that made life flourishing on the Earth. This post will explore what software development could learn from biological evolution. See my previous post for the review of evolution concepts and mechanisms and how they could be applied to software development.

Read full post >>

I can’t wait to share this simple secret with you right now.

The Secret: Effective Software Systems are the systems that easy to understand and operate with human brains.

Programmers are more productive with effective software systems. Programmers can better learn and grow these system. Programmers have less problems, work faster and make better decision with them.

Now, you can avoid spending time reading this post if you already know this secret and you know how to avoid building the software system that:

• almost impossible to understand in reasonable time
• has confusing and convoluted swamp of logic and structure
• scary to change as nobody has any clue what will be broken, but sure that it will be broken

If you are still interested, lets find out what makes software systems effective.

Software Development is a pure mental endeavor (except typing on keyboard) that includes 3 main activities:

• Understand – learn and know system concepts and implementation
• Evolve – build, modify and support growth of the system ideas in the code
• Share – communicate and exchange ideas about the system

Programmers should care about 7 areas to make the system better suited for our brains:

1. Knowledge Creation and Retention – parsing, memorization and comprehension of the system ideas
2. System Organization – elements, relations and structure in the system
3. Sustaining Emerging Order – support evolution of the system and gain control over chaos
4. Minimize Noise and Purify – avoid adding unnecessary stuff to the system
5. System Discovery and Learning – making sense of the system
6. Mental Models – our internal explanations for how things are working in the real system
7. Shared Knowledge – ideas exchange, reconciliation of opinions and creation of mutually enhanced knowledge.

Read full post >>

Can we say that clearing house from rubble and debris after home repair is a necessary waste for construction workers? Can we say that making design of the car that could be easily supported and repaired by mechanics is a necessary waste for engineers? Can we say that making ideas clear, easy to follow and understand is a necessary waste for the influential writer?

Amr Elssamadisy posted an interesting opinion on InfoQ about refactoring:

… there are two types of waste in Lean: pure waste, and necessary waste. Pure waste is one that has no value to either the team building the software or the customer using the software. Necessary waste, on the other hand, is the best way we currently know how to do a job even if it does not have customer value. Refactoring is clearly an instance of the latter.

Amr tries to make an important point – minimize refactoring that doesn’t contribute directly to customer requirements under development. It is a reasonable point, but I don’t like word the ‘waste’ and belittling of one of the most important practice of the modern developer. This post sparked a very interesting discussion around value of refactoring with many insightful comments (including Kent Beck and Tom Poppendieck).

We, software professionals, sometimes forget that source code is the most important representation of our ideas about how software should work and what it actually does. In the essence, software development is communication, processing and coding of people ideas. We substantially reduce value of our software if we leave messy, convoluted and difficult to understand software ideas, though correctly implemented. This code will waste time and effort of people who evolves and supports our software (including ourselves). And therefore, it will reduce value of the software for the customers who expects that it will be well supported, easy to change and become better over time. Refactoring is the practice that forces us to improve code and represent software ideas better.

Certainly, as with any complex human activity, we should learn when, how and why we should do refactoring. We should learn how to balance refactoring and development of the new features – software consolidation and expansion phases (read an excellent article from Brian Foote and William F. Opdyke). Refactoring should be not only targeted to reduce technical debt, but also to reduce comprehension debt – understanding, refinement and enhancement of implemented ideas. Finally, refactoring is the essential part of our professional mastery, which distinguish ace programmers from amateurs or indifferent hirelings.

Opinions about developers varied from janitors to “prima donnas” in comments to my previous post Do We Need Software Architects? 10 Reasons Why Not .
Beside discussion about the role of the software architects, the underlying philosophical problem is whether software development is primarily top down (centralized and planned) or bottom up (emergent and adaptive) process. If software development is top down, the architects are essential and crucial people on the project, who concentrate knowledge, establish technical leadership and guide development teams. If software development is bottom up, the developers become primary force for evolving the system, make key technical decisions and care about the architecture; architects (if they still needed) play coordination and mediation roles.

As Goethe said – between two opposite opinions you’ll find not the truth, but the problem. I incline to consider software development as a bottom up process, which occasionally (often in time of changing direction or crisis) needs centralized effort and top down approach.

Evolutionary, adaptive and emergent development of the software system leads to the most optimal solution. However, any software project usually has specific business goal, constraints and cannot evolve forever as natural systems do. Therefore, the main architecture concern is how to balance these two approaches.

Dynamic of the Complex Systems

Complex Adaptive System

Many complex systems show organization – galaxies, biological, market, society, etc. These systems can be explained and studied by referencing their parts, properties and laws (gravitational, supply / demand, etc.). Another approach is to look for the system as a whole, studying dynamic of the elements interaction and system properties – the science of self-organization.

Can we say that the development of the software system has features of self-organization? Mostly yes, as we have

• Fluctuations – optimal solution is not obvious and needs search; many useful vs. noise factors affect this search
• Local interactions – the software solution emerge from local interactions between involved people, business needs, technical platform and IT environment
• Dissipation – a software project consumes energy to keep going, mostly in the form of money 🙂
• Instability – software development deals with constantly changing situations
• Multiple equilibria – there are many possible satisfying solutions
• Complexity – any non-trivial software project has complexity
• Hierarchies – there are many perspectives and levels – organizational, technological and solution domain

The only fundamental property, which sometimes is missing for true self-organization – autonomy or absence of external control. Traditional corporation will always exercise some form of the control over software projects. And this is important to resolve – self-organized systems (evolving bottom up) achieve better solutions than rigid controlled systems. But why self-organized systems can produce better solutions? And what is the best way to control them?

Read full post >>

Recently I was reviewing requirements for Microsoft Architect Certification and had found description for architect roles. After reading I had a question: do we need this kind of architects in a software team? Can they really help with building successful software?

The summary of architect responsibilities (as per Microsoft):

• Enterprise architects: set the overall vision and framework for the IT environment from a strategic business perspective.
• Solutions architects: design the solution to take advantage of the existing assets, integrate them into the existing environment, follow the enterprise architecture, and solve the business problems of the business owner or unit.
• Infrastructure architects: responsible for creating an architecture that meets the business and service level agreement requirements of the business owners and supports the applications and solutions that are required to operate their day-to-day businesses.

The bottom line is that company hires expensive and canny software architects before start of the project. They talk with the business, come up with solutions and make bright key decisions about architecture. But they don’t program. During the course of the project they ensure that developers don’t spoil this great architecture.

Now business owners can hire not so bright programmers, who should just follow directives and implement this great architecture. There is even correspondence with theories of some management consultants that programmers are just house painters.

Completely irrelevant questions: Can you trust advise of your friend about Paris, if he saw the city only on TV? Can you trust battle plans if battle didn’t begin and your commander knows little about enemy? How many chess moves ahead can you predict with high certainty before start of the game?

I don’t reject a need in architecture, but I don’t agree who, when and how should make architecture decisions in a software project.

Read full post >>