Only entropy comes easy. – Anton Chekhov

What are the systems in software creation?

System is a group of interacting, interrelated, or interdependent elements forming a complex whole. – The American Heritage Dictionary

There are some powerful forces in our Universe that lead systems to more complex organization – from elementary particles to molecules, from space dust to stars and planets, from chemical elements to biological organisms and from early tribes to advanced civilization. Even more powerful forces lead systems to destruction or degradation back to primitive forms or complete mess.

System evolution is the constant fight between order (or more complex organization) and entropy (or degradation) of the systems.

System forces are working on 2 levels of the software creation:

• People, how they organized and the way they build software
• The software system itself

Both these levels intervene and affect each other to deliver desired result – a working software system. These levels are different – people systems are the product of biological, social and economic forces. Software systems are the product of technology, mental concepts and theories. But both of them share common challenges related to the system forces.

What are the system forces and challenges?

Structural

• Centralization vs. Self-Organization – which systems are more effective – directed (e.g. traditional corporation) or self-organized (e.g. classical market). Compare large centralized development of Windows with the effort of independent developers to build open-source Linux.
• Complexity vs. Simplicity – what should be a degree of internal complexity which enables complex behavior while maintaining consistency and integrity in the system. Compare collaboration software: simplicity and elegance of Basecamp and complexity and power of Sharepoint.
• Abstraction vs. Concrete – what should be the levels of abstraction in the system to allow most effective perspectives on structure, interactions and relations. Compare multi-layer perspective on enterprise Java system with the flat PHP solution containing all the logic in template files.
• Diversity vs. Similarity – how different should be elements to allow rich behavior while avoiding conflicts and maintaining the same direction and progress. Compare diverse team with different skills, backgrounds and opinions with single-minded and compliant team.

Behavioral

• Adaptation vs. Stability – how to balance adaptation to the new unexpected changes and stability to sustain disturbances and keep working what is good. Compare changing existing system for the new requirements with the need to stabilize it for reliability and safety.
• Planning vs. Emergence – what strategy provides better results – relying on planned system changes or anticipating new properties and behavior emerged from unpredicted interactions within the system and with external environment. Compare Waterfall and Agile processes.
• Feedback vs. Isolation – what should be a level of feedback to enable system self-correction and adaptation and avoid information overflow, noise and destabilization. Compare hour release cycles of Flickr with years of Windows Vista development.
• Expansion vs. Consolidation – what is the best way to combine two main types of the system evolution – adding new functions and improving internal organization. Compare adding new features to refactoring.

And the main fight is between Order vs. Entropy – how to avoid system degradation and maintain order. Compare well organized and messy systems (Ball of Mud).

How can we resolve and tame System Forces?

1. Favor what supports business needs and goals:
   • for life critical systems (medical, space shuttle): Stability, Planning, Centralization
   • for competitive and rapidly changing systems (most businesses and web) : Adaptation, Self-Organization, Emergence.
2. Favor what is most effective for the human way of thinking and working: Simplicity, Self-Organization, Abstraction (for many people Concrete approach is more effective).
3. Favor what keeps more options open for evolution and better fit for changes in environment: Adaptation, Feedback, Diversity.
4. Consider positioning and life stage of the system
   • expanding into the new markets: Adaptation, Expansion, Feedback
   • retaining and satisfying large user base: Stability, Consolidation, Centralization

Taming complex systems is difficult – it needs constant skillful effort to avoid Entropy and keep Order. Economic and Human forces add more challenges and nuances.

The art and mystery of the software creation is how the system of interacting imperfect people can build useful, reliable and complex software systems in time and within budget.

Question: Is it possible that computer systems will evolve in the future without human intervention?

Posted by Andriy Solovey | Permalink | Trackback