Software Creation Mystery - http://softwarecreation.org

Comparing Intelligent Software Evolution to Chaotic Biological Evolution

Software Evolution is similar to a Bible Story: we, software creators, have a plan and goals and pursue them to create the perfect solution. We carefully select and add new features according to our grand design. We try to keep design clean and optimal. Things sometimes go wrong and the software system goes berserk, but we intentionally intervene and fix the problems to continue fruitful functioning of the system.

Biological Evolution is a different story: chaotic raise and fall of different species, without any plan and goals. New traits and variations appear all the time and many are useless or even harmful. Small modifications accumulate over time, cause significant changes and emergence of new species, better adapted for the evolving world. Survival of individual organisms and species is the matter of chance, pressure from environment and competition with other organisms.

Intentional growth of software versus chaotic evolution of organisms… Do they have anything in common?

Biological Evolution

In biology, evolution is the process of change in the inherited traits of a population of organisms from one generation to the next.

Species – a group of organisms that can reproduce with one another and produce fertile offspring.
Population – a localized group of individuals belonging to the same species.
Traits – particular characteristics of an organism.
Genes -portions of DNA molecule, which control inherited traits.
Genotype
– complete set of genes within an organism’s genome.
Phenotype – the complete set of observable traits that make up the structure and behavior of an organism. These traits come from the interaction of its genotype with the environment.
Variations – the variation in phenotypes in a population reflects the variation in these organisms’ genotypes.

Most of the genome of a species is identical in all individuals of that species. However, even relatively small changes in genotype can lead to dramatic changes in phenotype: chimpanzees and humans differ in only about 5% of their genomes.

Mechanisms of increasing gene variations

  • Mutations – mistakes that occur during DNA replication, which alternate version of a gene (alleles). Mutations give the new characteristics to the organism and can have positive and negative effects.
  • Reshuffling of genes through sexual reproduction with random mixtures of their parents’ chromosomes.
  • Gene flow – genes from one population enter the gene pool of another, e.g. migration between distant populations.

Mechanisms of decreasing gene variations

  • Natural selection – a process causing heritable traits that are helpful for survival and reproduction to become more common in a population, and harmful traits to become more rare. Natural selection is the result of pressure from environment and competition between organisms.
  • Genetic drift – produces random changes in the frequency of traits in a population, which results from the role chance plays in whether a given individual will survive and reproduce.


Though the changes produced in any one generation by drift and selection are small, differences accumulate with each subsequent generation and can, over time, cause substantial changes in the organisms.

Natural selection usually predominates in large populations, while genetic drift dominates in small populations. Some theories suggest that genetic drift and mutation play much larger role than natural selection in evolution.


Fitness is the central concept of natural selection. Fit does not necessarily mean biggest, fastest, strongest or having more children. Instead fitness measures the organism’s genetic contribution to the next generations – the proportion of subsequent generations that carry an organism’s genes.

Now we can provide another definition of evolution: the change in the gene pool of a population over time.

Outcomes

  • Adaptation – structures or behaviors that enhance a specific function, causing organisms to become better at surviving and reproducing
  • Co-evolution – the evolution of one species causes adaptations in a second species, such as a conflict between predator and its pray.
  • Co-operation – mutually beneficial interactions. Can be found in social insects, such as bees, termites and ants, where sterile insects feed and guard the small number of organisms in a colony that are able to reproduce.
  • Speciation – the process where a species diverges into two or more descendant species, mostly occurs in populations isolated geographically. It caused by gradual accumulation of small genetic changes.
  • Extinction – the disappearance of an entire species. Indeed, virtually all animal and plant species that have lived on earth are now extinct.

Software Development Ecosystem

The first question: what is the object of evolution in Software Development?

People?
I don’t have guts to suggest interbreeding most successful programmers and receive new generations of the best programmers. Very impractical, too long to wait and modern people are not an easy target for experiments…

Code?
Code is a good candidate. We constantly evolve and change code. But individual code doesn’t have comparable set of genes and variations (unless it is a genetic algorithm or you create mutants with intensive copy and paste) and don’t compete with other code for survival (only at level of products where code is one factor from many). We could apply only limited evolutionary principles to code and they are mostly related to ideas behind the software systems.

Can we find a better candidate?

I believe that genome of software development is the knowledge about how to build software systems.


Knowledge, DNA of Software Development, has two components

  • process: principles, activities, roles and flow (Agile, Extreme programming, Scrum, CMM, RUP, Waterfall, Cleanroom, Crystal Clear, SWE-BOK, Context Driven Testing, etc.)
  • expertise: design, programming and domain (Algorithms, Object Oriented Design and Programming, Design and Enterprise Architecture Patterns, Refactoring, Domain Driven Design, Behavior Driven Design, Relational Databases, Web Design, Information Architecture, Usability, etc.)


In essence, this is the knowledge about what software team members do at any given time to create the system: understanding requirements, design and build, verification. Humans are the best carriers of this knowledge, but it is possible to transfer it with less effective documents, books and other media. People carry huge amount of intrinsic knowledge that comes with experience, which is difficult to extract, describe and transfer to other people. Our methodologies are created to represent software development knowledge (genome) in the formal way convinient to carry between people heads and projects. However, this genome immediately meets the harsh reality of environment pressure.

Software Development Environment:

  • people: experience, capabilities, skills, motivation, personalities
  • requirements: software needs, expectations, constraints, regulations
  • customer: knowledgeability, relations and attitude
  • market: trends, competition, standards
  • technology: development platforms and tools, technical infrastructure
  • organization: goals, plans, management, culture, leadership style, resources, finances


Software Development Ecosystem Characteristics:

  • Populations – there are different kinds of software development (web, desktop, embedded devices; system, productivity, games; business, military, science; commercial, in-house, open source; for fun, business-critical or life-critical, etc.)
  • Traits – characteristics of approaches used in software development
  • Genotype – software knowledge carried by individual team and used in the project to build software system
    • Encoding of information – contains enough information to build an individual software system
    • Inheritance – knowledge can be passed to the next project iterations, other projects or to the whole development community
  • Phenotype – all the activities in software project that are outcome of interaction between this knowledge and project environment.
  • Co-evolution – ideas compete (Waterfall vs. Agile, Procedural vs. Object-Oriented vs. Functional Programming), cooperate and enhance each other (Extreme Programming and Scrum, Refactoring and Design Patterns)
  • Mechanisms:
    • Mutations – continious invention and modification of the new approaches
    • Reshuffling – frequent recombination of various ideas
    • Gene flow – exchange of ideas with other populations and industries (e.g. NASA, Lean Manufacturing)
    • Natural Selection – software development knowledge is often under the pressure to achieve success, better meet customers needs and be effecient. Better methods have more chances to survive and become more common (e.g. adoption of Iterative approach that successfully solves challenge of changing requirements)
    • Genetic drift – some ideas and approaches become popular by chance, some great ideas disappear without wide acceptance. Management and leadership change, politics, better sponsors, strong opinion of less qualified people, lack of knowledge about alternative approaches could play significant role in establishing far from optimal development knowledge for the project.


Software Development Evolution is change over time in the essential knowledge about how to build software systems.


Examples of software development evolutionary genes (views, ideas and activities):

main ideas
understand requirements
design and build
verify
Extreme
Programming
iterations, emergent design, customer can change mind write user stories, customer prioritize,
developers estimate
write unit test, make it work, refactor; work in pairs build automated acceptance tests

CMMI
process improvement framework, five levels of maturity based on practice areas that
define goals, practices and work products
SWE-BOK guide to the Software Engineering Body of Knowledge: characterizes and organizes
contents of the software engineering discipline in ten knowledge areas
Traditional SDLC
sequential process separated into distinct phases: analysis, design, implementation,
maintenance
analyze, prepare functional and technical specifications design and build according to specifications test according to specifications; security, performance and user acceptance
testing

Context Driven Testing
testing is driven by an understanding of the environment, culture, and intended
use of software
exploratory testing: concurrent test design and test execution; adapt testing plans
based on test results

Domain Driven Design
the system design should be based on domain model form common language between business and developers build code that represents domain model and concepts use automated tests for domain model assertions
Object Oriented Design a software program is the system of interacting objects create object model for the problem space: base classes, responsibilities, collaboration;
UML diagrams
find abstractions, inherit, delegate, encapsulate, package; use design patterns

User Centered Design
active participation of real users on all stages; iterations of design solutions focus groups, interviews participatory design frequent usability testing

Information Architecture
organization of information and functionality to increase web site usability, intuitive
navigation and findability
understand business context, content and users build site map, wireframes, personas, user interaction scenarios
Web Design
web based user interface and content delivered via Internet understanding content, visual appearance build web pages with CSS, HTML and JavaScript; implement server side functionality
for dynamic pages
browsers and standards compatibility
Database Design
data storage and retrieval analyze data elements and relationship; build logical data model relational databases: design and normalize tables, build views, stored procedures
and functions


There is no best set of ideas. Success of the every software creation gene depends on interaction with a project environment. Biological evolution can teach us few lessons of selecting and evolving most advantageous genes for our projects. The next post will explore these lessons.

Resources:
Evolution, Wikipedia
Introduction to Evolutionary Biology

AddThis Social Bookmark Button AddThis Feed Button


Comments

very, very useful and interesting article. it made me realise that evolution in this sense is not an either/or proposition. you have intentional application of evolutionary principles.

Andrew

Comment by Andrew McVeigh | June 19, 2008 6:25 am

Andrew,

I would like permission to use some of the illustrations on your blog in a curriculum that we are writing for the Los Angeles Unified School District. Specifically – the biological evolution illustration & your cartoons on genetic variation. This curriculum is not sold or marketed in any way. We distribute it free to all of our teachers. I am trying to illustrate a concept about sexual reproduction & the maintenance of sex evolutionarily. If you are not the source of these illustrations I would greatly appreciate if you could share the source so that I can contact them regarding obtaining permission.

Thanks again.

Comment by Elizabeth | October 13, 2008 2:58 pm

“Biological Evolution is a different story: chaotic raise and fall of different species, without any plan and goals.”

This view of bio evolution is pretty outdated. Based on the latest review paper in evolutionary biology, “the genome as an intricately formatted Read-Write (RW) data storage system constantly subject to cellular modifications and inscriptions. Genetic change is the result of cell-mediated processes, not simply accidents or damage to the DNA. This cell-active view of genome change applies to all scales of DNA sequence variation, from point mutations to large-scale genome rearrangements and whole genome duplications (WGDs).”

How Life Changes Itself: The Read-Write (RW) Genome (Physics of Life Reviews,
2013, in press) –> http://goo.gl/32Re4p

Comment by Nick | August 3, 2013 1:13 am

This blog have little value without you and your comments, thoughts and discussions. Please, leave your comments. You are welcome to debate and criticize any idea, but, please, don't attack other people. Thanks for your contribution!

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Subscribe without commenting

Software Creation Mystery - http://softwarecreation.org
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 License .