Software Creation Mystery - http://softwarecreation.org

Should An Effective Developer Innovate, Imitate or just Integrate?

in·no·va·tion – introduction of new things or methods
im·i·ta·tion – the copying of patterns of activity and thought of other groups or individuals
in·te·gra·tion – an act of combining into an integral whole.

What is the best strategy for an effective developer – innovation, imitation or integration? Should you introduce new creative solutions, adapt other people ideas or just integrate existing components?

Software Development is an exciting intellectual endeavor without physical barriers. It is easy to start innovating – come up with new ideas and quickly submerge into their implementation. And I don’t mean here fundamental breakthroughs. I consider as innovation building of any non-trivial solution that is not directly stemmed from Google search results, development resources or available examples. And certainly, I pose the dilemma – innovate or not innovate – to skillful developers who are quite capable to innovate and who enjoy meaningful creative work.

Discount Machine

Lets start from a tournament organized by Kevin Laland of the University of St Andrews to find out what strategy works best to gain maximum pay-off:

  • innovation – a new behaviour randomly acquired by individual learning;
  • observation – a new behaviour acquired by learning from others or imitation;
  • exploitation – using a previously learned behaviour to gain pay-off.

The participant had to build a strategy that their virtual agents would use to decide between these options in a computer-generated world. The challenge was to create the strategy that generated the most successful agents.

New Scientist reported that the winner strategy, Discount Machine, spent almost all learning time observing rather than innovating. Optimal learning time was between 10-20% and spaced through agent’s life.

Therefore, a tournament showed that the best strategy is keeping up-to-date by learning what others are doing and using their successful solutions most of the time.

Can we apply these results to software development?

Strategies

You have three main strategies for approaching a new problem in software development

  1. Integrate into the system existing software product, component or service – commercial or open source (for example, Payment Gateway as PayPal, Blog Engine as WordPress, CMS as Drupal, UI Components as Telerik and so on)
  2. Imitate good enough solutions and adapt to your problem (Architecture Patterns as MVC, available code examples and guidelines as MSDN, borrow ideas from blogs, open source projects, Starter Kits, SDK and so on)
  3. Innovate and create new solutions or make significant improvements to existing approaches
Strategy comparison (*)
Integrate Imitate Innovate
Time to market Fast, if effort to integrate with other system components is low Slow, but predictable, if not many hidden pitfalls or adaptation problems are encountered Unpredictable as any innovative work
Cost Low, if components are reasonably priced and not much integration work needed More expensive and depends on complexity and adaptation effort Unpredictable
System integrity (with system architecture and environment) Acceptable if new components don’t screw and over-complicate core architecture Good, if developers adapt ideas to existing architecture Solution is built to match core architecture and customer needs
Required Expertise Not much specialized expertise is required, usually external support is available for integration Good developers can effectively adopt good ideas that are explained well High level expertise, creativity and specialized knowledge are required for good innovative solution
Control over code and future development Little control and you are on mercy of external developers Good control if ideas are applied well and not over-engineered Full control
Competitive advantage and uniqueness Not much for the standard solution that many can use Depends on quality and creativity in adaptation Innovation is an excellent opportunity to gain advantage
Maintenance, support and improving capabilities Work is outsourced to dedicated external developers who fix, support and improve the product Your effort is supported in original source of ideas if you are lucky Completely your own effort
Learning curve, tacit knowledge, help Usually supported by help, tutorials, training and community involvement Partially supported by original source, however can drift far as the result of internal implementation Should be covered by you to enable effective support and future development by existing and new developers

(*) Disclosure: I should confess that the table above has some assumptions. I assume that external components for integration have good quality, work as advertised and are backed by solid support and team . Also, I assume that internal developers involved in implementation have good skills and experience. They follow good practices and are motivated to do great job and know what they are doing. I fully realize that life is not simple, and my assumptions could be completely wrong and this would change the table and the whole game 🙂

As you can see, Integrationof existing components is the most effective way to develop a new system with lowest risk, effort and minimal future support. However, it still could be not the best approach as sometimes:

  • available solutions do not meet needs or compromises are not satisfactory
  • non-conventional and state-of-art solution is required for challenging important needs
  • the component is crucial for the competitive advantage and uniqueness of the software product
  • full control is required over code and future development of the component
  • the component has low compatibility with system ideas and core architecture, over-complicates technical solution and breaks integrity of the system that result in
    • unnecessary code and rough system seams to make components work together
    • limited refactoring and re-design options
    • reduced ability to expand the system

Imitation is a middle ground – you build solution yourself but use other people ideas and experience as a guidance.

Innovation is expensive and risky to solve the problems. However, it can be the only way if you face unique challenges, cannot find good ideas and cannot change requirements to use existing solution.
Good innovation makes the system better suited for customer needs, economically successful and more reliable. It could be
  • Improvement and simplification of the system design to make it easier to evolve and support
  • Removing technical constraints and solving technical challenges to make the system faster, more responsive and reliable
  • Introduction of important business features where no standard solutions exists
  • Significant improvement of users experience
  • Reducing cost of development and support
Innovation can be harmful. For example,
  • Developing system features or properties that are not required
  • Building alternatives for good available solutions (reinventing the wheel)
  • Playing with interesting ideas without customer awareness

The Effective Way To Build Software System

A short answer to dilemma: maximum integration and minimal innovation.
A long answer with nuances and description of approach:
  1. Understand purpose of the system, essence of customer needs and desired outcome
  2. Break downthe system into components and research if
    1. standard solutions exist (for integration)
    2. implementation ideas exist (for imitation)
    3. innovation is required
  3. Can you change needs and requirements? Transform customer needs and architecture ideas to minimize development effort
    1. by moving from innovation to imitation strategy
    2. by moving from imitation to integration strategy
  4. Evaluateeach component from the system and business perspective
    1. Should you avoid integration and use imitation if:
      • System integrity under the threat and the component is part of the system core
      • Control over code and future development is required
      • Competitive advantage and uniqueness are important
    2. Should you still go with innovation because of unresolved contradictions, challenging and unmet needs?
  5. Build prototypes clearly separated from mainstream development to confirm selected strategy

Becoming an effective software developer

How can a developer prepare for selecting and using the right strategy?
All strategies
  1. Systematically study other solutions in your area of specialization (at least a couple in a month – understand strengths, weaknesses, high-level architecture and interesting tricks)
  2. Learn concepts and language of your business domain to be able to understand customers and shape their needs together
  3. Enhance abilities to find and brainstorm alternatives (improve techniques, make them essential part of your process)
  4. Become an expert in Google search and fast evaluation (no kidding, these skills become very important for any modern developer)
  5. Master rapid prototyping, apply solutions in practice and seek for rapid feedback (answer in short time if proposed solution is good, learn and correct if you made a mistake)
  6. Develop a holistic view and knowledge of the system, infrastructure and environment (understand subsystems, connections, integration options and trade-offs)
  7. Keep up with latest software development trends, technologies and approaches (subscribe to blogs, magazines and other sources)
Innovation
  1. Achieve deep specialization in your core technical area and business domain (extensive experience and deep knowledge are great assets for innovator)
  2. Continuously develop creativity and problem solving (the post about creative problem solving)
  3. Enhance architecture and fundamental programming expertise based on own practice and ideas from others
  4. Master Evolutionary and Domain Driven Design

At the end,

The effective developer understands the purpose of the system and customer needs, selects a right strategy for the system components and builds a great solution with minimal effort.
AddThis Social Bookmark Button AddThis Feed Button


Comments

Great post, Andriy! Sent it to the whole team, definitely on the mandatory reading list.

Comment by Nariman | July 7, 2010 8:27 am

ever and ever again I realize that becoming an expert in Google search is the most important skill the software developers need to be successful in their area. Great posting, keep writing, Andriy!

Comment by Alice | July 12, 2010 8:59 am

Very high valued and potential blog post and most of the software engineers should now be satisfied enough with the guidence provided here in the blog post.

Comment by modern pilates courses | September 10, 2010 12:03 am

Just my personal idea: I think that is hard to generalize in this particular context. The proper strategy depends from project to project and from team to team.

Comment by Francesco | October 13, 2010 4:45 am

????? ? ??????????? ???????,??????-??????????. ????? ???????, ?? ???? ????? ???????????? ? ?????? ??????. ???????, ????????, ? ??? ????-?? ? ???????????
?????????? ??????????? ???????????? ?????? ??????????. ??? ????: ??????????? ?????, ???????? ? ???????? ?????, ????? ?? ?????? ?????, ?????????.
?????? ? ?????? ?????: ????? ??? ?????????, ???????? ????, ????????????? ????, ??? ????????? ??????????? ?????.
?????? ??? ??????, ??????????? ?????, ??? ????????, ??????? ????? ? ????, ????????, ?????, ?????????, ??????? ??? ??????, ?????????? ?????????.
?????? ? ?????. ?????????? ????? ??? ?????????. (?????) ?????? ?????. (?????). ???????? ?????. (?????). ??????? ????????????? ?????????. (?????? ???)

Comment by oppoppysmib | July 15, 2011 1:10 pm

This blog have little value without you and your comments, thoughts and discussions. Please, leave your comments. You are welcome to debate and criticize any idea, but, please, don't attack other people. Thanks for your contribution!

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Subscribe without commenting

Software Creation Mystery - http://softwarecreation.org
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 License .