What does a customer wants?

Usually your customer wants to know a simple thing – when your team will deliver working software based on agreed requirements. They naively want to safely bet on highly uncertain future outcome – delivery of the complex never created before system that completely fulfil their dreams.

The customer wants reliable McDonalds experience, but with a little tweak – they want to hand in their own exclusive menu, often created by people who don’t have any idea how to cook.

In short, the customer requires creative flexibility of Cooking Master Chef fortified by ability to predictably deliver hamburgers as McDonalds.

But, is it possible in our development reality to chase two opposing goals – Predictability and Flexibility in the same time?

Predictability vs Flexibility

The best way to achieve predictability is standardization – formalize process, nail-down rules, follow and optimize them mercilessly. However, standards have negative effects – lower morale, inhibited creativity, reduced flexibility and slower response to changes.

Rapid adaptation is the best way to to be flexible – eagerness to quickly regroup and self-organize to address new challenges and creatively solve new problems. However, constant changes and creativity sparks lead to unpredictable and ad-hoc results without much opportunity to stabilize and optimize delivery.

Standardization is a crystallized answer to past events – how to repeat effectively something that was solved before and proved to work.

Adaptation is the response to current events – how to embrace unforeseen requests, challenges and events.

Standardization has own cost – processes and rules should be maintained, reviewed and optimized otherwise they will become source of overhead, rigidity and frustration. But if there is no standardization, lessons and best practices could be lost or not followed.

It is difficult, but possible to chase these two contrary goals in the same time. Build standards to solve repeating problems and optimize recurring activities. Adapt to meet new challenges.

Team Maturity Matrix

Development teams have different capabilities to strike both predictable and flexible delivery.

• Amateurs – do even simple things… unpredictably
• Explorers – capable of solving complex non-standard problems, but cannot guarantee time and budget (Ph.D guys)
• Mechanics – very efficient with standard solutions, but struggle with new stuff (anal-retentive guys)
• Regulars – predictability of delivery lowers for more challenging problems
• Strike Force – deliver reliable results for any type of problem

Any team should grow from Amateur status, but rare team could become Strike Force that mastered both – predictable and flexible delivery. Than what direction of growth to choose? It depends on nature of projects and problems the team tackle.

Finding sweet spot

Predictable delivery requires reduced variations in development process: uniform, ordered and consistent approach with focus on high efficiency.

Flexible delivery requires increased variations in the process: enabling more options, wider range of responses to improve fitness in changing environment.

These 2 goals could compliment each other. Predictability (with standardization) could bring consistency and stability in volatile and evolving development ecosystem. Adaptation helps to improve and find better standards and ideas for established processes.

So, can team deliver as McDonalds, but using exclusive menu? Yes, if they not only excel in programming, but also excel in the way they work – persistently re-evaluating approaches and improving both predictability and flexibility.

There are three simple requirements for the team

1. Follow standards (minimal, adhering to principles)
2. Challenge standards (not relevant, rigid, impeding, excessive)
3. Introduce and Improve standards (from lessons learned, best practices, enhancing flexibility and value stream)

And you will build powerful development machine that continuously transforming to embrace fluid reality while keeping steadiness and predictability as McDonalds.

]]> http://softwarecreation.org/2011/can-software-team-deliver-like-mcdonalds/feed/ 0 http://softwarecreation.org/2011/stuck-on-a-big-hard-programming-task-read-this/ http://softwarecreation.org/2011/stuck-on-a-big-hard-programming-task-read-this/#comments Mon, 25 Apr 2011 00:18:45 +0000 Andriy Solovey http://softwarecreation.org/?p=232

Hit the road

Seasoned project managers will tell that delivery of software is result of many trade offs. The main trade off is between Time (when project will finish) and Scope (how much will be done). This post will show that using right tools you could gain improvement for both variables.

While it is possible to create orderly step-by-step process for increasing productivity of software teams, it will never be ideal – too many variations and situations will hinder it usefulness. I believe in set of useful tool that could be combined to craft custom optimal solution.

Strategies

There are several strategies that lead to increase in productivity – how many units of scope software team can produce within fixed time.

1. Increase Capacity (Capacity) - increase capacity by hiring more people or increasing work hours
   • productivity is increased as result of more resources and hours available
2. Improve Value Stream (Value) - increase added business value on each step and reduce waste and overhead
   • productivity is increased as result of optimized delivery of value
3. Adaptat to Reality  (Adaptation)- learn from practice and mistakes, validation of ideas by reality, adapt to changing situation
   • productivity increased as result of early corrections and improving how things are done
4. Empower Individuals (Individuals) - boost people knowledge, skills, morale and focus
   • productivity is increased as result of higher individual performance and motivation
5. Enhance Communication (Communication) - improve communication and mutual understanding inside and outside of the team
   • productivity is increased as result of availability of necessary information, clarity of what should be done and exchange of ideas for implementation
6. Organize Better (Organization) –  structure team and assign roles for better coordination and decision making
   • productivity is increased as result of better decisions and focus on important areas
7. Expand Expertise (Expertise) - increase range of skills and services offered by team
   • productivity is increased as result of better execution of necessary project activities
8. Scale Externally (Externality) – outsourcing and involvement of external communities
   • productivity is increased as result of involvement of more people outside of team
9. Tame Complexity (Design) – manage complexity and provide simple and well designed solutions
   • productivity is increased as result of reducing complexity burden on software development
10. Preserve Quality (Quality) – use defensive tactics to ensure high quality
    • productivity is increased as result of preventing system flaws and reduced effort to fix bugs

• People-oriented – people are the creators of software and have major effect on output
• Process-oriented – the way how people work has significant impact on outcome
• Development-oriented – development practices and approach to the system implementation matters a lot

People-oriented tools

Strategies: Capacity , Expertise , Communication ,  Organization

New people increase volume of possible work within the same time and new expertise increases range and quality of tasks. However, the increase in output is not linear and quickly diminishes, because more people cause communication and coordination overhead. In addition, compensation increase make this tool expensive and less attractive.

Resources:

• Methodology and Team Size, Alistair Cockburn
• Team Size Matters, schmula

2. Motivate

Strategies: Individuals , Quality

Happy motivated people are more productive, focused and concerned about quality and end result. While there is no common receipt for each individual, I believe there are some strong motivators: interesting and meaningful work, fair compensation, control over own tasks and outcomes, ability to learn and professionally grow, comfortable workplace and adequate tools, empathetical and caring management and so on.
Daniel H. Pink in Drive thinks that people are motivated by three elements:

• Autonomy: People want to have control over their work.
• Mastery: People want to get better at what they do.
• Purpose: People want to be part of something that is bigger than they are.

• Professional Growth

Tips: Motivate individually, but set also group targets to encourage cooperation

• Drive, Daniel H. Pink
• The Ideal Software Company. Utopia?

3. Professional Growth

Strategies: Individuals , Expertise , Quality , Adaptation

Software professionals should learn a lot to become productive and proficient. People learn in several ways and learning from mistakes is the most memorable, but expensive way. Learning by self-study, formal training and coaching by experienced colleagues are more effective ways. Encourage and direct learning to grow high performance experts and team. And certainly, continue learning as much as possible from mistakes and practice. :)

• How to Become an Expert. Top 7 Qualities
• How to Become an Expert. The Effective Way

4. Overtime

Strategies: Capacity , Individuals , Quality

Nobody recommends involuntary overtime for a long time as benefits will disappear quickly when a stressed team lowers quality and starts breaking apart.

5. Right leaders

Strategies: Communication , Organization

Good leaders jell teams, align people with goals, boost energy and remove barriers. Leaders vary from commanders to visionaries and each style has merits under specific circumstances. Right leaders are essential for the project success.

Tips: Adjust leadership style to situation

• What is The Best Leadership Style for The Software Team?

Strategies: Expertise , Organization

A software team requires range of skills and involved in different activities to achieve end goal – implemented software system for customer needs. Beyond software implementation the team deals with a customer domain, company vision, market demands, technologies, operation environments and other project aspects. The team is involved in research, analysis, coordination, design, architecture, usability, testing, deployment, hosting and other activities. Therefore team players should be able to play different roles and have expertise to cover various aspects to ensure good end results.

Resources:

• Why specialization in Software development is bad for business

Strategies: Capacity , Expertise , Externality , Organization , Communication

Tasks and challenges should match skills and experience of team players. It doesn’t make sense for backend developers to convert Photoshop files into html or conduct marketing research and focus groups. If team cannot afford to hire a permanent specialist, they should try to find professional mercenaries who will do work faster on higher quality level.

Strategies: Individuals

Flow (or Immersion) is a state of mental focus so intense that awareness of the real world is lost, generally resulting in a feeling of joy and satisfaction. When learning and tasks are too easy, people become bored. If they are too complex they become stressed and frustrated. Flow happens when perception and understanding are challenged near capacity without being exceeded.

• ability to focus
• clear goals
• immediate feedback
• control over actions, activities and the environment

• Flow: The Psychology of Optimal Experience, Mihaly Csikszentmihalyi

Strategies: Quality , Organization , Individuals

Few people will be always disciplined no matter what and few will always violate rules. Majority are context sensitive – relax than discipline is low and work hard when discipline is enforced. Build a minimal set of rules approved by majority and stick to these rules.

Resources:

• Lost Personalities: How our company alters us

Process-oriented tools

10. Increase feedback and early practical use of ideas

Strategies: Adaptation , Quality , Value

Reality is complex, fluid and unclear. Many assumptions and plans quickly go bust. The process that has built-in mechanism for signaling problems and self-correction is the most effective. Rapid feedback is the core part of such process.

Tips: Iterative development is one of the best examples of the process that relies heavily on feedback.

11. Build alignment

Strategies: Communication , Organization

People jump significant gaps to understand each other – sometimes without much success. Clear vision, smart priorities and honest evaluation allow team players moving in unison and enforce each other. Good understanding focus people on right goals and reduce wasteful activities. Communication and trust are important for alignment.

• vision – project direction and top-level goals
• interests – mix of personal, customer and companies interests (sometimes conflicting)
• understanding – clarity about what others mean
• trust – confidence in other people intentions and promises

12. Integrated decision making

Strategies: Communication , Quality , Adaptation

See Also:

• Self-organization

As project progresses you will find problems, wrong assumptions and challenges. You will have better experience and understanding. Even brightest ideas, best practices and excellent analysis and design could become outdated and ineffective. Continuously look for improvements, eliminate waste and bottlenecks.

Resources:

• How to rescue failing software projects: The Toyota Way
• Theory of Constraints
• Kaizen

Strategies: Individuals , Quality , Value

Highly productive team shouldn’t waste time on routine or non-core activities. Routine work should be automated and non-core work outsourced or covered by external components. The team should focus on high value-added distinctive work for business and application domain – this will bring maximum business results with minimal development effort.

• system thinking
• understanding what really matters and what are limitations
• broad vision and out-of-the box thinking
• knowledge of environment, context and system outside relations
• synthesis of concepts, information and experience

Strategies: Capacity , Expertise , Externality , Organization

Strategies: Value , Communication

You can better understand and change scope of application if you actively work with customers and become partners

• establish ubiquitous language – for better understanding of ideas and translation them into code
• tap into experience and get better insight for business domain and processes
• transform ideas – simplify and find better alternatives

• reducing scope
• simplifying solutions
• finding good enough alternatives

• Pareto Principle
• Ubiquitous Language
• Software Development is The Flow of Ideas. The Rest is Secondary

18. Select Right Process Flow

Strategies: Organization , Value

Select most optimal process for creating value.

• real-time flow –  immediately push tasks for implementation (continuous development flow for highly organized co-located teams)
• pull systems (Kanban) – pull tasks when previous batch is finished and the team is ready to continue (iterations and backlog)
• schedule the workload (Heijunka) – create schedule for contributors with limited availability (outside consultants or shared specialists as designers)

Tips: Use different flows for distinct stages of software creation. For example, a team process can internally use real-time flow, with external teams – Kanban and with consultants Heijunka.

Resources:

• The Toyota Way and Development Process

19. Self-Organization
Strategies: Communication , Individuals , Adaptation , Value

Tips: Not every team can self-organize – it requires maturity. And loss of direct control could be a problem for management if a project departs from company strategy and desired parameters (as budget, standards, resources).

• What is The Best Leadership Style for The Software Team?
• Organizing Self-organizing Teams
• Agile Self-Organizing Teams

Development-oriented tools

20. Increase reuse

Strategies: Design , Quality , Adaptation

Reuse code  (components, libraries, solutions) as much as possible to speed up development and minimize code base. Reuse will bring familiar tested solutions and prevent frequent re-inventions of the wheel.

See Also:

Tips: Design for reuse requires skills and time. It is easy to overly complicate the system by pursuing reusable code. Evolve and refactor code to achieve practically good reusability. Sometimes custome solutions are lighter and better suited for specific purposes than generic and reusable.

21. Generalize and Simplify
Strategies: Design , Quality , Communication

Complexity and poor understanding of the system is one of the worst problems in development. Design, code and concepts should be as simple as possible and clearly understood by the team. Simplicity and clarity are difficult to achieve and require special effort. It is much easier to produce convoluted over-engineered solution for complex problem (Ball of Mud) than simple and elegant solution.

Tips: Great design achieve simplicity by abstracting numerous details into simple concepts in process of solution evolution and learning from implementation.

• Big Ball of Mud, Brian Foote
• 11 Laws of The System Thinking in Software Development
• The Elements of Pragmatic Programming Style. Approach.

Innovation is expensive, uncertain and leave you with custom solutions on your own for support and future development. Piggyback on actively involving products, especially for non-core problems where good and proven solutions are available.Tips: Innovate to gain business advantage or solve serious technical challenges if you have an outstanding team and well-defined problems.

Resources:

• Should An Effective Developer Innovate, Imitate or just Integrate?
• Selecting The Best Strategy for Software Teams: Retreat, Evolution or Revolution

23. Domain Specific Design

Strategies: Design , Communication

Keep technical ideas and code concepts close to the customer domain. You will achieve consistency, better sync between system and business concepts. You can easier translate customer ideas into the system and customers will understand your implementation better.Tips: Domain Driven Design is one the best approaches to achieve synthesis of technical and business ideas for complex domains.

Resources:

• Domain Driven Design, Eric Evans
• The Secret of Building Effective Software Systems
• The Elements of Pragmatic Programming Style. Composition.

24. Evolution

Strategies: Adaptation , Value , Design

Start with the simplest primitive solutions, enforce early use and seek for feedback. Adjust and improve design before continuing adding new features. Look for better ideas and abstractions. Delay fundamental design decisions until you are confident about system development direction and validated ideas on practice. Do not over-engineer and add unnecessary features.

Tips: Evolution main deficiency is slowness. Modern businesses often don’t have time for many improvement cycles – they want results now.

Resources:

• What can Software Development learn from Biological Evolution?
• Ideas in Software Development: Revolution vs. Evolution. Part 1.

• How a beautiful software system becomes Frankenstein

25. Open architecture and API

Strategies: Externality , Design

Open your solution for external developers who can extend and add more useful features. If you are lucky, you will get much more functionality than you can build internally while still controlling a core solution (iPhone is a perfect example).

Also Open API (convenient and supported by good documentation) give developers eligible ways to interact with you system without hacking or completely ignoring it.

Tips: Once Open API is used it becomes liability – next design decisions will be affected by compatibility and legacy concerns.
Resources:

• API

26. Follow Design Principles

Strategies: Design

Some of design principles

• You should be able to extend system without modifying it
• A class should have one, and only one, reason to change
• Depend on abstractions, not on concretions.
• Make fine grained interfaces that are client specific.

Famous design patterns are built around these principles. Follow sound design principles to build flexible, easy to expand system.

Resources:

• Principles of OOD, Rob Martin
• The Elements of Pragmatic Programming Style. Approach.
• The Elements of Pragmatic Programming Style. Composition.

27. Shared Code, Ideas and Standards

Strategies: Design , Expertise , Communication , Quality

The team should collectively own code and pursue consistency and use of best practices. Otherwise the system will become clash of isolated impassible code islands – scary for people who didn’t build them. Share code and learn from each other to build outstanding system with high quality and integrity of every component.

Tips: Pair Programming is one the best ways to share code, ideas and best practices
Resources:

• Collective Code Ownership
• Pair Programming

28. Use Better technology and tools

Strategies: Individuals , Design , Value

Tips: Obsession with new technologies and race to use them is dangerous for project. Keep sanity until new technology is proven to be beneficial :)

• http://www.sdtimes.com
• http://www.computerworld.com

Guide for increasing team productivity

Questions

• Improving relationship
• Flying to space to be the first man on Mars
• Overweighting and health problems
• Winning bodybuilding or beauty contest.
• Procrastination
• Learning new technology
• Making more money
• Building next Facebook or Twitter
• Raising kids
• Boosting career growth
• Becoming Olympic champion or billionaire
• Making the world a better place
• Planning a dream vacation
• Seeking for happiness…

We often are inconsistent in solving problems and cannot find a good solution. This post focus on effective and creative problem solving by offering systematic approach and wide range of techniques.

Beginning

There are four main elements of Creative Problem Solving:

1. Problem - any question or matter involving doubt, uncertainty or difficulty
2. Solution - set of actions that solves a problem
3. You – your knowledge, beliefs, emotions and abilities
4. Context -forces, people and environment

And there are seven strategies:

1. Knowing - you know how to solve the problem in practice
2. Copying – copy and adapt working solutions
3. Logic - follow reasoning and step-by-step approach to find a solution
4. Lateral thinking - solving problems through an indirect and creative approach
5. Psychology - use intuition and subconscious to pop a solution
6. Dialogue - engage brains of other people
7. Trial and errors - make guess, verify and refine by experiments

Before you start finding a solution, you should board the problem – understand and accept it.

Boarding a problem in 3 steps

Step 1. What is the core of the problem?

• Essentials - What are symptoms, facts, key points, players, context?
• Root Conflict - Why is this a problem? What is the core of a problem? What is not a problem? Ask 5 whys
• Clarity - What is still unknown and unclear? Do I need more information?

• Feasible - Is the problem real and you can solve it? Continue by asking critical questions about the problem
  • Is it logical? Does the problem make sense for the rational part of my brain?
  • Is it subjective? Is it possible that problem exist only in my mind?
  • What are emotions and feelings? How do they contribute to my perception of the problem?
  • Is the problem based on hard facts and proven information?
• Fit - Why is it necessary to solve? Should I really solve this problem based on my
  • needs
  • priorities
  • values and principles
  • resources (time / money)
  • potential benefits / risks?

• Purpose - what is the purpose of solving the problem?
• Stretch - should I solve bigger problem?
• Squeeze - should I solve only part of the problem?
• Model - what are elements and parameters of the problem? What are forces?
• Question - use CIA Phoenix checklist of problem solving questions

Ideal solution

• Find contradictions that can be resolved - I want more money, but also want to work less and impossible - I have to be in two places in the same time. Use them as a creative stimulus to define an ideal solution.
• Understand when you can stop, what is good enough outcome and how much time, energy and money you are willing to spend on resolving a problem
• What are potential harms and cost of the solution?

Knowing

1. Is the situation same?
2. Should you improve or change something from your previous solution?
3. Can you come up with better alternatives?

Copying

1. Can you trust sources? Do they provide full unbiased information?
2. Do you pay attention not only to glorious outcomes of the solution, but also to problems?
3. Can you use this solution for your specific problem, situation and abilities?

Logic

Reorganize - break down the problem into elements or parameters and manipulate them. You can use SCAMPER method defined by Bob Eberle. (Example: a problem with failing relationship)

• S - Substitute - components, materials, people
  • substitute something in your behavior or find another partner
• C - Combine - mix, combine with other assemblies or services, integrate
  • start doing some things together
• A - Adapt - alter, change function, use part of another element
  • change your home responsibilities
• M - Modify - increase or reduce in scale, change shape, modify attributes (e.g. colour)
  • communicate more often, change your tone
• P - Put to another use
  • use your partner as a tennis or chess opponent, engage as an assistant in fishing
• E - Eliminate - remove elements, simplify, reduce to core functionality
  • separate or stop doing things that annoy your partner
• R - Reverse - turn inside out or upside down
  • say opposite to what you said before

• Think about a specific topic you want to cover. Find out how people start blogs in general, what works and what are common pitfalls. Apply to your topic and situation to grow successful blog.

Analyze Variations (Example: Develop and promote new Content Management System)

1. Parameters of a problem - list variations, combine differently
   • cost, time, features, people, quality, vendors, etc; time vs features, in-house vs outsourcing, quality vs speed
2. Attributes list - list attributes of the object (or problem) and try to improve them (Attributes analysis, Morphological Analysis)
   • architecture, design, performance, scalability, security, reusability, networking
3. Matrix - create related keywords in main areas, mix and match keywords to form new ideas (Leonardo da Vinci Ideabox)
   • solutions (blog, company website, e-commerce, news, social connections), needs (inform, advertise, sell, support), markets (individuals, companies, non-profit, schools, small social groups), technologies (web, video, mobile, aggregation), services (hosting, installation, support, training). For example, what could be opportunity in combination: blog + inform + schools + video + hosting

Change Context / Process / Forces (Example: Career advance)

1. Force field analysis - maximize positive forces, minimize negative (Force-Field Analysis)
   • find how to use your strengths more and avoid negative impact of weaknesses
2. Substitute rules, principles, people, places
   • change job responsibilities or job itself
3. Anti-action - compensate harmful effect by specific action
   • minimize overtime or uninteresting work by changing how you accept your tasks
4. Preliminary action - pre-arrange or change problem context in advance
   • build consensus with potential opponents before important meeting
5. Theory of Constraints - find the worst bottleneck and improve until it is no longer a bottleneck, continue with the next bottleneck (Theory of Constraints)
   • find what are limiting factors of your growth, work on reducing their impact and increase your abilities and potential
6. Eliminate waste
   • eliminate wasteful activities at work to boost your productivity
7. Lemonade from Lemons - use waste or harmful effect to achieve a positive effect
   • learn from your mistakes and use them to improve and become better next time. As Friedrich Nietzsche said: “What doesn’t kill us makes us stronger.”
8. Feedback - introduce or increase feedback to improve action or process
   • seek for feedback and objective evaluation of your performance to advance faster by fixing your shortcomings

Explore future scenarios - project future scenarios to find unexpected opportunities (Alternative Scenarios) (Example: Deciding where to invest money)
“The best way to predict the future is to invent it” – Alan Kay

1. Identify forces
   • identify current economic forces and your personal situation
2. Come up with 4-5 scenarios - good and bad
   • what could happen in the future with both economic and you?
3. Search big opportunities in each
   • find how you can benefit from investment in each of scenario

Lateral thinking (Creative Spark)

Challenge assumptions, break away from obvious perspectives (Assumption Busting) (Example: Raising troubled teenager)
“The significant problems we face cannot be solved at the same level of thinking we were at when we created them. “ – Albert Einstein

1. Expose and challenge all assumptions
   • challenge your beliefs about role of a parent
2. Reverse problem in some way (Reversal)
   • praise for something you was punishing before
3. Paradox - opposites exist simultaneously (True and False)
   • accept that you and your teenager are both right about controversial topic. Try to negotiate from this position
4. Provocation - wishful thinking, thought experiments
   • unleash your imagination, go for crazy ideas about solving a problem
5. Changing viewpoint - think from different perspective (Six hats)
   • imagine that you are a bird or live in another time. How would you resolve your relations with kid?
6. Finally, challenge any part of the problem – ask why for anything

Force Associations (Example: Starting your own software business)
“Creativity is the power to connect the seemingly unconnected” – William Plomer

1. Random keywords - select any word from dictionary and try to connect to your problem (Random Words)
   • For example, how a tree can be related to your new business?
2. Words of great thinkers - use their quotes and wisdom to associate with a problem.
   • For example, how can Oscar Wilde’s quote: “Be yourself; everyone else is already taken” helps with business ideas?
3. Free associating with common attributes - select 12 attributes, choose random 2, consider separating and combined (Combinatory Play)
   • break down into areas – finances, sales, development, clients, technology, usability, etc. Start pairing randomly
4. Metaphors - similarities and connections of your problem with other distant area
   • How analogies of railroad or tropical forest can help with business idea
5. Hieroglyphs - use Egyptian hieroglyphs to associate with your problem
   • arrange randomly hieroglyphs from The Book of Dead to find clues for creating business

Visual clues (Example: Designing web user interface for the new complex domain)

1. Diagram to explore, organize and analyze (Lotus Blossom, Mind Map, Concept Fan)
   1. Explore – search for new areas that can be related to the problem
   2. Organize – put together as one picture what you know about the problem
   3. Analyze – go deep into a problem to find missing details and relations
   • Draw these 3 types of diagram for web site and business domain: put web site purpose in the center, start drawing branches for big ideas and aspects, branch for smaller and so on
     
2. Draw and link abstract symbols for attributes
   • create symbols for web patterns and business concepts, shuffle and find ideas in random associations
3. Drawing - draw anything that can be related to your problem; lets your subconscious drive your hand (Doodling)
   • just draw any screen ideas

Psychological tools

“Everything you can imagine is real.” – Pablo Picasso

Use intuition and subconscious to find answers. Our subconscious is a large scene where conscious thought is a spotlight. Try to elicit ideas from these dark corners of the most powerful part of your mind. (Example: Finding new kicking-ass website idea)

1. Intuition - ask and listen to your intuition
   • come up with website ideas without thinking
2. Incubation - best ideas come than you don’t expect them, just don’t miss them
   • deeply dig into a problem, put into your mind and forget, and probably soon you will jump from the bath and run to the street screaming ‘eureka!’
3. Fantasies - unleash unreal or crazy questions, suspend judgment and answer them (Crazy Ideas)
   • what if you build site for aliens? now you can think about usability for complete dummies, space design, navigation as a star system, etc.
4. Dreaming - capture ideas from dreams
   • set questions before you sleep, try to recover answers from dreams immediately after you wake up
5. Drink beer - to slip into different mind state. I don’t suggest becoming drunk or use something stronger or illegal :)
   • drink and start thinking about website, don’t foget to record your thoughts early enough :)
6. Surrealistic images - find association of hypnogogic imagery with your challenges
   • relax, quiet mind and eyes, become almost unconscious- record experiences immediately after they occur
7. Guided imaginary scenarios to find ideas in unexpected places (Imaginary Excursion)
   • choose scenario (e.g. going to camp and lost in the forest) and while you experience the journey in the head, search for clues
8. Psychosynthesis - spiritual advisers (Role-playing)
   • invite Bill Gates, Sigmund Freud or Karl Marx to talk with you inside your head about the kicking-ass website
9. Streaming - writing out what is in your mind to associate later with the problem
   • hide in a comfortable place and start writing – smart thoughts or gibberish. Find how it could be related to the new website later.

Dialogue

1. Talk with strangers
   • ask about vacation ideas people you don’t communicate often and you’ll get plenty of new ideas
2. Brainstorming - exchange ideas and suggestions without judgment, evaluate and revise ideas later (KJ Brainstorming)
   • ask group of people to generate as many as possible vacation ideas without discussion
3. Brainwriting - parallel generation of ideas in silence (index cards); exchange and develop each other ideas in turn (Brainwriting)
   • everybody in group writes vacation idea on card, give to the next in circle who continue refining this idea
4. Survey - collect ideas from large group of people (Crawford Slip Method, Delphi method)
   • post survey (e.g. Facebook or surveymonkey.com) for vacation ideas
5. Storyboarding - put ideas on the wall as a group works out problems
   • post vacation ideas on the wall and let anybody to contribute (add or extend idea)
6. Socratic dialogue - have a conversation with a smart person and probe various dimensions of thinking: purpose, evidence, reasons, data, claims, beliefs, interpretations, deductions, conclusions, the implications and consequences of thought, response to alternative thinking from contrasting points of view, and so on.
   

Trials and Errors

Solving a problem by trial and error is an expensive, but reliable way to find a solution. In science it called scientific method. Most scientific discoveries are done using this method. (Example: Becoming happy)

1. Ask a Question
   • form a problem statement – what is happiness for you?
2. Do Background Research
   • learn what other people think about happiness
3. Construct a Hypothesis
   • what could make you happy?
4. Test Your Hypothesis by Doing an Experiment
   • try your ideas in practice
5. Analyze Your Data and Draw a Conclusion
   • are you happy now? should you change a hypothesis and start again?
     
6. Communicate Your Results
   • share your results – help other people to become happier

Summary

Your life success depends on your ability to choose right problems and solve them. Do not give up or settle for an ineffective solution. Use the power of your mind to creatively solve problems the best way possible. Control your life.

“He is able who thinks he is able” -Buddha

References

• Thinkertoys: a handbook of creative-thinking techniques, Michael Michalko
• http://www.mycoted.com/Category:Creativity_Techniques
• http://creatingminds.org/
• http://www.mindtools.com/pages/main/newMN_CT.htm
• http://www.creativethinking.net/WP03_Techniques.htm

His main points:

• Ideas. The best ideas are not coming from users and they are bad in providing feedback (iPod). So, there is no point to release early to get their opinion and ideas.
• Features. Minimal early set of features could be unattractive for majority of users and will turn them down for future use (Apple Newton)
• Quality. A buggy and unpolished product could ruin your reputations
• Architecture. An incorrect initial architecture creates waste and serious problems down the road (Netscape, Twitter)

Therefore, Jason against releasing early and often. I don’t agree.

My answer: it depends!

Evolution is the process of small frequent changes to improve and adapt to environment.

One of my posts reviewed how software team should select the best strategy: evolution, revolution or retreat. Another post compared evolution to revolution. Evolution (and aligned core Agile principles) requires early and frequent releases. Revolution pushes innovative product that should disrupt market. Requirements for a revolutionary product ideas are much higher, because it should overcome resistance, create new niche and gain acceptance of the new paradigm. Therefore, revolutionary product should be well thought and prepared to hit the mark. I should note that revolution often becomes evolution after initial release.

Decision factors for selecting strategy Release Early, Release Often (evolutionary):

1. People. Are they highly talented and capable to come up with great ideas from the beginning ? Or should they learn and understand better the problem space, release incrementally and get more feedback from the users for initial assumptions?
2. The Game. Is the problem (software requirements and business domain) unclear, complex, challenging and require a lot of trials and feedback? Or is the problem space well known and team already have good experience with it and can release a great product from the first attempt?
3. The Dynamic. Is the team under a pressure to release a product early, catch at the market opportunity or help a company to survive? Or do they have luxury to take a time for designing properly and release a polished product?

There are some cases where long release cycles make sense for the company:

1. A company has deep expertise and hired highly talented and experienced people who know how to build successful product.
2. A company have enough funds and time to sustain long development and tolerant to inefficiency because incorrect assumptions or lack of feedback.
3. Customers have low tolerance for risk and the software is mission- or life-critical.

However, I expect that for many software teams the most optimal strategy will be evolutionary - Release Early, Release Often.

I partially agree with Jason that most users will not help with ideas or provide meaningful feedback. Also it is a bad idea to ship buggy, unfinished and useless product. But a team could get more than user feedback from early and frequent releases:

• release! This is a huge and most important criteria of development success – the product is releasable and working
• integrate and put the product pieces together, which is almost impossible to replicate in the test environment
• practically experience work of the live product - infrastructure and production problems, performance, scalability and user interactions in their environment. These problems can radically change the view on architecture
• reality check - learn from how people use the product: validate initial assumptions, collect praises and complains, real usage patterns, statistics and maybe even some ideas from users :)
• emergence of the new ideas and process improvement after experiencing product in the wild live environment

A software company should avoid treating users as guinea pigs for their early experiments, but nothing can beat practical benefits of releasing code and learning from it.

“In theory there is no difference between theory and practice. In practice there is.” — Yogi Berra

Referenced post: Releasing Early Is Not Always Good? Heresy! by Jason Cohen @onstartups.com

Anxiety started to penetrate our souls. We know what is wrong: our team is short of people, we have too many commitments, our code is becoming a big mess, new technology and our new software version makes something bad with servers. A snowball of different problems makes us stressed, distracted and incapable of productive work.

What could our manager do next?

1. Distrust. Become a dictator, make own decisions including hiring external consultants to recommend what to do or even replace us. However,
   • we are good programmers and know our business well – the problem is not in lack of skill and knowledge
   • external people will take a lot of time to understand the system and they will have different motivation and won’t care about the long-term success
   • people will be demotivated and the manager cannot make effective decisions without active team involvement
2. Faith. Give to team the full power to fix a problems and make own decisions in hope that smart people, motivation and technical expertise will do magic. However,
   • fresh outlook and thinking out of box are hard when a team immersed for a long time into difficult situation
   • a team possibly doesn’t have understanding and control over external forces – management, customers, finances
   • changing of reality is tough (especially in people heads) and requires more than technical experience

There is a third way. Place improvement practices in the core of development process. Make self-improvement inevitable and required for any activity. Do it every day.

Toyota Way is the best example of large-scale reliable self-improvement process. It focuses on eliminating waste, solving problems at root cause and making right decisions. Toyota Way reduces problems, increases internal efficiency and makes a company successful. This is the best receipt for coming out of crisis.

Targets:

• Problems - emergencies, fires that require immediate fix: bugs, server crushes, deadline slips

• Waste - inefficient and non-value adding activities: waiting, misinformation, stress

• Challenges - adaptation to external forces (market, competitors, customers, society): new trends and technologies, changes in users expectations for user interface and functionality

Practices to see waste and stop to fix problems

1. Seeing waste

The team and managers should learn to see targets – real problems and waste. Otherwise improvements will be wild shots in the dark.

There are many targets in software development:

• stressed people – reduced energy, less productivity, more mistakes
• waiting – delays, tools / system problems and downtime, capacity bottlenecks (waiting for results of other people work)
• over-engineering – producing features and complicate design without real need
• unfinished work – functionality not used in a live system, probably still in design or under development or simply discarded (but not removed)
• defects – complete waste of time and money
• unused creativity – loosing time, ideas, skills, improvements and learning by not engaging or listening to your employees
• inadequate information – unclear, misleading or simply wrong information that causes useless activity and leads to rework at the end

• over-processing – taking unneeded steps to build software because of poor process and system design, overhead, bureaucracy, compliance, cumbersome tools
• motion – how much effort to get necessary information, access systems or use tools
• multi-tasking – losing time to switch between projects, tasks or different activities

2. Jidoka - stop to fix the problem to get quality right the first time

It is not enough to see your targets. The Team should carry commandment to shoot targets immediately. Otherwise the best intentions will be buried under growing avalanche of problems.

How to stop:

• quality for the customer drives all the processes – prevents temporary patches and bad for quality decisions
• low tolerance for quality problems and immediate detection are core work principle
• use Andon – a system to signal for help, notify about a problem and stop the process

• build in tools and process capabilities of detecting problems and stopping itself
• use all modern QA methods available
• managers encourage stops to fix problems and support implementation of counter measures

Making Decisions

“Make decisions slowly by consensus and thoroughly considering all options, implement rapidly.“ – Toyota Way

Even knowing problems and committed to solve them, the Team should make right decisions how to do it.

Make decisions slowly

1. Go and See (Genchi Genbutsu)
2. Understand underlying causes (Kaizen)
3. Broadly consider alternative solutions and develop a detailed rationale for preferred solution delaying certain decision as long as possible (Set-Based concurrent engineering)
4. Build consensus within a team and partners where group decision is preferred (while management can step in if consensus is not achieved)
5. Use very efficient communication devices – preferably one side of one sheet of paper

Implement rapidly

1. Put in place solutions and counter measures.
2. Evaluate the results
3. Standardize if solution is effective.

Practices to Eliminate Waste and Solve Problems

3. Genchi Genbutsu – go and see for yourself to thoroughly understand situation

How often do we jump to conclusion based on partial information, vague assumptions and what other people say? Information creates reality in your mind. This reality is a base for your decisions. So, you and the Team should get right information to make right decisions:

• observe situation with blank mind
• avoid assumptions and preconceptions
• use personally verified information

In short, base decisions on what really is going on

4. Kaizen (5 why’s) -continuous learning and improvement

“We view errors as opportunities for learning” – Toyota Way
The Team should find the root causes of the problems. Kaizen helps to find the root cause by repeatedly asking why the problem occurs.

Example of Kaizen
Problem: there are persistent javascript errors on a live site

1. Why? A developer didn’t build correct logic for complex web UI components interaction
2. Why? A developer built own solution without guidance and enough experience in this area
3. Why? A team expert didn’t tell about existing proven solutions, didn’t help and didn’t share knowledge
4. Why? The team is under stress, over-committed and don’t have time to communicate
5. Why? Managers accept too much work without consulting with development team
6. Why? you can continue…

Kaizen forces us to overcome desire to find a first convenient explanation and patch problems without fixing root causes. By ruthlessly applying this practice, we get deeper insight into reality and better learn our product, processes, people, environment and tools. Kaizen is a core practice to see waste, solve problems and improve process.

To avoid forgetting learning from Kaizen, it is important to standardize the improved process and make it a base for further improvements.

Practices to Support Flow

5. Standards – best you know today which is to be improved tomorrow

Standardized work is easier, cheaper and faster – stable repeatable methods can maintain predictability, high productivity and support quality.
Effective standards are not coming from theories, they come from

• best practices
• accumulated learnings and individual experience
• lessons from applying existing standards

The Team should try to use standards in many areas: project phases and activities; development practices; architecture and design approaches; code conventions; tools; programming techniques; libraries and third-party code; reuse of components and solutions; testing and so on.
Standardization in software development is a controversial topic – some theorists want to bring programming closer to standard-dominated engineering, practitioners are keen to reduce standardization to minimum promoting creativity and self-organization. In the rigid interpretation, standards are “must to follow” rules for any situation, in other interpretation standards are well defined steps and guidelines highly recommended for specific context. I support the latter definition. A productive team should have standards in place to focus on customer needs instead of fighting with the same puzzles and problems over and over again.

The system of standards shouldn’t be a heavy bureaucratic conduit, but a light and fluid book of knowledge. The book that contains most helpful and important rules and checklists. Standards will be effective if they are minimal, reviewed often (Kaizen) and followed by every team member.

6. Reliable thoroughly tested technology

The Team should be conservative with new technologies. Software development and IT thrive on change and innovation. However, Toyota Way suggests to be conservative in adapting technology and considers stability and reliability of operations as much more important goal than keeping on the cutting edge of technology.

Considerations for using technology

• primary goal is to improve flow and support people, process and values.
• process is driven by business, not technology concerns; software and tools do not eliminate themselves waste
• technology is visual and intuitive – people can use it correctly and effectively

• process manually before adapting technology to support the process – understand what problems it solves and how technology could help
• important: people / processes are easy to Kaizen, machines are difficult

Adopting new technology:

• new technology is unreliable and difficult to standardize, therefore it endangers flow
• proven process takes precedence over new and untested technology
• conduct actual tests before adapting new technology
• reject technology if it conflicts with culture or might disrupt stability, reliability and predictability

In the same time encourage people to consider new technologies while looking into new approaches. If technology improves process and flow – quickly implement after thorough testing.

7. Visual Controls

The Team should have clear status of information. Visual controls can convey complex information in fast and efficient for our brains way. We can use controls as a user story board; status of projects, servers or code build; burn down charts and others.
Simple visual indicators help people determine immediately whether they are deviating from the standards, provide quick gist of situation and direction for solving problems.

• use simple and most important indicators

• than provide clear picture for decisions and what to do next
• reduce reports to one screen / piece of paper even for the most important decisions

People, leaders and teams

8. People

People who build software are the people who should improve the process. They are directly involved and have first-hand experience of problems and waste.

Toyota Way expects that each team member is a problem solver and values job experience more than theoretical knowledge. The Team will beat any external consultants and find better way to work if people are open about problems and eager to find good solutions.

9. Leaders

The Team desperately needs strong leaders to build great products to overcome problems. Toyota grows leaders within who thoroughly understand the work, live the philosophy and must understand the daily work in great details

Chief Engineer is a key person in Toyota projects.:

• blessed by top management
• has control over project
• exceptional engineer
• critical link: between engineers and customer satisfaction
• coach for other engineers
• focus on concepts first, technicalities later

Chief Engineer concept is an excellent example for software technical leadership. A software team leader often lacks authority or makes too technical decisions without good understanding of customer needs.

10. Teams

The Team should be diverse and capable of solving wide range of problems. Toyota builds cross functional product teams, which

• use integrative decision making
• fast and accurate in implementation
• enhance process and flow by solving difficult technological problems

Software developers and their leaders are foundation of success in any project. Management, process and technology can only support them. And anyway, the process is as good as people follow it. Therefore, it is important to make software teams a key player in process improvements – they know problems, they understand work and they are capable to find good solutions.

Using Toyota Way

Can The Team revert a situation and win? Can it build the optimal process and expertise for fast development of high quality and low cost solutions?

This post shows the most effective option – build continuously improving process into the heart of development. The process that focuses on quality, eliminates waste and fixes problems at root cause. I believe this approach is a foundation of long term success. Your managers and company would love it!

An expert have much better models of reality and methods to build them than an ordinary specialist. The expert, armed with these models, can quickly put pieces of a problem puzzle together, find explanations and solve the problem.

Models can be related to anything – software systems, business domain or your personal relationships.

Predictions and Solutions!

A theory is a good theory if it satisfies two requirements: It must accurately describe a large class of observations on the basis of a model that contains only a few arbitrary elements, and it must make definite predictions about the results of future observations.

Stephen Hawking, A Brief History of Time

Any expert as good as he can effectively predict future outcomes and solve new unforeseen problems. Solutions and predictions are not coming out of blue. They stem from good understanding of reality in an expert’s head. Internal models of reality help to explain problems, find relations and play with future scenarios. These models establish the base for effective thinking and direct expert’s effort to solve problems. If you have poor models of reality, you will make bad decisions. Good models of reality allow reliable predictions and efficient solutions than vague guesses and ad-hoc spontaneous fixes.

Characteristics of a good model:

1. Explanation - provides understanding of elements, processes, events, root causes and effects
2. Testable predictions - a model that makes no predictions that can be observed is not a useful model.
3. Simple - you can remember and use a model without titanic effort
4. Frameworks - knowledge how to recognize common patterns and deal with range of similar problems. Frameworks allow to extend and apply existing models to new situations without building new models.

Brains – the main tool of an expert

I think, therefore I am – Rene Descartes

An expert’s brain carry and operates with models of reality. Unfortunately, our brains are not ideal for this task. Models they create are not identical to reality. Here is why.

limitations

• perception - our senses deliver to brain partial and incomplete information
• memory - our brains can process and memorize only fraction of available information; and even more – we quickly forget this knowledge without practice
• imagination - brains automatically filling gaps for missing information, substitute with interpretation based on internal representation of the outside world

cognitive biases (bugs in brain software)

• anchoring or priming – previous irrelevant experiences prime your consequent thinking and decisions
• need for closure – we are uncomfortable with doubt and uncertainty; we have urge to resolve and find convenient explanations quickly without much thinking
• confirmation bias -  from all facts we unconsciously pick facts that confirm our beliefs and predispositions
• symbolic reduction – we are anxious to reduce complexity for better understanding and tend to form simplified generalizations ignoring complex details and relations.
• and many others

social behavior

• subconscious models and believes – our minds carry beliefs and views imposed by our parents, teachers and culture.  These subconscious models deeply influence our models of reality without much awareness.
• peer pressure, conformity – our minds are wired to conform and agree with other people making us blind for inconvenient facts, ready to follow crowd and accept conventional views without critical revision

Objective Reality

There are no facts, only interpretations.  – Friedrich Nietzsche

Even smartest brains will have challenges to understand reality.

We deal with complex situations:

• incomplete information – we rarely have access to full information and even if it is available we don’t have capacity to collect and absorb all of it
• perpetual changes – reality is very fluid and changes every second
• complex relations and behavior – complex systems are difficult to understand: causes and effects, correlations, feedback loops, influence of processes in external systems
• difficult to test – we often cannot afford large number of experiments, trials and errors to come up with right models and explanations

We deal with people who make our reality very complicated:

• unpredictable - people are difficult to predict, they are often irrational and inconsistent
• confusing - people cannot clearly explain their thinking and feelings or even don’t understand themselves
• deceiving - sometimes people have difficulty to tell truth or simply lie for own advantage

Building Models

The test of a first-rate intelligence is the ability to hold two opposed ideas in the mind at the same time, and still retain the ability to function. -  F. Scott Fitzgerald

We understand reality different ways – learning, investigating problems, finding new solutions and practicing. For years we can concentrate on solving immediate problems and do our work without attempts to understand the big picture. Many good specialists have incomplete understanding and fragmented knowledge that still allow to perform well. But it is impossible to become an expert without deep understanding and conscious effort to build good models of reality.

For example, we can stumble for a long time with challenges and failures in building software without understanding its essence and forces. (By the way, search for this understanding is the reason for this blog existence).

Potentially everything what we do in life could help to build and improve our models of reality. We just need to consciously relate our experience and learning to our understanding of reality. And we can do it on the go without special preparation, when we actively engage our minds – working, talking, reading, practicing, traveling, thinking and even sleeping. Just focus on reality modeling.

The models of reality could be explicit (facts and theories) and implicit (intuition and tacit knowledge).

You can use several strategies to build models in your whole mind.
A. Analytical

• find what is most important, map and connect main elements, players and subsystems; learn relations; draw Mind Maps
• understand influence of context and main forces
• create stories, metaphors and patterns – make your models better suited for your brain
• repeat the process for the most important subsystems

B. Empirical

• find most pressing problems, contradictions or real life facts
• understand how do they fit into your models of reality
• it is great if they don’t fit – you have opportunity to improve your models

C. Synthesis – reconcile your models with existing knowledge and leading models

• find what are dominating models, views and believes; what other experts think
• critically analyze this information and theories behind
• find the best opposing theories and try to combine them to form better model

D. Scientific method – use it for practical evaluation of models and confirmation of your theories.

• build hypothesis about reality and outcomes of your actions
• test it in practice
• learn from results and adjust your models

E. People interests and agenda – people are often most surprising and unpredictable element of reality. Your models will be much closer to reality if you understand motives, interests and agenda of involved people.
F. Change perspective – switch different perspectives to enrich models – economic, emotional, social, physical, etc. Try to view from opposite viewpoints and set of principles. Imagine yourself as user of your system, investor of your company, manager, any animal, computer part, etc. (Try to get back without loosing your identity :))  Web developers have interesting concepts of Personas – imaginary users with distinct behavior on website that help to design a system for various cohesive sets of needs.
G. Lateral thinking

• random association – associate models with recent books, distant knowledge fields, your favorite characters
• provocative thinking – make unacceptable, funny, stupid ways to explain problems and shock your models
• ask “why?” – continue asking ‘why’ until your reach questions that nobody could answer on this Earth (or the only answer is 42).
• expand concepts borrowed from other sources to have wider horizon for ideas

In overall, you will switch between two grand strategies – Evolution (discover reality and adopt to it) and Revolution (breakthrough and change reality). The choice depends on models maturity, certainty and your people capabilities.

Therefore

Few people have the imagination for reality.  – Johann Wolfgang von Goethe

You can be certain that reality in your head is different from objective reality, which is complex, changing and controversial. Effective experts don’t have choice and master reality by
1. Accepting own limitations in understanding of elusive reality
2. Building good models of reality applying different strategies
3. Using, testing and improving models in practice

Good models of reality will make your thinking, decisions and solutions much better. They will make you a master of reality instead of a victim of reality. Good luck with embracing reality!
References:
The Opposable Mind: How Successful Leaders Win Through Integrative Thinking , by Roger L. Martin
Pragmatic Thinking and Learning: Refactor Your Wetware, by Andy Hunt

Disclaimer: This post is devoted to a person who wants to become an expert – the top player in a specific field as programming, soccer or chess. This post will be not interesting for people who are satisfied with their current performance and not interested to be the best.

If you want to become an expert, it is not enough to follow your work assignments or occasionally play with interesting stuff at home. You have to push yourself hard in specially designed way.

@Work
Your paid work tasks and projects are not designed to make you an expert. Your company expects results from your work: reliable, with minimal mistakes and focused on the company main goal – make money. Your employer could provide minimal training to help you with job requirements. However, your growth will be constrained by company needs, timelines, work assignments and acceptable methods. We cannot blame our organizations – this is part of the deal – they pay for your work and expect specific results. But… is this the best way for you to become an expert, acquire new skills and gain knowledge? To become an expert, you have to make many mistakes, learn from them, experiment with alternatives and work hard on your weaknesses. How many organizations do allow this risky, unproductive and unreliable way of working?

@Home
Your play at home with interesting stuff has problems too. To satisfy your programming instincts and curiosity, you will probably select what you enjoy to do and eager to try. You’ll immense in this activity and find great satisfaction from doing it. But…  is this the best way to become an expert by doing only what you like? Becoming an expert requires hard, sometimes unpleasant work, specifically designed to improve your performance and push you over comfort zone.

Genesis of an expert

Research shows that experts do not have more innate talent, but put more effort and dedication. Newcomers to the field need at least 10 years to become an expert. They move through several stages: from a novice, who blindly follows rules, to an expert, who does not need rules.
Most people will stop on the level of competent specialist acceptable for the majority of jobs in the field. Only few will continue pushing forward to achieve the top level.

We agree that expert performance is qualitatively different from normal performance and even that expert performers have characteristics and abilities that are qualitatively different from or at least outside the range of those of normal adults. However, we deny that these differences are immutable, that is, due to innate talent. Only a few exceptions, most notably height, are genetically prescribed. Instead, we argue that the differences between expert performers and normal adults reflect a life-long period of deliberate effort to improve performance in a specific domain. – Anders Ericsson

Even the novice engages in effortful study at first, which is why beginners so often improve rapidly in playing golf, say, or in driving a car. But having reached an acceptable performance–for instance, keeping up with one’s golf buddies or passing a driver’s exam–most people relax. Their performance then becomes automatic and therefore impervious to further improvement. In contrast, experts-in-training keep the lid of their mind’s box open all the time, so that they can inspect, criticize and augment its contents and thereby approach the standard set by leaders in their fields. – Scientific American

Kathy Sierra wrote that you, to become an expert who kicks ass, should keep reminding: “I’ll keep pushing myself. There’s always some way to do it better…”. How should we push ourselves?

If you do same tasks over and over again, you will not become an expert. If you do what you are good at, you will not improve your overall performance. You should focus on improving skills beyond your present competence. Outcome of your practice as not as important as developing your capabilities.
You can spend years doing your favorite activities and still stay at amateur level while dedicated novice can quickly leave you behind. Practice should concentrate around your weaknesses and areas that are important, but you probably try to avoid. You should repeatedly progress to the higher level through persistent training and overcome your limits.

Anders Ericsson (his research is the most influential in this area) calls this approach deliberate practice:

“Deliberate practice is different. It entails considerable, specific, and sustained efforts to do something you can’t do well-or even at all. Research across domains shows that it is only by working at what you can’t do that you turn into the expert you want to become.”

Design your practice

Follow 7 steps

1. Who is an expert?
You should understand what is the profile of an expert that you want to be – required capabilities, knowledge, areas of expertise and types of the problems experts work on.
Have a beer with guru and ask for opinion (early enough), study job advertisements for high skilled positions (don’t be carried away), read about top players – why they are considered experts, what makes them special (skip part how they spend money) and how they achieved top level.

2. Find your gaps and weaknesses.
You can identify areas which require improvement once you know the profile of the ideal expert. Put them on the list and assign importance.

3. Build roadmap.
Answer the questions: in what order you want to work on these areas and what are the major steps.
Instead of focusing for a long time on one area, I would recommend to go in spiral – learn and practice various areas in parallel. Each spiral cycle should add to your experience small, but integral chunk of core expertise. Often developing diverse skills together brings synergy – deeper perspective, holistic view and increased capabilities for wider range of tasks.

4. Design practice to improve performance
Set specific goals and methods – how do you improve performance, gain skills and knowledge for each area.
Consider famous SMART goals for practice: specific, measurable, achievable, relevant, time-boxed. Coach, who can effectively guide you, is very helpful on this stage. However, you can go alone if you are motivated and disciplined enough.
Don’t burn yourself too much – your mind and body should be ready and eager to practice. Give yourself time for recovery, life and other favorite activities, otherwise you’ll soon hate your practice.

5. Everyday practice on the edge of capabilities – at least 30 minutes
Deliberate practice is highly demanding and requires intensive concentration and focus. The practice should be scheduled, clear and focused – don’t give yourself chances to procrastinate. Concentrate on the excellence of execution – sloppy practice is not acceptable for top players. You should practice challenging tasks on the edge of your capabilities. Compare playing basketball with professional players or beginner kids from the neighborhood. Who will push you more and show how good you are?
Observe and understand how and what you do – this process will develop intuition and patterns in your brains for future fast automatic decision making. Train yourself to think and act on the higher level – what is difficult for other people, should be trivial for you.

6. Feedback
Constant feedback loop is the must in deliberate practice. Correct assessment of performance and results with following adjustment of practice will move you much faster. And certainly, interested and qualified coach capable of giving constructive and painful feedback is very important. Each day of practice should move you forward against higher and higher standards.

7. Correct your roadmap and practice.
Feedback, self-evaluation and practical use of your skills will tell if you are on the right track. Be fluid and open for change of direction, new knowledge and unexpected opportunities. Learn and adjust goals, plans and methods on the go. As Eisenhower said: plans are nothing, planning is everything. Your rise as an expert will open for you new horizons that definitely change your plans and original intentions.

Additional Requirements

• Discipline, motivation, unsatisfaction – top experts are extremely dedicated, disciplined and motivated to grow and improve performance. Without the goal of improving performance, the motivation to engage in practice vanishes, and practice becomes boring routine.
• Support – you’ll have ups and downs, moments of despair and disbelief. You need somebody who can understand, compassionate and encourage – friends, family, colleagues or a coach.

Deliberate practice in context

Practice @Work
Deliberate practice methods and work objectives are different. However, it is possible to grow as an expert and work productively.

• First, your company could have work in target areas of your expertise growth. Position yourself to take advantage of any opportunity to move there.
• Second, any company will benefit from motivated and competent player, who wants to grow his capabilities. A smart company will support and encourage your drive… and use for own benefits.

Strategy:

1. Job intelligence – what are opportunities, potential projects and tasks to assist with your progress as an expert? How can your help your company while upgrading your skills? Invent new opportunities. You have to build case for your advance and show benefits for the company.
2. Call for the job redesign – many companies will be flexible for people who wants to progress and contribute more (if no, maybe you are in the wrong place)
3. Productive work and intensive practice – work in the best way to help with your goals, but don’t make a company victim of your practice. Separate prototyping and experiments from production work. Make them open, controllable with a clear statement of benefits. Seek for challenging assignments on the edge of your capabilities.
4. Tap into company pool of talents, expertise and knowledge – this is free (actually paid) opportunity to use coaching and valuable resources for growth acceleration. Ask for advises, guidance and feedback.

You will find, as your star is rising (expertise growing), that you have more freedom to work in areas that have the most interest for you.

Practice @Home, @Train or @Cafe
This is time for

• research, learning, thinking
• crazy experiments
• irrelevant to your work practice
• practice that need special focus, time and conditions, difficult to achieve at work

Formal training
Qualified and relevant training can move you much faster. If you decide to invest in education be very specific what you expect and make sure that you’ll not waste your time.

Sample Road to become Web Development Expert
You can identify few key areas:

• HTML / CSS
• Graphic Design / Art
• Usability / Aesthetics
• Information Architecture
• Client – side programming / javascript / AJAX
• RIA (e.g. Silverlight / Flash)
• Server – side programming and databases

And design practice for each of them, for example:

Experts: Executive Summary

Experts are essential for the company/team competitive advantage. They enable revolutions and support evolution.

Experts are made, not born. Experts are made by themselves. It is a long, thorny and hard road. But this road makes their life interesting, positive, meaningful and brings happiness from achieving a rare gift – mastery in the field they love.

Albert Camus said: But what is happiness except the simple harmony between a man and the life he leads?

For few people the harmony is a journey to become The Master.

Resources:
The Expert Mind , Scientific American
How to be an expert , Kathy Sierra
The Role of Deliberate Practice in the Acquisition of Expert Performance, K. Anders Ericsson, Ralf Th. Krampe, and Clemens Tesch-Romer

source

Qualities of well composed code:

1. Quick discovery and understanding of programming logic and components
2. Clear organization (for human brains)
3. Ease of reuse, modification and evolution
4. Close connection between customer ideas and system implementation

Style Components:

• Intention - understand your task and how to get it done
• Approach - basic principles of writing code
• Composition - organization of code
• Expression - expressing ideas in code
• Object Oriented Pragmatic Style

1. Design for customer problems
Make the customer problem space a core for the software system design. Make every line of code accountable for solving customer needs. It is true that any software system requires a lot of plumbing beyond pure implementation of customer needs. However, technology concerns shouldn’t prevail. High-level languages and modern programming platforms give us a power to concentrate on customer problems more than on solving technical problems.

Code could be divided into two categories:

• customer oriented – solves customer problems directly, e.g. implementing domain logic, UI interactions and display of information;
• system oriented – solves technical and application specific problems – data access, event handling, interfaces between subsystem, utilities and other code required for using libraries, systems and platforms.

Customer-oriented code brings most value and should be primary design concern. System-oriented code puts in place infrastructure and support for running customer oriented code.

Therefore, start with design of customer-oriented code and support it with minimal system-oriented code. This doesn’t mean that system-oriented code is not important. Complex system environments, sophisticated user interfaces and challenging non-functional requirements (performance, availability, reliability, etc.) could demand major system-oriented development effort. But still customers needs should pull and direct this effort and the whole implementation.
There are many ways to make technical problems dominant over customer problems:

1. Making predetermined technical decisions (databases, programming platforms, messaging systems, expensive middle tier etc.) without considering simpler alternatives.
2. Coming up with complex up-front design instead of evolving a simple design into optimal design.
3. Starting programming around interesting technical problems forgetting what is important for a customer.
4. Building frameworks and system-oriented code before writing code for immediate customer problems.

For a example, some teams could start with this picture in mind stretching customer requirements to fit it (even for relatively simple applications).

source
2. Organize and evolve code around domain concepts and customer ideas
Make system design reflecting customer domain and problem space. Keep customer ideas and system implementation connected and synchronized all the time. This is as important as refactoring practice for effective creation of the software system.
Domain-driven design is an excellent way to build software systems. It puts emphasis on ubiquitous language, distilled domain knowledge and shared domain models that drive software development.

3. Think as a customer
What we see depends mainly on what we look for – John Lubbock
A programmer often considers programming as an opportunity for solving interesting technical problems. Often a programmer even doesn’t understand what customers really want. There are 2 consequences. First, the programmer is not interested in simpler solutions focused on the customer problem. Second, the programmer is missing coherent view on the customer problems and purpose of the software system. Over-engineered, complex and misaligned code is one of the most serious problems in software development. An army of business analysts, stressed project managers and smart customer proxies will not force indifferent and ignorant programmers to write code in the best interest of customer.
The programmer should constantly ask himself: “Will this solution work for my customer?”. To give correct answer on this simple question the programmer should understand a lot – customer language, needs, problems, a big picture and business purpose.

Certainly, it is difficult to think as a customer if you are at the bottom of software creation chain:

• separated from a customer by project and product managers, architects, designers, consultants, experts and panels of stakeholders
• pressed by the heavy process, rigid organization structure and inability to make decisions
• fed with over-processed, distorted and disjointed doses of information coming down through long chain

Beautiful software ideas can turn into monster applications – unusable and disconnected from real needs. Did you feel (as a user) frustrated by complex confusing and irrational program screens and logic? Yes, we, programmers, can easily produce bad stuff if we cannot think as people who need our systems.

4. Sketch programming ideas with unit tests.
Write unit tests first. Any programmer should understand what he is trying to solve with the new code and how it will be used – possible input, expected output and public interface. A programmer should think about behavior in isolation, under border conditions or within context of existing code. An unit test is an excellent place to sketch nonexistent code, play with it without compilation and see if it is convenient and makes sense. Once the programmer likes sketched ideas in unit tests, he just need to build simple implementation to satisfy these unit tests. The process of writing unit tests focuses a programmer on solving customer problems, understanding code intent and making optimal design decisions. As an additional side effect, the programmer receives comprehensive suite of automated regression tests and executable specifications.

For a example,

CustomerDB.Save(TEST_USER, TEST_PWD)
Customer customer = CustomerDB.Login(TEST_USER, TEST_PWD);
Assert.IsNotNull(customer, "customer should be logged in");
customer.Purchase(TEST_PRODUCT);
List<Order> orders = OrderDB.LoadFor(customer) or maybe... customer.Orders //what is better?
Assert.AreEqual(1, orders.Count, "count");

There are many design decision are made in this unit test without writing any production code. And now programmer can concentrate on implemention without throes of creation.

5. Eliminate duplication
Simplification is ultimate sophistication – da Vinci
Elimination of duplication is the moving force of evolutionary design. This is straightforward and powerful method to grow an optimal system. Duplication in the code tells that you need better design. Many interesting design ideas are born from the simple need to eliminate duplication in the growing system. These ideas are based on deeper understanding and gained experience as oppose to speculative up-front design ideas.
The main goals of removing duplication are

• reduce code
• simplify solution
• make the system more manageable for programmers’ minds.

Some sources of duplication:

• copy and paste – acceptable only temporary
• similar logic different in details – generalization is required
• another implementation for the same problem – better team communication and reuse are required

Master the core skill of good evolutionary designers – how to detect and eliminate duplication.
The knowledge of design patterns is the most useful on this step. The initial simple solution rarely requires design patterns. But as a system grows, design patterns can effectively and elegantly solve new complex design problems.
Example of elimination of high level duplication with Template method design pattern
This code was used in many data access classes almost the same way, but with different inner steps

        using (SqlConnection conn = NewSqlConnection())

        {

            SqlCommand cmd = conn.CreateCommand();

            cmd.CommandType = CommandType.StoredProcedure;

            cmd.CommandText = "SearchCustomers";

            cmd.Parameters.Add(new SqlParameter("@city", city));

            ... //long list of parameters

            conn.Open();

            try

            {

                using (SqlDataReader reader = cmd.ExecuteReader())

                {

                    foreach (DbDataRecord record in reader)

                        customers.Add(new Customer(record["id"], record["name"]))

                }

            }

            catch (Exception e)

            {

                //error handling logic here

            }

        }

The common logic could be moved into one template method with placeholders for different steps:

List<Customer> customers =  LoadRecordsFromSP<Customer>("SearchCustomers",

    (SqlCommand cmd) => {

         cmd.Parameters.Add(new SqlParameter("@city", city));

         ...     //long list of parameters

    },

    (DbDataRecord record) => { new Customer(record["id"], record["name"]); }

);

6. Reduce code smells
Write code that doesn’t smell. Train your sense of smell for code and design problems. They will provide hints where code is probably bad.
These are common bad smells from a long list

• Duplicated code
• Large method
• Large class
• Long parameter list
• Dead Code
• Over-generalized code
• Lazy class

7. Use design patterns and abstractions to make code simpler
Learn design patterns. At some level of complexity they will provide excellent solutions and simplify design.
However, wrong abstractions and over used design patterns will make code more complex and less manageable. So, apply them carefully.

8. Decouple and isolate components
Only talk to your immediate friends; Don’t talk to strangers – Law of Demeter.
Reduce connections between programming units – subsystems, modules and objects – as much as possible. More relations between components make software system more complex and rigid. Decoupled and isolated system components have significant benefits:

• better understanding – easier to keep in memory and make sense for less states and relations
• increased reuse – components with minimal well-defined public interface and relations less dependent on context and can be used in many scenarios
• decreased breakdowns – caused by unpredictable run-time combination of relations
• effective testing – highly connected system require much more testing beyond isolated testing of components

9. Keep related code together
Put related code to the same package. It will enhance

• discovery - easier to find and reuse
• design - allows isolation, minimal interactions and consistent structure for related components
• maintenance and testing - reduce effect of new changes to fewer packages and system components.

There are two package approaches:

• Feature – code related to the same problem placed together
• Application layers – code for similar programming concepts

Benefits of packaging:

• Feature: related code in one place, changes together and form self-sufficient granule for reuse and release
• Application: separation of concerns, logical layering and reduced dependencies,  consistent global structure

Composing the program – Putting Things Together

1. Understand what customer wants and form intent.
2. Translate customer ideas into programming concepts (technology, platform and language, UI, libraries, components)
3. Make them simple, minimal and clear with pragmatic approach.
4. Organize code around domain concepts and customer needs; use domain-driven design.
5. Describe UI ideas with paper sketches
6. Specify program ideas with automated unit tests.
7. Merge UI and program ideas
8. Package related ideas together, isolate subsystems and components
9. Express ideas in code and UI screens
10. Remove duplication and bad code smells; improve design; consider using design patterns
11. Evolve and refactor common code into framework; emerge layers and abstractions for similar concepts

I was always against Copy and Paste programming. I condemned people who use this practice as discrediting the honorable profession of The Programmer. This evil activity inflates application code without adding anything useful. This shameful practice causes various complications – more code, more things to remember, needless complexity, and difficulty to manage and synchronize related changes in the future. But at some point, I admitted to myself – things are not so black and white. I do Copy and Paste in my code, it is difficult to resist this seductive activity.

The only indulgence is that I’m very keen to eliminate bad effects of Copy and Paste before finishing my task.

I use two main approaches for writing new code. The first is Top-Down – I have clear design ideas how I should program and I know what classes and components to use. I go ahead and write code from scratch using my memory and power of intellect :). There are few problems with this approach. Unfortunately, my memory is not so great and knowledge is not so vast. Regularly I have new tasks that I don’t know or don’t remember how to program. Sometimes I don’t get immediately what other people programmed without debugging. And often I want to save time and avoid re-implementing the same idea again. And that is why I also use the second approach – Bottom-Up. I copy and paste code first. And after I figure out how this code works, how to use, fit and improve it.

When is Copy and Paste approach useful?

Shamelessly, I can come up with few cases:
1. Work by example – you found new code through search and want to  use it immediately to solve your problems.
2. Quick start – you copy code (including your old code) to save time.
3. Test in the new context – you want to quickly assess if code will be correctly running in the context of your application
4. Too hard to reuse – if you cannot reuse, just copy code. For example, you don’t have time to build common framework with separate application or another team owns code and don’t want you to touch and spoil it.
5. Series of experiments – you try to find the best way to solve a problem and copy code to try if it solves a problem well.

How to eliminate bad effects of Copy and Paste

Now I’m going to return to canonical view on evilness of Copy and Paste. Even if you have legitimate reason to use Copy and Paste, it is acceptable only on temporary basis before you commit your code or your team granted absolution (probably for very short time) :).
Therefore, after committing Copy and Paste scandalous act, do following:
1. Understand what have you done - any code that you don’t understand is ineffective for future use, and potentially excessive, wasteful and dangerous.
2. Clean, minimize and shape – remove all useless, over-engineered and questionable pieces; consider yourself as a jeweler, who is converting rough crystals in beautiful diamonds that fit well into solid code and concepts in your program.
3. Refactor and Integrate – use refactoring techniques to integrate new cleaned code into existing application: improve design, simplify and increase power of your code.
4. Learn - the best outcome of this sinful activity is gained knowledge and experience that will make Copy and Paste unnecessary next time.

With age and experience of harsh programming reality, I’m becoming less purist and more pragmatic. I have started to accept everything what makes me effective. The only rule – I shouldn’t be ashamed of my code after the finish… even if I did Copy and Paste couple times ;)

What do you think?